Lecture note on the third lecture: Hypothesis Testing PDF

Preview

Summary

Annoted notes from the videos on zonecours on what is hypothesis test with exercises included with the solutions explained great to prepare the exam...

Description

Note-taking in connection with the capsule

Hypothesis Testing Definition: A hypothesis test is a statistical procedure designed to determine whether a statement, called the nodal hypothesis, is plausible or whether it should instead be rejected.

Steps of a hypothesis test: 1. Formulate the hypotheses of the test. H0 : null hypothesis à ex: μsubscribers = μnon-subscribers

H1 : alternative hypothesis à ex: μsubscribers ≠ μnon-subscribers

(Complete the text below using the appropriate words from: opposites, similar, parameters, samples, populations, estimators.) The two hypotheses are always H0 and H1. The hypotheses always relate to the sample(s) under study. Very often, they relate more specifically to one or more unknown parameters, as in the example where μsubscribers and μnon-subscribers are compared.

2. Collect a random sample and use the sample data to calculate the

experimental threshold (p-value) A difference between what is observed in our samples and what is expected under H0 does not necessarily indicate that H0 should be rejected, as the sampling process carries a risk of estimation error.

Test Statistic: Distance between what is expected under the H0 hypothesis and what is observed in our sample(s), corrected for the level of estimation of error committed.

p-value: The probability of observing sample data as far or further from the H0 hypothesis than what we have observed in practice, assuming that the H0 hypothesis is true.

what is observed in our sample(s)

what is expected under

Small “distance”

large test statistic

large p-value

what is observed in our sample(s)

what is expected under

Large “distance”

Match correctly

small test statistic

small / large

small p-value

small / large

Data surprising

Data plausible

If is true

if is true

Argument in favour of the H0

Argument in favour of the H1

hypothesis

hypothesis

surprising/ plausible

/

3. Make a decision based on the p-value. a. Determine what a small or a large p-value is by fixing the significance level of the test denoted α. Typical values are 10%, 5% or 1%. b. Make a decision:

p-value

< α

reject H0

p-value

≥ α

do not reject H0

Exercises 1. A business sells sporting goods online and advertises on their website that delivery time is less than 48 hours, on average. In the hope of validating this information by means of a hypothesis test, the following hypotheses are considered: hours

hours

a) What does represent in this context? The time delivery b) Which of the two hypotheses, if true, would corroborate what the website displays? It would be H1. Data from a sample of 25 clients is considered. These show an average delivery time of 44.7 hours. c) Can we conclude that is necessarily true? No because of the estimation error. Still for the sample data, the p-value for a test on and is established to be 0.002. d) Interpret the p-value of 0.002. e) Does the data corroborate the information posted on the website? Is the conclusion the same regardless of the significance level selected from the usual significance levels?

2. In the 1-605 course, a series of five Schlitz beer commercials, surrounding the 15th Super Bowl, was discussed. Remember that each advertisement consisted of a blind taste test, live on television, carried out with 100 fans of a competing beer, pitting Schlitz against this competing beer. This campaign was actually based on the assumption that Schlitz tasted too similar to the beers it was compared to for tasters to be able to distinguish between beers, such that the probability that a given taster would choose Schlitz was 50%. Under this assumption, one would therefore expect about 50 fans of the competing beer to choose the Schlitz out of the 100 fans participating in the taste test. Assuming this hypothesis is true, you have calculated that the probability of at least four of the five ads being effective is 99.7% (you used the binomial law for this). This means that the probability that three or less ads will be effective is 0.3%.

Note that an advertisement is considered to be effective once at least 40 of the 100 opposing beer fans choose Schlitz (remember that Schlitz expects a result of around 50). In retrospect, the results that were obtained during the five blind tests are as follows: Budweiser 46

50

Miller 38

37

Michelo b 50

Schlitz performed poorly: only three of the five advertisements were found to be effective. A posteriori, are the results of the five commercials compatible with the hypothesis that the other beers taste the same as the Schlitz? We want to answer this question by means of a hypothesis test, the hypotheses of which are: the probability that a given taster will choose Schlitz is 50%; the probability that a taster will choose Schlitz is less than 50%.

a) What is the p-value of the test? b) Conclude in the context of the problem, using the 1% significance level.

Solutions 1. a) the average delivery time for purchases on this website, in hours. b) The hypothesis corresponds to what is displayed on the website (the delivery time is less than 48 hours, on average). c) Not necessarily. Indeed, the sample mean of 44.7 hours is only an estimate of. There is (probably) an estimation error between and this estimate. It is possible that is actually above 48 hours, and the estimate of 44.7 hours is simply underestimating its value. d) If the average delivery time is actually 48 hours or more (that is, if is true), then the probability that a sample of 25 customers leads to an average delivery time as small as or smaller than 44.7 hours (i.e. as far from ) is only 0.002. It is therefore very unlikely. e) The conclusion is the same regardless of the significance level selected from the usual levels:  , so we reject . At the 1% significance level, the data corroborates what the website says.  , so we reject . At the 5% significance level, the data corroborates what the website says.  , so we reject . At the 10% significance level, the data corroborates what the website says. 2. a) Assuming the hypothesis, according to which all beers taste the same, we calculated that the probability of obtaining a result as bad or even worse than that obtained, i.e. having only 0, 1, 2 or 3 of the five effective ads was 0.3%. Thus, the probability of having a result as extreme or more than that observed in reality assuming to be true is 0.3%. By definition, the p-value is therefore 0.3%. Since the p-value of 0.3% is very small, it was very unlikely to achieve such poor results assuming that all beers taste the same. This calls into question this hypothesis. b) We reject since . At the 1% significance level, the results of the five ads show that the ad designers were wrong to think that Schlitz tasted the same as other beers....