1111830991 318285 stats PDF

Preview

Summary

exercices for stats behavioural stats and marking research...

Description

Chapter 8: Introduction to Hypothesis Testing Chapter Outline 8.1 The Logic of Hypothesis Testing The Four Steps of a Hypothesis Test Step l: State the Hypotheses Step 2: Set the Criteria for a Decision Step 3: Collect Data and Compute Sample Statistics Step 4: Make a Decision A Closer Look at the z-Score Statistic (as a recipe and as a ratio) 8.2 Uncertainty and Errors in Hypothesis Testing Type I Errors Type II Errors Selecting an Alpha Level 8.3 An Example of a Hypothesis Test In the Literature - Reporting the Results of the Statistical Test Factors that Influence a Hypothesis Test Assumptions for Hypothesis Tests with z-Scores 8.4 Directional (One-Tailed) Hypothesis Tests The Hypotheses for a Directional Test The Critical Region for Directional Tests Comparison of One-Tailed Versus Two-Tailed Tests 8.5 Concerns about Hypothesis Testing: Measuring Effect Size Measuring Effect Size 8.7 Statistical Power Power and Effect Size Other Factors that Affect Power

Learning Objectives and Chapter Summary 1. Students should understand the logic of hypothesis testing. In the generic hypothesis testing situation, a sample is selected from a population, a treatment is administered to the sample, and the individuals in the sample are measured. If the sample mean is noticeably different from the original population mean, then we have evidence that the treatment had an effect. Instructor Notes – Chapter 8 – page 106

2. Students should be able to state the hypotheses and locate the critical region. A hypothesis test always begins with a null hypothesis stating that the treatment has no effect (no change, no difference, no relationship, etc.). The next step is to determine what kind of sample data would be reasonable if this hypothesis is true, and what kind of sample data would be very unlikely. The term “very unlikely” is defined by the alpha level for the test. The critical region consists of the set of sample outcomes that would be very unlikely to occur if the null hypothesis is true. If the research study produces a sample in the critical region, we conclude that the sample data are not consistent with the null hypothesis, and we reject the null hypothesis. 3. Students should be able to conduct a hypothesis test using a z-score statistic and make a statistical decision. As noted earlier, if the mean for the treated sample is noticeably different from the mean for the original population, then we conclude that the treatment had an effect. The problem however, is that the difference between the sample mean and the population mean may be simply chance (sampling error). The goal for the hypothesis test is to rule out chance as a plausible explanation for the mean difference. To accomplish this goal, we first calculate how much difference is reasonable to expect between M and µ if there is no treatment effect (the standard error). Then, we compare the actual obtained difference with this value. The z-score statistic makes this comparison. M  μ z = ───── σM

=

Actual mean difference ────────────────────── standard difference between M and µ

A large value for the z-statistic (as defined by the critical region) means that we can conclude that the obtained mean difference is more than can be explained by chance.

4. Students should be able to define and differentiate Type I and Type II errors. A hypothesis test uses limited information from a sample to make a general conclusion about a population. It is always possible that the sample information is misleading, and leads to an incorrect conclusion. Sometimes the sample appears to show evidence of a treatment effect, when in fact the treatment has no effect. In this case, the researcher falsely concludes that the treatment has an effect - a Type I error. It is also possible that the treated sample does not appear to be noticeably different from the original population even though the treatment did have an effect. In this case, the researcher falsely concludes that the treatment does not have a significant effect - a Type II error.

Instructor Notes – Chapter 8 – page 107

5. Students should understand the purpose of measuring effect size and power, and they should be able to compute Cohen’s d. A hypothesis test determines whether the mean difference obtained in a research study is greater than is expected simply by chance (sampling error). The standard error is used to determine how much difference is reasonable to expect. However, in some cases (especially with large samples) the standard error can be very small. In these cases, a tiny mean difference may be enough to be statistically significant. Thus, concluding that a treatment effect is “significant” does not tell you anything about the actual size of the effect, and does not imply that the effect is large. To gain information about the size of the treatment effect, it is recommended that researcher also provide a report of effect size. Cohen’s d provides a measure of effect size. Cohen’s d is computed by dividing the obtained mean difference by the standard deviation. Thus, a d value of 0.50 indicates that the difference between the sample mean (after treatment) and the original population mean (before treatment) is equal to one-half of a standard deviation. Students should understand the concept of power: the probability that the hypothesis test will reject the null hypothesis when there really is a treatment effect. As the size of the treatment effect increases, the power of the test also increases. 6. Students should be able to incorporate a directional prediction into the hypothesis test and conduct a directional (one-tailed) test. If the expected treatment effect is an increase in scores, the null hypothesis for a directional test simply states that there is no increase (no effect). Sample data that show an increase (large values in the right hand tail) tend to refute this null hypothesis, and thus the critical region consists entirely of values in one tail of the distribution.

Other Lecture Suggestions 1. The general purpose for a hypothesis test can be demonstrated using Figure 1.2 which introduces the concept of sampling error, and shows that there will be some difference between a sample mean and the population mean even when the sample is not receiving any treatment. Thus, M = μ just by chance. In Chapter 8 we are administering a treatment to the sample and we would like to conclude that the difference between M and μ is caused by the treatment. To justify this conclusion, however, we must demonstrate that the obtained mean difference is significantly more than can be explained by chance or sampling error (as in Figure 1.2). This is the job for a hypothesis test. In simple terms, the goal for a hypothesis test is to rule out chance (random, unsystematic factors) as a plausible explanation for the research results.

Instructor Notes – Chapter 8 – page 108

Exam Items for Chapter 8

Multiple-Choice Questions 1. Which of the following accurately describes a hypothesis test? a. a descriptive technique that allows researchers to describe a sample b. a descriptive technique that allows researchers to describe a population c. an inferential technique that uses the data from a sample to draw inferences about a population d. an inferential technique that uses information about a population to make predictions about a sample 2. What is measured by the numerator of the z-score test statistic? a. the average distance between M and µ that would be expected if H0 was true b. the actual distance between M and µ c. the position of the sample mean relative to the critical region d. whether or not there is a significant difference between M and µ 3. What is measured by the denominator of the z-score test statistic? a. the average distance between M and µ that would be expected if H0 was true b. the actual distance between M and µ c. the position of the sample mean relative to the critical region d. whether or not there is a significant difference between M and µ 4. (www) Which of the following accurately describes the critical region? a. outcomes with a very low probability if the null hypothesis is true b. outcomes with a high probability if the null hypothesis is true c. outcomes with a very low probability whether or not the null hypothesis is true d. outcomes with a high probability whether or not the null hypothesis is true 5. If α is held constant at .05, what is the relationship between sample size, the critical region, and the risk of a Type I error? a. As sample size increases, the critical region expands and the risk of a Type I error increases. b. As sample size increases, the critical region shrinks and the risk of a Type I error increases. c. As sample size increases, the critical region expands and the risk of a Type I error decreases. d. There is no relationship between sample size, the critical region, and the risk of a Type I error. Instructor Notes – Chapter 8 – page 109

6. What is the relationship between the alpha level, the size of the critical region, and the risk of a Type I error? a. As the alpha level increases, the size of the critical region increases and the risk of a Type I error increases. b. As the alpha level increases, the size of the critical region increases and the risk of a Type I error decreases. c. As the alpha level increases, the size of the critical region decreases and the risk of a Type I error increases. d. As the alpha level increases, the size of the critical region decreases and the risk of a Type I error decreases. 7. Even if a treatment has no effect it is still possible to obtain an extreme sample mean that is very different from the population mean. What outcome is likely if this happens? a. reject H0 and make a Type I error b. correctly reject H0 c. fail to reject H0 and make a Type II error d. correctly fail to reject H0 8. Even if a treatment has an effect it is still possible to obtain a sample mean that is very similar to the original population mean. What outcome is likely if this happens? a. reject H0 and make a Type I error b. correctly reject H0 c. fail to reject H0 and make a Type II error d. correctly fail to reject H0 9. Which of the following correctly describes the effect of increasing the alpha level (for example from .01 to .05)? a. increase the likelihood of rejecting H0 and increase the risk of a Type I error b. decrease the likelihood of rejecting H0 and increase the risk of a Type I error c. increase the likelihood of rejecting H0 and decrease the risk of a Type I error d. decrease the likelihood of rejecting H0 and decrease the risk of a Type I error 10. Which combination of factors will increase the chances of rejecting the null hypothesis? a. a large standard error and a large alpha level b. a large standard error and a small alpha level c. a small standard error and a large alpha level d. a small standard error and a small alpha level

Instructor Notes – Chapter 8 – page 110

11. By selecting a larger alpha level, a researcher is ______. a. attempting to make it easier to reject H0 b. better able to detect a treatment effect c. increasing the risk of a Type I error d. All of the above 12. Increasing the alpha level (for example from α = .01 to α = .05) _____. a. increases the probability of a Type I error b. increases the size of the critical region c. increases the probability that the sample will fall into the critical region d. All of the other options are results of increasing alpha. 13. Which of the following is directly addressed by the null hypothesis? a. the population before treatment. b. the population after treatment. c. the sample before treatment. d. the sample after treatment. 14. (www) Which of the following accurately describes the effect of increasing the sample size? a. increases the standard error and has no effect on the risk of a Type I error b. decreases the standard error and has no effect on the risk of a Type I error c. increases the risk of a Type I error and has no effect on the standard error d. decreases the risk of a Type I error and has no effect on the standard error 15. Which of the following accurately describes the effect of increasing the alpha level? a. increases the standard error and has no effect on the risk of a Type I error b. decreases the standard error and has no effect on the risk of a Type I error c. increases the risk of a Type I error and has no effect on the standard error d. decreases the risk of a Type I error and has no effect on the standard error 16. (www) Which of the following is an accurate definition of a Type I error? a. rejecting a false null hypothesis b. rejecting a true null hypothesis c. failing to reject a false null hypothesis d. failing to reject a true null hypothesis 17. Which of the following is an accurate definition of a Type II error? a. rejecting a false null hypothesis b. rejecting a true null hypothesis c. failing to reject a false null hypothesis d. failing to reject a true null hypothesis

Instructor Notes – Chapter 8 – page 111

18. What is the consequence of a Type I error? a. concluding that a treatment has an effect when it really does b. concluding that a treatment has no effect when it really has no effect c. concluding that a treatment has no effect when it really does d. concluding that a treatment has an effect when it really has no effect 19. (www) What is the consequence of a Type II error? a. concluding that a treatment has an effect when it really does b. concluding that a treatment has no effect when it really has no effect c. concluding that a treatment has no effect when it really does d. concluding that a treatment has an effect when it really has no effect 20. When is there a risk of a Type I error? a. whenever H0 is rejected b. whenever H1 is rejected c. whenever the decision is "fail to reject H0" d. The risk of a Type I error is independent of the decision from a hypothesis test. 21. When is there a risk of a Type II error? a. whenever H0 is rejected b. whenever H1 is rejected c. whenever the decision is "fail to reject H0" d. The risk of a Type II error is independent of the decision from a hypothesis test. 22. A two-tailed hypothesis test is being used to evaluate a treatment effect with α = .05. If the sample data produce a z-score of z = 2.24, then what is the correct decision? a. Reject the null hypothesis and conclude that the treatment has no effect. b. Reject the null hypothesis and conclude that the treatment has an effect. c. Fail to reject the null hypothesis and conclude that the treatment has no effect. d. Fail to reject the null hypothesis and conclude that the treatment has an effect. 23. The critical boundaries for a hypothesis test are z = +1.96 and 1.96. If the z-score for the sample data is z = 1.90, then what is the correct statistical decision? a. Fail to reject H1. b. Fail to reject H0. c. Reject H1. d. Reject H0.

Instructor Notes – Chapter 8 – page 112

24. (www) A researcher conducts a hypothesis test to evaluate the effect of a treatment. The hypothesis test produces a z-score of z = 2.37. Assuming that the researcher is using a two-tailed test, what decision should be made? a. The researcher should reject the null hypothesis with α = .05 but not with α = .01. b. The researcher should reject the null hypothesis with either α = .05 or α = .01. c. The researcher should fail to reject H0 with either α = .05 or α = .01. d. Cannot answer without additional information. 25. A researcher conducts a hypothesis test to evaluate the effect of a treatment that is expected to increase scores. The hypothesis test produces a z-score of z = 2.37. If the researcher is using a one-tailed test, what is the correct statistical decision? a. Reject the null hypothesis with α = .05 but not with α = .01. b. Reject the null hypothesis with either α = .05 or α = .01. c. Fail to reject the null hypothesis with either α = .05 or α = .01. d. cannot answer without additional information 26. A researcher administers a treatment to a sample of participants selected from a population with µ = 80. If a hypothesis test is used to evaluate the effect of the treatment, which combination of factors is most likely to result in rejecting the null hypothesis? a. a sample mean near 80 for a small sample b. a sample mean near 80 for a large sample c. a sample mean much different than 80 for a small sample d. a sample mean much different than 80 for a large sample 27. A researcher administers a treatment to a sample of participants selected from a population with µ = 80. If a hypothesis test is used to evaluate the effect of the treatment, which combination of factors is most likely to result in rejecting the null hypothesis? a. a sample mean near 80 with α = .05 b. a sample mean near 80 with α = .01 c. a sample mean much different than 80 with α = .05 d. a sample mean much different than 80 with α = .01 28. A researcher administers a treatment to a sample of participants selected from a population with µ = 80. If the researcher obtains a sample mean of M = 88, which combination of factors is most likely to result in rejecting the null hypothesis? a. σ = 5 and α = .01 b. σ = 5 and α = .05 c. σ = 10 and α = .01 d. σ = 10 and α = .05

Instructor Notes – Chapter 8 – page 113

29. (www) A researcher administers a treatment to a sample of participants selected from a population with µ = 80. If the researcher obtains a sample mean of M = 88, which combination of factors is most likely to result in rejecting the null hypothesis? a. σ = 5 and n = 25 b. σ = 5 and n = 50 c. σ = 10 and n = 25 d. σ = 10 and n = 50 30. A sample of n = 25 individuals is selected from a population with µ = 80 and a treatment is administered to the sample. What is expected if the treatment has no effect? a. The sample mean should be very different from 80 and should lead you to reject the null hypothesis. b. The sample mean should be very different from 80 and should lead you to fail to reject the null hypothesis. c. The sample mean should be close to 80 and should lead you to reject the null hypothesis. d. The sample mean should be close 80 and should lead you to fail to reject the null hypothesis. 31. If a treatment has a very small effect, then what is a likely outcome for a hypothesis test evaluating the treatment? a. a Type I error b. a Type II error c. correctly reject the null hypothesis d. correctly fail to reject the null hypothesis 32. (www) A researcher is conducting an experiment to evaluate a treatment that is expected to increase the scores for individuals in a population which is known to have a mean of μ = 80. The results will be examined using a one-tailed hypothesis test. Which of the following is the correct statement of the null hypothesis? a. μ > 80 b. μ > 80 c. μ < 80 d. μ < 80 33. A researcher expects a treatment to produce an increase in the population mean. The treatment is evaluated using a one tailed hypothesis test, and the test produces z = +1.85. Based on this result, what is the correct statistical decision? a. The researcher should reject the null hypothesis with α = .05 but not with α = .01. b. The researcher should reject the null hypothesis with either α = .05 or α = .01. c. The researcher should fail to reject H0 with either α = .05 or α = .01. d. cannot answer without additional information Instructor Notes – Chapter 8 – page 114

34. (www) A researcher is conducting an experiment to evaluate a treatment that is expected to increase the scores for individuals in a population. If the researcher uses a one-tailed test with  = .01, then which of the following correctly identifies the critical region? a. z > 2.33 b. z > 2.58 c. z < 2.33 d. z < 2.58 35. Under what circumstances can a very small treatment effect can still be significant? a. if the sample size (n) is very large b. if the sample standard deviation (σ) is very large c. if the standard error of M (σ M) is very large d. All of the other factors are likely to produce a significant result. 36. (www) A sample of n = 16 individuals is selected from a population with μ = 60 and σ = 6 and a treatment is administered to the sample. After treatment, the sample mean is M = 63. What is the value of Cohen’s d for this sam...