Lecture 9- Beyong the Null Hypothesis PDF

Preview

Summary

Outline: Research or Alternative hypotheses, null hypothesis, alternative hypothesis, one-tailed versus two-tailed estimates, degrees of freedom, power and effect, alpha and beta, etc....

Description

Lecture 9: chpt 9- Beyond the Null Hypothesis Learning Objectives • In this chapter, we’ll study • Research or alternative hypothesis • One-tailed and Two-Tailed Test Scenarios • Power and Effect • Some Other Things You Should Know Research or Alternative Hypotheses • Null Hypothesis: it is the hypothesis that is tested or a stated non-relationship between two or more variables. • Using the previous example of drug awareness, we can say alternatively: • H1: Drug awareness scores after exposure to the film will be higher than prior to exposure to the film; • H2: Drug awareness scores after exposure to the film will be lower than prior to exposure to the film; • H3: Drug awareness scores after exposure to the film will be different. • H1 and H2 are directional alternative hypotheses (one-tailed test) while H3 is nondirectional or two-tailed test. • What is an alternative or research hypothesis? • An alternative or research hypothesis is a hypothesis that stands in contrast (or in opposition) to the null hypothesis • What is a directional hypothesis and what is a non-directional hypothesis? • A directional hypothesis specifies that the nature or direction of a hypothesized difference. It asserts that there will be a difference in a particular direction. • A non-directional hypothesis does not specify the nature or direction of an expected difference. It simply asserts that a difference will be present. • A null hypothesis is the hypothesis that is tested or a stated non-relationship between two or more variables. • A research or alternative hypothesis is a stated relationship between two or more variables • Hypotheses must be mutually exclusive, exhaustive, and falsifiable. • Hypotheses stating causal relationships should indicate direction of causality (independent variables and dependent variables). Hypothesis Summary 1. Null Hypothesis (H0) • There is no difference (or the difference is caused by random chance or sampling error). • The H0 always states there is “no significant difference.” 2. Alternative hypothesis (H1) • The difference is real (two-tailed) • One group is more than the other (one-tailed) • One group is less than the other (one-tailed) • H1 always contradicts the H0.

One (and only one) of these explanations must be true. One-Tailed Versus Two-Tailed Estimates • One-tailed tests determine how likely it is that an observation is above or below a specific threshold value. They are directional and they bet the probability on one side of the distribution. • Two-tailed tests are non-directional, and they emphasize the difference, but not emphasize the direction of the relationship. They bet the probability on both sides of the tails.

Degrees of Freedom • The t-distribution is a family of distributions, all of which are a function of sample size. • To determine the degree of freedom, we take the sample size – 1 or (n - 1). • This number is known as the degrees of freedom. • t-value converges upon those for z as degrees of freedom increase. After df = 120, they are identical. What is two-tailed test scenario? • A two-tailed test scenario is a research situation in which the researcher is looking for an extreme difference that could be located at either end of the distribution. • When it is appropriate? • A two-tailed test is appropriate when the alternative or research hypothesis is nondirectional. • For example, if we have found, through the univariate analysis of 11 respondents respectively, that the mean scores of religious participation (0 to 100) for urban residents is 68.45 and for rural residents is 77.27. The estimate standard error of the difference of means is 4.42. Two-tailed test scenario • H0: There is no difference between the means of rural and urban residents in their religious participation (the means are equal). • H1: There is a difference between the mean scores of rural and urban residents (note: is this hypothesis directional?) • Set the level of significance at .05 • t = (Xrural – Xurban) - 0 / sxrural - xurban • = (77.27 – 68.45) / 4.42 = 1.995 • df = (n1 + n2) – 2 = (11 + 11) – 2 = 20 • With 20 degrees of freedom, the critical value in Appendix B is 2.086.

•

Since our calculated t-value (1.995) is smaller than the critical t-value, we fail to reject the null hypothesis that there is no difference. We conclude that there is no statistical difference between rural and urban residents in their religious participation.

What is a one-tailed test scenario, and when is it appropriate? • A one-tailed test scenario is a research situation in which the research is looking for an extreme difference that is located on only one side of the distribution. • A one-tailed test is appropriate when the alternative or research hypothesis is directional. • This directional expectation comes from the researcher’s observation or from a theory. The difference is conceptual, and the consequence is huge. One-tailed test scenario • H0: There is no difference between the means of rural and urban residents in their religious participation (the means are equal). • H1: The rural residents have a higher mean score of religious participation than the urban residents (one-tailed test). • Working at the .05 level of significance with 20 degrees of freedom, the critical value for a one-tailed test (Appendix C) is 1.725. • Our calculated t = 1.9955 • Since our calculated t-value is larger than the critical t-value, we reject the null hypothesis that there is no difference and accept the alternative hypothesis. We conclude that rural residents are indeed having a significantly higher mean score of religious participation than urban residents. • Also called the theoretical standard deviation Power and Effect • First, take a moment to think about the difference between Type I and Type II errors. • Type I error: reject null hypothesis (H0) even though it is true (mistakenly think you have a relationship—“false knowledge”). • Type II error: fail to reject H0 even though it is false (mistakenly think you do not have a relationship—“unrecognized relationship”). Rejecting the null hypothesis when we should have retained it is Type I error. In other words, p = .05 that we have committed Type I error. Conversely, retaining the null hypothesis when we should have rejected it is Type II error. Decision Fail to reject (Retain) Reject H0 is true Correct decision Type I error (p = α) Reality H0 is false Type II error (p = β) Correct decision Alpha (α) and beta (β) • Alpha (α) equals the maximum probability of incorrectly rejecting the null hypothesis that we are willing to risk – that is, the probability of rejecting a true H0 (Type I error).

• • • •

Beta (β) refers to the probability of incorrectly failing to reject the null hypothesis – that is, of failing to reject a false H0 (Type II error). α = probability of Type I error (false knowledge) β = probability of a Type II error (unrecognized relationship) The smaller we set alpha, the larger beta will be. There is a trade-off. The relationship is complex, depending on sample size and how different the true population mean is from the mean specified by the null hypothesis.

Power and effect • Always remember that there are two possibilities with respect to the null hypothesis. • What is the power of a test? The power of a statistical test is the ability of the test to reject a false null hypothesis. The alpha (α) value is the level of probability at which the null hypothesis can be rejected with confidence. In our research, we must choose different tests carefully and use the more powerful test for our research purpose. • Effect is the change in measurement that is attributable to a treatment condition or stimulus of some sort. In designing our survey or our experiment, we must pay attention to the measurement and their nuanced difference under different scenarios. Some other things you should know • The material on power and effect has particular relevance for those in the field of experimental psychology. These issues are typically of minimal consequence in fields such as sociology or political science, which rely on large-scale survey data. • There are many situations in which the choice between one-tailed and two-tailed tests is not available. • Be aware of how the logic of one-tailed and two-tailed tests is dealt with when working with z and the Table of Areas Under the Normal Curve. Statement on statistical significance and P-Values (American Statistical Association, 2016) • 1. P-values can indicate how incompatible the data are with a specified statistical model. • 2. P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone. • 3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold. • 4. Proper inference requires full reporting and transparency. • 5. A p-value, or statistical significance, does not measure the size of an effect or the importance of a result. • 6. By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis....