Stats Notes - Professor Dale Glaser PDF

Preview

Summary

Professor Dale Glaser...

Description

1

Chapter 8: Hypothesis Testing Hypothesis: a tentative assumption (that is testable) made in order to draw out and test its logical or empirical consequences Hypothesis test: a statistical method that uses sample data to evaluate a hypothesis about a population - Goal: rule out chance (sampling error) as a plausible explanation for the results from a research study - Purpose: to decide between 2 explanations: the difference can be explained by sampling error… or the difference is too large to be explained by sampling error… there was a treatment effect - Steps: 1. State the hypothesis about the population 2. Use the hypothesis to predict the characteristics the sample should have a. Alpha level… critical region 3. Obtain a sample from the population 4. Compare data with the hypothesis prediction a. Compute the test statistic; (z-score) forms a ratio comparing the obtained difference between the sample mean and the hypothesized population mean vs. the amount of difference we would expect without any treatment effect In a hypothesis test with z-scores, we have a formula for z-scores but we do not know the value for the population mean Type 1 error: when a researcher rejects the null that is actually true. - Concludes that a treatment does have an effect when it does not - Occurs when a researcher unknowingly obtains an extreme nonrepresentative sample - Alpha level: the probability of obtaining sample data in the critical region even though the null is true Type 2 error: when a researcher fails to reject the null that is actually false - Means that the hypothesis test has failed to detect a real treatment effect - Occurs when the sample mean is not in the critical region and often happens when there is a small effect - Usually has less consequences than type 1 - It is impossible to determine a single, exact probability for a type 2 error - Represented by beta Critical region boundaries: - a= .05 +- 1.96 - a= .01 +- 2.58 - a= .001 +- 3.30 Factors that Influence a Hypothesis Test - The final decision is determined by the value obtained for the z-score statistic - Two factors determine whether the z-score will be large enough to reject - Higher variability can reduce the chances of finding a significant treatment effect - Increasing the number of scores in the sample produces a smaller standard error and a larger value for the z-score

2

Assumptions for H-tests with z - It is assumed random selection - Values must consist of independent observations (not affected by the prior occurrence) - The standard deviation for the unknown (after treatment) is assumed to be the same as it was for the population before treatment - To evaluate hypotheses with z-scores, we use the unit normal table to identify the critical region Directional Tests (one tailed test) - The statistical hypotheses specify either an increase or a decrease in the population mean but not the other - The critical region is only in one tail of the distribution, so this is commonly called a one-tailed test Comparing one and two tails - Major distinction is the criteria they use for rejecting the null - One-tailed allows you to reject the null when the difference between sample and population is relatively small provided the difference is in the specified direction - Two-tailed requires a relatively large difference independent of direction Limitations with hypothesis testing - When the null is rejected, we are actually making a strong probability statement about the sample data… not about the null hypothesis - Demonstrating a significant treatment effect does not necessarily indicate a substantial treatment effect Effect Size: a measurement of the absolute magnitude of a treatment effect, independent of the size of sample(s) being used - Cohen’s d : measures the size of the mean difference in terms of standard deviation - Small: 0.2 - Medium: 0.5 - Large: 0.8 Statistical Power: the power of a statistical test is the probability that the test will correctly reject a false null hypothesis - The probability that the test will identify a treatment effect if one really exists - Typically calculated before they actually conduct the research as a means of determining whether a research study is likely to be successful - Necessary to make assumptions about a variety of factors such as: sample size, size of treatment effect, and alpha level value - Factors that affect statistical power: - As the effect size increases, the probability of rejecting null also increases… which means the power of the test increases - Size of the sample - Reducing the alpha level reduces the power

3

Chapter 9: Introduction to the t statistic Sometimes there isn’t enough information to calculate z The estimated standard error ( σ M ) is used as an estimate of the real standard error when the value of standard deviation σ is unknown - It is computed from the sample variance or sample standard deviation and provides an estimate of the standard distance between a sample mean M and population mean μ Standard error: provides a measure of how well a sample mean approximates the population mean…. Specifically determines how much difference is reasonable to expect between a sample mean (M) and the population mean The t statistic: used to test hypotheses about an unknown population mean μ when the value of (sigma) is unknown - Formula has the same structure as the z-score formula, except the t uses the estimated standard error in the denominator Degrees of freedom: the number of scores in a sample that are independent and free to vary - Because the sample mean places a restriction on the value of one score, there are n-1 degrees of freedom for a sample with n scores The t distribution: the complete set of t values computed for every possible random sample for a specific sample size (n) or a specific degrees of freedom (df) - Approximates the shape of a normal distribution - Exact shape changes with degrees of freedom (skinnier with larger df, flatter with less) ** we use a t distribution table to find proportions for t statistics - As the value for df increases, the t distribution becomes more similar to a normal distribution Hypothesis Tests with the t statistic - We begin with a population with an unknown mean and unknown variance, often a population that has received some treatment - Null states treatment has no effect and population mean is unchanged… the null provides a specific value for the unknown population mean - The sample data provide a value for the sample mean - The variance and estimated standard error are computed from the sample data - These values are used in the formula - When the numerator > denominator we obtain a large value for t (positive or negative) and we conclude that we “reject null” - When t is small near zero, we “fail to reject” The Unknown Population

4

-

-

t can also be used in situations for which you do not have a known population mean to serve as a standard - The t test does not require prior knowledge about the population mean or population variance… you just need a null and a sample from the unknown population Steps with only a sample 1. State the hypotheses and select alpha 2. Locate the critical region a. Because the population variance is unknown, the test statistic is a t… therefore df must be found before critical region can be found on the table 3. Calculate the test statistic a. Calculate sample variance b. Use the sample variance and sample size to compute the estimated standard error c. Then use the estimated standard error to compute t 4. Make a decision about null

Assumptions of the t test: - The values in the sample must consist of independent observations - The population sampled must be normal ** the number of scores in the sample and the magnitude of the sample variance both have a large effect on the t statistic and influence the decision - A larger e. Standard error produces a smaller (closer to zero) value for t **The larger the variance, the larger the error ** the estimated standard error is inversely related to the number of scores in the sample - The larger the sample, the smaller the error - If all other factors held constant, large samples tend to produce bigger t statistics and are more likely to produce significant results ** a hypothesis simply determines whether the treatment effect is greater than “chance” which is measured by standard error - It’s possible for a very small treatment effect to be “statistically significant” so its recommended that Cohen’s d is reported also R squared: the percentage of variance accounted for by the treatment - Small: 0.01 - Medium: 0.09 - Large: 025 ** sample size has no effect on effect size!! Confidence Interval: an interval or range of values centered around a sample statistic - The sample statistic should be relatively near to the corresponding population parameter Constructing a confidence interval:

5

-

First step: select a level of confidence and look up the corresponding t values in the table This is plugged into the estimation formula (on formula page) u = M +/- t s M - The basic equation for a confidence interval.. An interval around the sample mean

Factors affecting the width of the confidence interval: - To gain more confidence in your estimate, you must increase the width of the interval…. A smaller, more precise interval has less confidence - The bigger the sample (n), the smaller the interval Steps of a One-Tailed Test: 1. State the hypotheses, and select an alpha level 2. Locate the critical region 3. Calculate the test statistic 4. Make a decision

Chapter 10: the t test for two independent samples: Independent Measures - Two sets of data could come from two completely separate groups of participants - Goal: evaluate the mean difference between two populations or two treatment conditions - Uses subscripts to differentiate the two populations - The null is expressed as a difference of means = 0 Between-subjects design - The sets of data could come from the same group of participants Estimated Standard Error of M1-M2 - The standard error is defined as a measure of the standard or average distance between a sample statistic (M1-M2) and the corresponding population parameter (u1-u2) - When the null is true, the standard error is measuring how big, on average, the sample mean difference is. In general: standard error measures how accurately a statistic represents a parameter Calculating the Estimated Standard Error: 1. Each of the 2 sample means represents its own population mean.. But in each case, there is some error 2. The amount of error associated with each sample mean is measured by the estimated standard error of M 3. For the I.M t statistic, we want to know the total amount of error involved in using two sample means to approximate two population means a. If the samples are the same size, we will find the error from each sample separately and then add the two errors together

6

b. If they are different sizes, a pooled or average estimate, that allows the bigger sample to carry more weight is used. Degrees of freedom for independent measures t statistic are determined by the df values for the two separate samples: (n1 - 1) + (n2 - 1) = n1+n2 - 2 Assumptions underlying the independent measures t formula: - The observations within each sample must be independent - The two populations from which the samples are selected must be normal - The two populations from which the samples are selected must have equal variances Homogeneity of variance: states that the 2 populations being compared must have the same varience - Most important when there is a large discrepancy between the sample sizes Hartley’s F-max test: provides a method for determining whether the homogeneity of variance assumption has been satisfied - If the population variances are the same and the sample variances are similar = good - But if the if one sample variance is more than 3 or 4 times larger = bad Steps: 1. Compute the sample variance for each of the samples 2. Select the largest and the smallest and compute a. F max = (largest)/(smallest) b. A relatively large F max indicates a large difference, while a number near 1 indicates similarity 3. Compare with critical value. If the F max is larger than the critical value, then you conclude that the variances are different and the homogeneity assumption is not valid a. To locate the critical value in the table, you need: i. k = number of seperate samples ii. df = n-1 iii. The alpha level Measuring Effect Size for the Independent Measures t : - Cohen’s d - R squared Confidence Intervals for Independent Measures: - First step: solve the t equation for the unknown parameter - Although the value for the t statistic is unknown, we can use the degrees of freedom to estimate Chapter 11: The t Test for Two Related Samples Repeated-Measures Design (or within-subject design) is one which the DV is measured 2 or more times for each individual in the sample - Researcher would like to know if there is any difference between the two treatment conditions - Main advantage: uses the same individuals in all treatment conditions … there is no risk that participants in one treatment are substantially different from participants in another

7

- The t is similar, but it is based on difference scores rather than raw scores (X values) - Difference score = D = X2 - X1 Matched-subjects design: each individual in one sample is matched with an individual in the other sample - Done do that the two individuals are equivalent with respect to a specific variable the researcher would like to control - Related-samples designs or correlated samples designs Hypothesis Test for the Repeated Measures Steps: 1. State the hypotheses and select alpha 2. Locate critical region 3. Calculate t 4. Make decision Directional Hypotheses and One-Tailed Tests: Requires 2 basic assumptions - The observations within each treatment condition must be independent - The population distribution of difference scores must be normal Time-Related Factors and Order Effects: - Primary disadvantage of a repeated measures design = the structure of the design allows for factors other than the treatment effect to cause a participant’s score to change from one treatment to the next - This is dealt with through counterbalancing - The partipants are randomly divided into 2 groups with one group receiving treatment 1 followed by treatment 2, and the other group receives treatment 2 first...