Replicability, Generalization, and the Real World - Chapter 14 PDF

Preview

Summary

Dr. Prichett's Research Methods chapter 14....

Description

Replicability, Generalization, and the Real World - Chapter 14 A Study Must Be Replicable... Why? ● Replicable: reproduced/reproducible ● Replication studies: ● Direct replication ● Conceptual replication ● Replication-plus-extension Types: Direct Replication ● Some variable, some operationalization Types: Conceptual Replication ● Same variables, different operationalization ○ Measuring the variable in different ways ○ The overarching variables are the same Types: Replication-Plus-Extension ● Same variables, plus some new variable ○ Adding more extensions to the variables ■ As long as the basic concept remains, adding more variables is not bad. Considering the value of the replication: Replications by independent researchers ● Independent versus original labs ○ Role of environment/conditions ■ Prevent internal validity Replication, Importance, and the Weight of Evidence ● A single study is only important if it can be replicated. ● Even statistically significant results must be replicated. ● Science progresses through replication of studies. Meta-Analysis: What Does the Literature Say? ● Meta-analysis: is a statistical analysis that produces a quantitative summary of scientific literature. ● Example: religiosity and depression ● Example: violent video games and behavior ● Effects sizes Cohen’s d ○ 0.20 = small ○ 0.50 = moderate ○ 0.80 = strong

○ Small effects are more likely to miss the sign. Strengths and Limitations of Meta-Analysis ● File drawer problem ○ Went into a file and left alone (figurative statement) ● Null results and opposite results are rarely published.

Replicability in the Popular Press ● Journalists do not always consider the importance of replicability. ● A responsible journalist will not only report on the latest and most exciting studies, but they also will summarize the body of literature on the topic. To Be Important, Must a Study Have External Validity? ● Review: External Validity ● It’s a population, not the population ● External validity comes from how , not how many . ● Just because a sample comes from a population doesn’t mean it generalizes to that population. Generalizing to Other Settings ● Ecological validity: an aspect of external validity in which the focus is on whether a laboratory study generalizes to real-world settings Does a Study Have to Be Generalizable to Many People? ● How is external validity related to the importance of a study? Generalization Mode ● Frequency claims: always in generalization mode ● Association and causal claims: sometimes in generalization mode Theory Testing Using WEIRD Participants ● Most research in psychology has been conducted on North American college students. ● WEIRD = Western, educated industrialized, rich and democratic Does a Study Have to Take Place in a Real-World Setting? Vs. External Validity and the Real World ● When a study takes place it the real world, it occurs in a field setting and has high external validity.

● Laboratory research can be just as realistic as research conducted in the real world, which is known as experimental realism. Generalization Mode and the Real World ● External validity  is important when in generalization mode. ● Ecological validity c onsider how a laboratory study generalizes to the real world....