Research Methods Final Study Guide PDF

Preview

Summary

Download Research Methods Final Study Guide PDF

Description

Chapter 7: Control Techniques in Experimental Research What is Random Assignment? (best control technique) Also called randomization: most important and basic of all control methods. Designed to equate experimental groups at the start of an experiment on all extraneous variables. The only control technique for controlling both known and unknown extraneous variables - When is it used? - At the start of an experiment - Why is it important? - It equates groups- so there will be no systematic group differences or extraneous variables to bias the study results What is matching? (next best control technique) Matching is another control technique used when random assignment isn’t possible. Using any of a variety of techniques for equating participants on one or more variables. Ex: to match participants on intelligence you need to know intelligence scores. Weakness: the groups are equated only on the matching variables… Matching by Holding Variables Constant: Hold the extraneous variable constant for all groups in the experiment. All participants will have the same degree or type of extraneous variable. This creates a more homogenous participant sample. Disadvantages: the technique restricts the size of the participant population and it may be hard to find enough participants. The results of the study can be generalized only to the type of participant who participated in the study. Matching by Building the Extraneous Variable into the Research Design: (sometimes called “blocking”) treating the extraneous variable as an independent variable. This is recommended if one is interested in the differences produced by the various levels of the extraneous variable and other independent variables - Some experts recommend that a quantitative variable such as intelligence not be categorized into a few groups… they say the matching variable, intelligence, should be left in its natural units and then entered into the study during statistical analysis (this form of control = statistical control ) Matching by Yoked Control A matching technique that matches participants on the basis of the temporal sequence of administering an event. The yoked control technique controls for the possible influence of participant-controlled events. When the experimental participant engages in behavior and receives an outcome, the “yoked” participant is given the same outcome. (monkeys getting shocked)

Matching by Equating Participants Similar to matching by building the extraneous into the design: both eliminate the influence of the

extraneous variable by creating equivalent groups. This kind of matching matches individual participants on the variable to be controlled = individual matching/ subject matching (a matching technique in which each participant is matched with another participant on selected variables) - Requires a case-by-case analysis for each variable - Advantage: the participants in the various groups are equal on the matched variable= the treatment groups are equated on the matching variables which rules out the extraneous variables as explanations for the relationship between IV and DV - Disadvantage 1: (when used without RA) it is difficult to know which matching variables should be used and which of the potential matching variables are most critical. (the variables selected should be those that show the lowest intercorrelation but the highest correlation with the DV) - Disadvantage 2: (no RA) the difficulty in finding matched participants increases disproportionately as the number of matching variables increases. In order to match individuals on many variables, one must have a large pool of individuals available in order to obtain a few who are matched on the relevant variables. - Disadvantage 3: (no RA) matching can limit the generalizability of the results of the study. This occurs if you have to throw out participants for whom you cannot find adequate matches. Eliminating people produces unrepresentative samples. You can only generalize the exact kinds of people that remain in your study. - Disadvantage 4: some variables are very difficult to use in matching. Matching can only be as accurate as the available measurement of the matching variable What is counterbalancing? A technique used to control for sequencing effects. The control method used only in repeated measures designs. - When? - In a repeated measures (or a within-participants design: where all research participants receive all treatments) - Why? - When all participants receive all treatments, at different times (within-participants) it is not a good idea to administer the treatments to all participants in the same order. = sequencing effects - Sequencing effects: occur when participants participate in more than one treatment condition - Order effect: a sequencing effect arising from the order in which the treatment conditions are administered to participants. Because the participant has changed somewhat from the first to last condition regardless of the treatment. The participant’s increased familiarity and practice with the whole experimental environment can produce order effects - Carryover effect: occurs when performance in one treatment condition is partially dependent on the previous conditions. - Minimized by substantial time in between conditions called a “wash out” period - Randomized Counterbalancing: where sequence order is randomly determined for across

-

-

-

individuals. Intrasubject Counterbalancing: having the participants take the treatment conditions in more than one order… once normally, then reversed. (only used when each participant receives all levels of independent variable more than once. ) ABBA - Any observed difference between the averages would not be attributable to carryover or order effects because they would have been equalized, or held constant across conditions - Disadvantage: each treatment condition must be presented to each research participant more than once. As the # of treatment conditions increases, the length of the sequence increases. Group Counterbalancing: administering different sequences to different groups of participants; sequences are varied across two or more groups of participants Complete Counterbalancing: all possible sequences are used. And an equal proportion (this is important!!) of research participants are RA to each sequence. - When there are 2 conditions, there are two possible sequences : 1,2 and 2,1 - When there are 3 conditions, there are 6 possible sequences: - You can determine the number of sequences with n! - Limitation: when the number of condition grows… the number of possible sequences becomes super big Incomplete Counterbalancing: most frequently used !! using fewer than all possible sequences and requiring different groups of participants to take each of the sequences. - For the sequences enumerated, each treatment condition must precede and be followed by every other condition an equal number of times. If the number of treatment conditions is odd, then we must enumerate additional sequences that are exactly the reverse of the first sequences. - Sequencing effects?? Controlled for because each treatment occurs at every possible time… as long as they are linear… - Differential carryover effect: a treatment condition affect participant’s performance in a later condition in one way and in another way when followed by a different condition.

Blind and Double Blind procedures? (One of the best techniques for controlling demand characteristics.) Neither the experimenter nor the research participant is aware of the treatment condition administered to the participant. - Why? - Eliminates the development of differential participant perceptions because all participants are told the same thing. And because the researcher does not know, they cannot communicate this to participants. - When? - Executed when expectancies for two comparison groups needs to be held constant- trial vs. placebo (ex. Drug research) (pg. 207) Deception? Giving the participant a bogus rationale for the experiment Used to solve the problem of participant perceptions. - How much?

-

Deception can range from minor deceit (omission or slight alteration of truth) to more elaborate schemes - It is better to give false explanations so the participants’ curiosity is satisfied. - To everyone or only some subjects? - All participants should receive the same false information to produce relatively constant participant perceptions. What are demand characteristics? - participants form an interpretation of the experiment's purpose and unconsciously change their behavior to fit that interpretation. - How do we control for them? One way is through the use of deception. Using deception may reduce the likelihood that participants are able to guess the hypothesis of the experiment, causing participants to act more naturally How are subjects perceptions controlled? Retrospective verbal reports: an oral report in which the participant retrospectively recalls aspects of the experiment in an interview after the experiment is over = post experimental inquiry Concurrent verbal reports: a participant’s oral report of the experiment, which is obtained as the experiment is being performed - Include techniques such as sacrifice groups: groups of participants who are stopped and interviewed at different stages of the experiment. - Concurrent probing: obtaining a participant’s perceptions of the experiment after the completion of each trial - Think aloud technique: method that requires participants to verbalize their thoughts as they are performing the experiment - Disadvantage: verbalizing ones thoughts during the experiment might affect their behavior and therefore the DV. Experimenter Effects: the unintentional biasing effect that the experimenter can have on the results of the experiment - Recording Errors: the misrecording of data - Can be minimized if the person recording the data remains aware of the necessity of making careful observations. Even better approach: use multiple observers or data recorders. - Improved even further: data recorders kept blind regarding the experimental conditions to which the participant was responding - Attribute Errors: - Should not significantly affect the mean differences among groups (chances are the effect was constant across all individuals so the difference of the mean shouldn’t change. - As long as experimenter 1 administers condition A and condition B, any observed difference between conditions A and B cannot be due to the experimenter (because they were constant across groups A and B) - *** key point: never use one experimenter for one condition and another for the other condition - If the experimenter attribute is correlated with the dependent variable, then it should be controlled

-

Experimenter Expectancy Error: - The blind technique: a method whereby knowledge of each research participant’s treatment condition is kept from the experimenter - Partial blind technique: a method whereby knowledge of each research participant’s treatment condition is kept from the experimenter through as many stages of the experiment as possible - The experimenter can use random assignment to designate which group participants are in - Best possible: if the experimenter can leave the room immediately following administration of the IV and allow another person (who is ignorant of the experimental conditions ) measure the DV - Automation: the technique of totally automating the experimental procedures so that no experimenter- participant interaction is required - Ex: having instructions written, tape-recorded, filmed, televised, or presented by means of a computer or a similar device.

**** if the groups differ only on the independent variable (and not on any extraneous variables) the researcher can conclude that the independent variable is the causal influence of any observed difference. Why is control difficult to achieve in a single study? - Control techniques are effective, but not 100% - The key: use the strongest control methods available and do your best to collect additional data to determine how well the control worked and adjust interpretations accordingly Why is generalizability difficult to achieve in a single study? - Small sample size?? - Many control methods involve restricting pop size or satisfying specific characteristics of participants What two components must be present for a study to be deemed a “true experiment” - RANDOM ASSIGNMENT, and a manipulated IV Chapter 8: Understand differences between weak and strong experimental designs - Research design: the outline, plan or strategy used to investigate the research problem Weak Experimental Design: (more difficult to draw a causal relationship) designs that do not control for many extraneous variables (threats to internal validity) and provide weak evidence of cause and effect. Should be avoided if using a strong design is possible. Strong Experimental Design: designs that effectively control E. variables = strong evidence that the effect of the IV on the DV has been isolated and tested = greater internal validity. - Goal: eliminate potential rival hypotheses!! This is done using Control Techniques (RA = te=he most important one) and Control Groups - Control groups: the group of participants that does not receive the active treatment - standard of comparison for determining a causal effect

-

2 functions: - 1) a source of comparison- used to estimate the counterfactual: what the responses would have been without treatment - 2) a control for rival hypotheses; Goal: all variables to operate on the control and experimental groups identically - RCT: (randomized control trial): experimental design with random assignment to experimental and control groups Describe three weak designs in the text: - One-group posttest-only design: administration of a posttest to a single group of participants after they have been given an experimental treatment condition. - Rarely useful- allows no evidence of the DV had they not received treatment; no notreatment control group - Threats to internal validity: History, Maturation, Attrition - Disadvantage: we cannot know if the IV influenced the DV - Pretest: measure of the DV before treatment - Advantages: a good way to measure a change in performance - Disadvantages: we cannot measure the influence the test has on participants - One-group pretest-posttest design: Design in which a treatment condition is interjected between a pretest and posttest of the DV - Threats to internal validity: History, Maturation, Instrumentation, Testing, Regression artifact, (Attrition is controlled for) - Does not allow us to control or test for possible influence of tests, but still helpful in situations where it’s impossible to obtain an equated comparison group - Disadvantage: we cannot know if the IV influenced the DV - Posttest only with nonequivalent groups design: design in which the performance of an experimental group is compared with that of a nonequivalent control group at the posttest. - Implements a control group! Yay - Threats to internal validity: Selection, Additive/interaction effects (History, Maturation, Regression effect, and Attrition controlled for) - Disadvantage: the comparison group is a nonequivalent group, thus the participants in the comparison group may differ from those in the experimental group. (this threat to internal validity = selection:) - “Selection threat” : when a differential selection procedure is used to place participants into comparison groups=nonequivalent; you want your groups to only vary on the IV - What does a control group “buy” in an experiment? - Why?

Describe the strong experimental designs: (Threats to internal validity are controlled for) Between-Participants Designs (also called randomized designs): groups are produced by RA; different groups are exposed to diff. levels of the IV (between-subjects IV) - The number of participants needed = (# of participants needed for one condition) * (# of

treatment conditions) - Between groups: - Posttest-only control group design: (a form of BPD) administration of a posttest to 2 or more RA groups that receive diff. levels of the IV - Randomized control group accounts for all threats to internal validity - Advantages: - Inclusion of a control group and RA = eliminate threats to internal validity - Disadvantages: - RA does not provide complete assurance of equivalence (especially with a small sample size) - Lacks a pretest- lacks those potential benefits - 2 groups: but seldom experiments are confined to only two levels of variation Within-participants design(also called repeated measures designs): all participants receive all conditions (within-subjects IV); participants serve as their own control - Often the designs that are employed in research using cognitive or psychological measures - WP posttest-only design:all participants receive all conditions and a posttest is administered after each condition is administered. - Uses counterbalancing due to the likelihood of carryover effects - Advantages: WP designs are maximally sensitive to the effects of the IV - participants serve as their own control = they are perfectly matched in the various treatment conditions; increasing the sensitivity of the experiment… especially if counterbalancing is used (all threats are controlled) - Doesn’t require as many participants as the between participants designs - Disadvantages: - WP designs can be taxing on participants because they have to be present for multiple conditions - Sequencing effect: go back i talked about this earlier - Because all participants participate in all conditions, a sequencing rival hypothesis is a real possibility - Pretest-posttest control-group design: administration of a posttest to 2 or more RA groups of participants after the groups have been pretested and administered the diff levels of the IV - This is a mixed design- it has a between subjects IV - Includes “time”; all participants receive pretest @ time 1 and posttest @ time 2 - Advantages: pretest allows the researcher to check and see how well the RA worked; they see if the groups are similar on the DV after RA - Researcher can determine if a ceiling effect or floor effect is likely - If groups differ on the DV, researchers can use a statistical technique analysis of covariance to control for these differences - Researcher gains an empirical demonstration of whether an overall change occurred from pretesting to posttesting - Disadvantages: the participants may change in some way due to the pretest and can

weaken external validity. Ceiling effect: situation where participant’s pretest scores on DV are too high to allow for additional increases - Floor effect: where participant’s scores on DV are too low for additional decreases - Analysis of covariance: statistical procedure: group means are compared after adjusting for pretest differences Factorial designs: 2 or more IV are studied to determine their separate and joint effects on the dependent variable - Allows investigation of 2 types of effects: main effects and interaction effects - Main effect: the separate influence of each IV on DV - Interaction effect: when the effect of 2 or more IVs on the DV is more complex than indicated by main effects - Two-way interaction effect: the effect of one IV on the DV varies with the different levels of the other IV; when you have 2 IVs, you analyze the data for 2 main effects and one interaction effect - IVs can be between-subjects, within-subjects, or a combination (a mixed design) - To determine if there is an interaction: draw a line plot - No interaction rule: if the lines are parallel= no interaction - Interaction rule: not parallel = there is an interaction - “Non-significant” effects: determined through statistical tests; uses the marginal mean to determine if main effects are present; when means are identical = no variation - Between subjects IVs: different participants receive diff levels of IV - Ex: one IV has 3 levels, another has 2 - the combinations of the two = a cell - # of cells: (# of cells)* (# of levels of IVs) - Within-subjects IVs: all participants receive all levels of IV - Key point: regardless of the kind of IV, you check for main effects and int...