Print Preview Ch 9 PDF

Preview

Summary

book chapter...

Description

Print Preview

3/29/16, 9:36 PM

Chapter 9: Experimental Designs: Within-Subjects Design Chapter Contents Book Title: Research Methods for the Behavioral Sciences Printed By: jasmine castaneda ([email protected]) © 2015 Cengage Learning, Cengage Learning

Chapter 9 Experimental Designs: Within-Subjects Design Chapter Introduction 9.1 Within-Subjects Experiments and Internal Validity Characteristics of Within-Subjects Designs Threats to Internal Validity to Within-Subjects Experiments Separating Time-Related Factors and Order Effects Order Effects as a Confounding Variable 9.2 Dealing with Time-Related Threats and Order Effects Controlling Time Switch to a Between-Subjects Design Counterbalancing: Matching Treatments with Respect to Time Limitations of Counterbalancing 9.3 Comparing Within-Subjects and Between-Subjects Designs Advantages of Within-Subjects Designs Disadvantages of Within-Subjects Designs Choosing Within- or Between-Subjects Matched-Subjects Designs 9.4 Applications and Statistical Analysis of Within-Subjects Designs Two-Treatment Designs Multiple-Treatment Designs Chapter Review Chapter Summary Key Words Exercises

http://ng.cengage.com/static/nbreader/ui/apps/nbreader/print_preview/print_preview.html

Page 1 of 44

Print Preview

3/29/16, 9:36 PM

Engagement Activities Chapter 9: Experimental Designs: Within-Subjects Design Chapter Introduction Book Title: Research Methods for the Behavioral Sciences Printed By: jasmine castaneda ([email protected]) © 2015 Cengage Learning, Cengage Learning

Chapter Introduction

Learning Objectives LO 1 Describe the general characteristics of a within-subjects experimental design and identify these designs when they appear in a research report. LO 2 Describe how time-related factors such as history, maturation, instrumentation, statistical regression, and order effects can threaten the internal validity of some within-subjects experiments. LO 3 For a within-subjects experiment, explain how the time delay betweentreatments can influence time-related threats to internal validity and why it may be better to switch to a between-subjects design. LO 4 Define counterbalancing and explain how it is used to minimize or eliminate threats to internal validity from time-related factors. LO 5 Describe the limitations of counterbalancing and explain why partial counterbalancing is sometimes used. LO 6 Explain the general advantages and disadvantages of within-subjects designs compared to between-subjects designs and be able to decide which design would be better under specific circumstances. LO 7 Define a matched-subject design and explain how it attempts to achieve the advantages of both within- and between-subjects designs without their disadvantages. LO 8 Describe the different ways that within-subjects designs are used to compare two or more treatment conditions, identify the statistical techniques that are appropriate for each application, and explain the strengths and weaknesses of each application.

http://ng.cengage.com/static/nbreader/ui/apps/nbreader/print_preview/print_preview.html

Page 2 of 44

Print Preview

3/29/16, 9:36 PM

Chapter Overview Step 6 of the research process involves selecting a research design. In this chapter, we discuss in detail another type of experimental research design: the within-subjects design. The threats to internal validity, advantages, disadvantages, and different versions of withinsubjects designs are considered. Chapter Preview Gaucher, Friesen, and Kay (2011) recently reported a series of studies examining genderbiased wording in job advertisements. In one study, they started with six job advertisements and manipulated each one to create a feminine-worded version (using words such as compassionate, gentle, sensitive, and nurturing) and a masculine-worded version (using words such as competitive, independent, analyze, strong). Participants were then given a set of six advertisements, three with masculine wording and three with feminine wording, and were asked to rate the appeal of each job. The researchers then computed the average rating for the three masculine ads and for the three feminine ads to obtain two scores for each participant. The results show that women found the ads with masculine wording to be significantly less appealing than the ads with feminine wording. In another study, the researchers found that masculine wording is more common than feminine wording in job advertisements, especially for jobs in male-dominated areas such as engineering or plumbing. Together, these two results may help explain the persistence of gender inequality within some professions. If traditionally male-dominated professions are advertised using masculine-themed language, then women are likely to find the jobs unappealing and will not apply. From our perspective, the first study is interesting because each participant produces two scores: a rating for the masculine-worded ads and a rating for the feminine-worded ads. Thus, the two groups of scores are both obtained from a single group of people. The two groups of scores are then compared to demonstrate the effect of the two different wordings. This kind of research study is known as a within-subjects design because it is examining differences between scores within one group of participants. In this chapter, we introduce within-subjects experimental designs, examine threats to validity for these designs, and discuss the relative advantages and disadvantages of within-subjects experiments compared to the between-subjects experiments that were presented in the previous chapter. Chapter 9: Experimental Designs: Within-Subjects Design: 9.1 Within-Subjects Experiments and Internal Validity Book Title: Research Methods for the Behavioral Sciences Printed By: jasmine castaneda ([email protected]) © 2015 Cengage Learning, Cengage Learning

9.1 Within-Subjects Experiments and Internal Validity

http://ng.cengage.com/static/nbreader/ui/apps/nbreader/print_preview/print_preview.html

Page 3 of 44

Print Preview

3/29/16, 9:36 PM

Learning Objectives

LO 1 Describe the general characteristics of a within-subjects experimental design and identify these designs when they appear in a research report. LO 2 Describe how time-related factors such as history, maturation, instrumentation, statistical regression, and order effects can threaten the internal validity of some within-subjects experiments.

Chapter 9: Experimental Designs: Within-Subjects Design Characteristics of Within-Subjects Designs Book Title: Research Methods for the Behavioral Sciences Printed By: jasmine castaneda ([email protected]) © 2015 Cengage Learning, Cengage Learning

Characteristics of Within-Subjects Designs In the preceding chapter, we described the basic elements of the between-subjects experimental research design. Recall that the defining characteristic of a between-subjects experiment is that it requires separate but equivalent groups of participants for the different treatment conditions compared. In this chapter, we introduce an alternative research procedure: the within-subjects design (A research design in which the different groups of scores are all obtained from the same group of participants. Also known as repeated-measures design.) . The defining characteristic of a within-subjects design is that it uses a single group of participants and tests or observes each individual in all of the different treatments being compared. Thus, in a within-subjects study, the sample is not separated into several groups but rather exists as a single group that participates in every treatment condition. Using the terminology of experimental research, in a within-subjects experimental design the same group of individuals participates in every level of the independent variable so that each participant experiences all of the different levels of the independent variable. In one sense, a within-subjects study is the ultimate in equivalent groups because the group in one treatment condition is absolutely identical to the group in every other condition. In the context of statistical analysis, a within-subjects design is often called a repeatedmeasures design (See within-subjects design.) because the research study repeats measurements of the same individuals under different conditions (Figure 9.1). Figure 9.1

The Structure of a Within-Subjects Design http://ng.cengage.com/static/nbreader/ui/apps/nbreader/print_preview/print_preview.html

Page 4 of 44

Print Preview

3/29/16, 9:36 PM

The same group of individuals participates in all of the treatment conditions. Because each participant is measured in each treatment, this design is sometimes called a repeated-measures design. Note: All participants go through the entire series of treatments but not necessarily in the same order.

In this chapter, we examine the within-subjects experimental design (An experimental design in which the same group of individuals participates in all of the different treatment conditions. Also known as a repeated-measures experimental design.) ; that is, the withinsubjects design as it is used in experimental research comparing different treatment conditions. We should note, however, that the within-subjects design is also well suited to other, nonexperimental types of research that investigate changes occurring over time. For example, studies in human development often observe a single group of individuals at different ages to monitor development over time. Examples of nonexperimental withinsubjects designs are examined in Chapter 10. Chapter 9: Experimental Designs: Within-Subjects Design Threats to Internal Validity to Within-Subjects Experiments Book Title: Research Methods for the Behavioral Sciences Printed By: jasmine castaneda ([email protected]) © 2015 Cengage Learning, Cengage Learning

Threats to Internal Validity to Within-Subjects Experiments A within-subjects experimental study must be concerned with threats to internal validity from environmental variables that may change systematically from one treatment to another, and from time-related factors that may influence the participants’ scores (Chapter 6, Environmental Variables: General Threats to Internal Validity for All Studies, Individual Differences: Threats to Internal Validity for Studies Comparing Different Groups and TimeRelated Variables: Threats to Internal Validity for Studies Comparing One Group over Time). Thus, there are two major sources of potential confounding for a within-subjects design. 1. Confounding from environmental variables. Environmental variables are characteristics of the environment that may change from one treatment condition to another. For example, one treatment may be evaluated during the morning and another treatment during the afternoon. Any such variable may cause differences in

http://ng.cengage.com/static/nbreader/ui/apps/nbreader/print_preview/print_preview.html

Page 5 of 44

Print Preview

3/29/16, 9:36 PM

scores from one treatment to another, and therefore provides an alternative explanation for the differences between treatments. 2. Confounding from time-related variables. A serious concern of within-subjects designs comes from the fact that the design often requires a series of measurements made over time. During the time between the first measurement and the final measurement, the participants may be influenced by a variety of factors other than the treatments being investigated, and these other factors may affect the participants’ scores. If this occurs, then the internal validity of the study is threatened because a change in a participant’s score from one treatment to the next could be caused by an outside factor instead of the different treatments. In this section, we identify five time-related threats to internal validity. 1. History: The term history (A threat to internal validity from any outside event that influences the participants’ scores in one treatment differently than in another treatment.) refers to environmental events other than the treatment that change over time and may affect the scores in one treatment differently than in another treatment. Events that occur in participants’ lives at home, in school, or at work may affect their performance or behavior in different sections of the research study. For example, suppose a group of students serves as participants in a research study that extends over several days with a different treatment condition each day. If there is an outside event that is likely to affect many of the students on one particular day, but not on another day, then the event may provide an explanation for unusual performance on that particular day. For example, suppose that in the middle of this study a fire alarm sounds in the main campus dormitory just after midnight, and the students are left standing outdoors for hours. When the students are tested later in the day, they are likely to show poor memory scores, not because of the treatment condition but because they are all exhausted from missing sleep the night before. Note that history effects are usually events that occur during the course of the research study. However, a study can be influenced by an event that occurs prior to it. If the midnight fire alarm sounded on the night before the study started, then it could affect performance on the first day of the study but not on subsequent days. In this case, the event is still a threat to internal validity, even though it occurred prior to the start of the study. To be a confounding variable, history effects must influence at least one treatment condition differently and influence enough of the participants to have an influence on the overall group

http://ng.cengage.com/static/nbreader/ui/apps/nbreader/print_preview/print_preview.html

Page 6 of 44

Print Preview

3/29/16, 9:36 PM

performance. 2. Maturation: Any systematic changes in participants’ physiology or psychology that occur during a research study and affect the participants’ scores are referred to as maturation (A threat to internal validity from any physiological or psychological changes that occur in a participant during the time that research study is being conducted and that can influence the participant’s scores.) . Maturation effects are of particular concern when the research participants are young children or elderly adults. Young children, for example, can gather new knowledge and skills or simply grow bigger and stronger in a relatively short time. As a result, their performance at the end of a series of treatment conditions may be very different from their performance at the beginning, and the change in performance may not have been caused by the treatments but instead by maturation. With elderly participants, maturation effects often have a detrimental effect. As people age, they may experience losses in vision or hearing that could affect their performance in a research study. In general, maturation threatens the internal validity of a research study conducted over time because it weakens our confidence that the different treatment conditions are responsible for observed changes in the participants’ scores. Maturation is a particular concern in research situations in which the series of treatments extends over a relatively long time. 3. Instrumentation: The term instrumentation (A threat to internal validity from changes in the measurement instrument that occur during the time a research study is being conducted. Also known as instrumental bias or instrumental decay.) (sometimes called instrumental bias (See instrumentation.) or instrumental decay (See instrumentation.) ) refers to changes in a measuring instrument that occur over time. For example, a scale used to weigh participants may gradually wear out during the course of the study. In this case, the measurements change during the study, not because of the different treatments but because of the changes in the scale. Behavioral observation measures (discussed in Chapters 3 and 13) are much more subject to instrumentation than other types of measures. For example, from one testing to the next, the researcher doing the observing may change the standards on which the observations are based, or become more skilled or fatigued, and as a result, judge the same behavior differently at different times. Notice that the changes in the participants’ scores are not caused by the treatment but instead by a change in the measurement instrument (the researcher). Like history and maturation, instrumentation is a particular concern in research situations in

http://ng.cengage.com/static/nbreader/ui/apps/nbreader/print_preview/print_preview.html

Page 7 of 44

Print Preview

3/29/16, 9:36 PM

which the series of treatments extends over a relatively long time. 4. Regression toward the mean: Statistical regression, or regression toward the mean (A statistical phenomenon in which extreme scores (high or low) on a first measurement tend to be less extreme on a second measurement; considered a threat to internal validity because changes in participants’ scores could be caused by regression rather than by the treatments. Also known as regression toward the mean.) , refers to the tendency for extreme scores on any measurement to move toward the mean (regress) when the measurement procedure is repeated. Individuals who score extremely high on a measure during the first testing are likely to score lower on the second testing, and, conversely, individuals who score extremely low on a measure during the first testing are likely to score higher on the second testing. Statistical regression occurs because an individual’s score is a function both of stable factors such as skill and of unstable factors such as chance. Although the stable factors remain constant from one measurement to another, the unstable factors can change substantially. Your grade on an exam, for example, is based on a combination of knowledge and luck. Some of the answers you really know; others you guess. The student who gets the highest score on the first exam probably combines knowledge and good luck. On the second exam, this student’s knowledge is still there, but luck is likely to change; thus, the student will probably score lower on the second exam. This is regression toward the mean. In research, regression is a concern whenever participants are selected for their exceptionally high (or low) scores. Suppose a clinical psychologist is examining how a specific treatment influences the social skills of autistic children. A sample of autistic children is selected because a preliminary test indicates that they have exceptionally poor social skills. The psychologist administers the treatment and then once again measures social skills. Because of the children’s extremely low scores on social skills at the beginning of the study, it is possible that the children’s scores improve, not because of the treatment but because their scores regress toward the mean. In general, statistical regression threatens the internal validity of a research study because it creates the possibility that the observed changes in the participants’ scores are caused by regression instead of by the treatments. 5. Order effects (practice, fatigue, and carry-over effects): Whenever individuals are tested in a series of treatment conditions, participation in one treatment http://ng.cengage.com/static/nbreader/ui/apps/nbreader/print_preview/print_preview.html

Page 8 of 44

Print Preview

3/29/16, 9:36 PM

may have an influence on the participants’ scores in the following treatments. For example, becoming fatigued in one treatment may lead to poorer performance in the next treatment. You should recognize this problem as a threat to internal validity. Specifically, the experience of being tested in one treatment may explain ...