Chapter 7 PDF

Preview

Summary

Chapter 7...

Description

Quantitative Research Designs There are three types of quantitative research design: experimental forms, quasi-experimental forms, and descriptive forms.These forms differ in fundamental ways on two characteristics: manipulation of independent variables and random assignment of participants to treatments or conditions (Pedhazur & Schmelkin, 1991). The first characteristic on which quantitative research designs differ—manipulation of independent variables—occurs when the researcher intentionally varies or changes how the independent variable is presented to participants. This fundamental characteristic must be satisfied to locate a research study in the classic experimental framework. Manipulation of independent variables also occurs in quasiexperimental research designs but is absent from descriptive forms. The second characteristic on which quantitative research designs differ—random assignment of participants to treatments or conditions—is unique to experimental forms. After being selected to participate in the experiment, the researcher randomly assigns individuals to one of at least two groups. One group is the control group; this group serves as a baseline against which the treatment groups are evaluated. The control group can be used in two ways. First, the control group can receive no treatment. For example, in a study evaluating the effectiveness of public speaking instruction, the control group would not receive any type of instruction. Second, the control group can receive the standard treatment. Using the same example, the control group would receive a routine face-to-face classroom lecture on public speaking. In either case, the control group is the baseline of comparison for the experimental, or treatment groups. Continuing with the public speaking example, one treatment group would receive the same information in an interactive face-to-face lecture on public speaking; a second treatment group would receive the same information in an online self-guided study of public speaking.

THE EXPERIMENTAL FRAMEWORK When researchers are curious about causes, they often turn to experimental research, which has a long tradition in the social sciences, including communication. This type of research is most often conducted in the laboratory or in other simulated environments that are controlled by researchers. Alternatively, researchers can conduct experiments in the field, or naturally Page 114occurring environments. This type of experiment is very popular with communication scholars who study applied communication problems. The traditional definition of an experiment, one often associated with the physical sciences, would characterize it as the manipulation of the independent variable in a controlled laboratory setting conducted on a randomly selected sample of participants who are randomly assigned to control or treatment groups. A broader definition of experiment, and one that fits the study of communication phenomena more appropriately, is the recording of measurements and observations made by defined procedures and in defined conditions. The data collected or

produced by these procedures are then examined by appropriate statistical tests to determine the existence of significant differences between and relationships among variables. Experimental research is chosen when a researcher wants to determine causation. In other words, a researcher has developed a hypothesis that asserts that one variable, the independent, causes change in a second variable, the dependent. Experimentation allows a researcher to evaluate hypotheses that have been developed from theories in the literature.In this case, results from previous research studies generate new questions, and the researcher wants to answer the questions “Why?” and “How?” Experimentation is also chosen when researchers want to test a new method or technique. This is especially true of instructional techniques that are believed to have potential application in the classroom. Conducting an experiment allows the researcher to test one technique against another to see if differences between techniques exist. In other cases, experiments are conducted to explore the specific conditions under which a phenomenon occurs. By varying the conditions of the experiment, researchers can identify which environmental conditions, for example, are most likely to make speakers nervous. When research is identified as experimental, the goal of the researcher is to establish or explain what caused a person’s behavior, feelings, or attitudes to change. Because this is the goal, certain characteristics must be satisfied. First, the research design must have a temporal component, with one element occurring before another. For something to cause something else, the causal agent must precede the change in behavior, feelings, or attitudes. In this way, an experiment provides control of one variable, or the independent variable, to test its effect on another variable, the dependent variable.Second, there are comparisons between at least two groups. Finally, the entire experiment is conducted within a limited time frame, seldom more than 1 hour, with all of the interaction under the control and observation of the researcher. When the word experiment is used, most people think of laboratory experiments. The primary defining characteristic of laboratory experiments is that the researcher structures the environment in which the investigation takes place and in which the data are collected (Weaver, 2008).Conducting research in the lab environment serves several purposes. First, it physically isolates the research process from the day-to-day and routine interaction of participants. This isolation gives a researcher greater control over what participants are and are not exposed to. By limiting and controlling exposure in this way, a researcher is attempting to eliminate extraneous variables and influences that are not central to the investigation (Kerlinger, 1986). Second, exploring communication in the lab allows researchers to confine and examine theoretical relationships that would be more difficult to do in the field. Using manipulation and random assignment, researchers design and conduct a study, evaluate the evidence, and then develop a causal explanation for what has occurred. Experimental designs are deliberate, standardized, and used as the research protocol in many disciplines. Their strength lies in the control they provide to researchers, which in turn helps them eliminate rival explanations for the changes they observe and record. Several of the experimental designs more commonly used in communication research are described in the following sections. The Classical Experiment In a classical experiment, the researcher controls the treatment or the manipulation of the independent variable by randomly assigning participants to treatment or control groups. A treatment, or manipulation, is one of the ways in which the researcher varies the type of stimuli

or the amount or level of stimuli presented to research participants. This fundamental characteristic must be satisfied to locate a research study in the classical experimental framework. Experiments allow researchers to test two types of hypotheses—those that predict differences and those that predict relationships. Recall from Chapter 4 that hypotheses test differences and relationships between and among variables, not the variables themselves (Kerlinger, 1986). For the first type of hypothesis—the difference hypothesis—the experiment is designed so that the independent variable precedes the dependent variable in temporal order. Thus, the corresponding hypothesis predicts that the independent variable causes changes, or effects, in the dependent variable. For the second type of hypothesis—the relational hypothesis—the experiment is designed so that two variables, the independent and dependent, occur close together. The hypothesis predicts that the two variables exist together in some type of relationship where the value of the independent variable is causing a change in the value of the dependent variable. See Chapters 10 and 11, respectively, for the statistical tests that accompany these hypotheses. The researcher also controls the order of variables in an experiment. One element, the independent variable, cannot be considered the cause of another element, the dependent variable, if the independent variable occurs after the dependent variable (Selltiz, Jahoda, Deutsch, & Cook, 1959). Simply put, the independent variable can be considered the cause of changes in the dependent only if the independent precedes the dependent or if the two occur close together. Random Assignment of Participants In any experiment in which the researcher wants to compare two or more groups, the underlying principle is that individuals in the groups are equivalent before the treatment. To achieve equivalency, participants are randomly assignedto treatment or control groups that represent the independent variable. This means that each participant has an equal chance of being assigned to either group. Selecting a random sample from an appropriate population is not the same as randomly assigning individuals to treatment and control groups. The two procedures together help eliminate any true differences between individuals in the groups before the treatment is applied. Thus, the researcher can argue that differences that result after the treatment is applied are caused by the independent variable. Creating Treatment and Control Groups The treatment group is the group of participants who receive a stimulus—anything that the researcher is interested in studying.Alternatively, if a participant is assigned to a control group, no treatment or a baseline treatment is offered. In the experimental framework, creation of the treatment and control groups provides the opportunity for the manipulation of the independent variable. It is important to note here that identification of what constitutes a treatment, or condition, is driven by theory (Boruch, 1998); not just any treatment will do. Manipulation Checks When an independent variable is manipulated, a researcher should conduct a manipulation check. This test, or check, verifies that participants did, in fact, regard the independent variable

in the various ways that the researcher intended. This check is conducted prior to statistical analyses of the hypotheses. Why is this necessary? First, researchers need to confirm that participants were sensitive to the different treatments. Second, researchers need to confirm that differences in treatments as perceived by participants were in line with the differences the researcher intended. Without this confirmation, a researcher might assume that differences in the dependent variable were due to differences in the independent variable when they were, in fact, not. Types of Experimental Design Posttest Only After randomly assigning the sample to treatment and control groups, researchers need to measure the dependent variable either during or after the participants’ exposure to the independent variables. This type of research design—the posttest only, or simple comparison— allows a researcher to conclude that any significant differences found are due to the fact that the treatment group received some different stimuli or that the treatment group received a stimulus that participants in the control group did not. Thus, this type of design answers the questions, “Do treatment groups differ based on different manipulations of the stimulus?” or “Do the two groups differ after the stimulus is presented to only one group?” Pretest–Posttest By adding one step to the posttest-only design, a researcher achieves the pretest– posttestexperimental form. Here a researcher measures the dependent variable before the treatment group is exposed to the stimulus. After the stimulus is given, the dependent variable is measured again in exactly the same way with the same participants. In some written research reports, the researcher refers to these measurements as Time 1 and Time 2. Time 1 is the measurement before any stimulus is given to the treatment group, and Time 2 is the measurement after the stimulus. Adding the pretest, or Time 1, measurement allows the researcher to determine the degree of change in the dependent variable from one measurement to the next and to make more definitive assessments about the influence of the treatment. Although many researchers agree that this experimental form is more powerful, there is one caveat. Because measurements at Time 1 and Time 2 are conducted in the very same way, there is some danger that participants become overly Page 120sensitized to the measurement, especially when data are collected through questionnaires. Two different effects can occur. In the first, participants may try to respond at Time 2 as they responded at Time 1, or, in other words, try to be consistent even though a change has occurred. In the second, participants assume that the researcher is looking for differences and may try to make sure they answer differently at Time 2 than at Time 1. In either case, the participants’ motivations or expectations can confound the research design. Factorial Design In a factorial design experiment, treatment groups are based on two or more independent variables. Researchers use this type of research design to investigate complex cause–effect relationships that cannot be adequately tested with only one independent variable. A factorial

design allows the researcher to test for the effects of each independent variable. In addition, he or she can test for the interaction effect, or how the independent variables can combine to influence the dependent variable. Factorial designs allow researchers to test for the treatment effects of each independent variable separately, as well as for the joint effect of the two independent variables together. In research reports, the simple influence of one independent variable by itself is referred to as a main effect. In other words, the influence of one independent variable is examined without considering the influence of the other independent variable. A researcher can also examine the interaction effect, or how the dependent variable is jointly affected by the two or more independent variables.

In a factorial design, the minimum configuration requires two independent variables with at least two levels each. But the design can be extended to consist of several factors with as many levels as necessary and feasible for each one. Generally, no more than four independent variables are built into one factorial design. Likewise, seldom are more than four levels of an independent variable considered. When a factorial design is extended to more than two variables, the number of interactions increases as well. Longitudinal Designs When experiments of the types just described are designed so that there are multiple measurements of the dependent variable, the design is also labeled as longitudinal. Researchers often call for longitudinal experimental designs as they discuss the limitations of their research in the discussion section of the written journal report. Strengths of Experimentation The primary advantage of experimentation is that researchers can manipulate the independent variable to observe changes in the dependent variable. Because the researcher controls the manipulation, and because participants are randomly assigned to treatment and control groups, any significant difference found is assumed to be the cause of variations in the independent variable (de Vaus, 2001). Other primary advantages of experimentation are that the design and researcher control over variable manipulation allows testing of extremes and multiple replications. Operationalizations and measuring techniques are decided on in advance, and a researcher repeats the procedures and methods in exactly the same way for each participant or each group of participants. This leads to precision (Kerlinger, 1986). A less obvious benefit is

that when the lab environment remains static, experimentation can be cost effective and more convenient for the researcher. After a lab environment is secured and outfitted for a particular experiment, the researcher uses it over and over. Seldom can all participants of an experiment be tested at the same time. Limitations of Experimentation Despite its strengths, experimentation has several limitations. Not all communication scholarship can be conducted using one of the experimental forms. In some cases it would be immoral or unethical to conduct the research because subjecting participants in one group to some stimuli could be negative or hurtful (de Vaus, 2001). Legal standards might also make experimentation impossible. Thus, moral, ethical, and legal issues prevent a researcher from assigning participants to some categories or from otherwise manipulating an independent variable. In other cases, it is impossible to manipulate the independent variable because these qualities are fixed in the participants. For instance, variables such as sex, socioeconomic class, and age cannot be manipulated. These characteristics and attributes are fixed within participants; the researcher has no control over them. Thus, the first limitation is that experimentation by the nature of its design characteristics will be inappropriate or impossible to use in all situations. The second major limitation is that even with its stringent design, experimental forms cannot guarantee that some factor other than the treatment factor produced the significant effect (Sapsford & Jupp, 1996). Researchers can control or minimize the influence of variables they know about. But researchers can never know if they succeeded completely, and they certainly cannot control the influence of a variable that is unknown to them. Third, laboratory experiments rely on a researcher’s manipulation of the independent variable. In some cases, such pure manipulation may not exist in reality. Thus, participants may react to the artificiality of the lab environment as well as to the potentially artificial manipulation of the independent variable. Another limitation to experimentation is derived from one of its strengths. Experimentation is hailed because it allows researchers to test several variables or combinations of variables at once. Although the variables can be manipulated and measured in the lab, this activity does not always equate with how those variables exist and interact in natural communication settings. The most frequent complaint about experimentation is the lack of reality in experimental designs. Conducting research in sterile laboratories, available classrooms, or other environments unusual to participants could influence how participants will respond or react, and that may be different from how they would behave in more natural surroundings or environments. In general, then, experimental research, especially that conducted in laboratories, may be investigating communication behaviors that are less complex than they are in the communication environments in which they are found day to day (Miller, 1970). Other scholars, however, suggest that lab experiments should not attempt to re-create natural interaction. Rather, artificiality is necessary to reduce or eliminate all the confounding variables that exist in the real world (Pedhazur & Schmelkin, 1991). QUASI-EXPERIMENTS Because of the limitations mentioned earlier, sometimes researchers rely upon natural variations in the independent variable. Called quasi-experiments, or natural experiments, this

alternative form of research design is possible because some variation in the independent variable exists naturally. In other words, participants are not assigned randomly to treatment and control groups. Lacking this assignment opportunity, variation in the independent variable is not under the control or direction of the researcher. The three basic designs of posttest only, pretest–posttest, and factorial, can still be used as long as natural variation of the independent variable can be substituted for manipulation of the independent variable.

Thus, the goal for researchers using quasi-experimental forms is to create and communicate clarity about the differences d...