A Beginner's Guide to Structural Equation Modeling, by Randall E. Schumacker, Richard G. Lomax PDF

Preview

Summary

Evaluation and Program Planning 28 (2005) 245–246 www.elsevier.com/locate/evalprogplan Book review Randall E. Schumacker and Richard G. Lomax, Unfortunately, learning to use SEM can challenge even A Beginner’s Guide to Structural Equation Modeling, the most mathematically skilled. Not only is the lo...

Description

Evaluation and Program Planning 28 (2005) 245–246 www.elsevier.com/locate/evalprogplan

Book review Randall E. Schumacker and Richard G. Lomax, A Beginner’s Guide to Structural Equation Modeling, second ed., 2004, Lawrence Erlbaum and Associates, Mahwah, NJ Is structural equation modeling (SEM) a mathematical Godsend or an instrument of the devil? Well respected researchers have staked a claim on each side of this debate, and I outline both positions briefly below. But no matter which stance one takes, SEM has acquired such a stature and prevalence in the social sciences that, for better or worse, evaluators of all stripes must come to grips with it. Even hard-core qualitative researchers who think statistical analysis is no more informative than numerology can benefit from an understanding of SEM. For example, consider reports of quantitative social science (such as that television viewing in infancy causes attention deficits in later childhood or that day-care fosters anti-social behavior) that frequently find their way to the front pages of the newspaper. Because such findings are often derived from SEM methods, a thoughtful critique requires an understanding of at least the rudiments of that methodology. Of course, SEM has more to offer than assistance in reading the newspaper. SEM can be used for a variety of purposes in program evaluation, including the following four. First, SEM can be used to assess the factorial structure of measurement instruments and to compare differences in factorial structure across groups of recipients. For example, SEM can be used to assess how the interrelationships of the items on a satisfaction survey differ across treatment conditions, genders, or ethnic groups. Second, SEM can be used to take account of selection differences in quasiexperimental or correlational designs. Without random assignment to treatment conditions, differences among the participants who receive different treatments can bias the estimates of the effects of the treatments. SEM provides a means to model the effects of the differences among the participants and thereby remove bias. Third, SEM can be used to model the mediational paths by which an effect is transmitted from a treatment to the outcome. For example, an evaluator can use SEM to determine how much of the effect that treatment X has on outcome Y arises because X affects Z which then affects Y and how much arises because X affects W which then affects Y. Fourth, SEM can be used to assess the separate effects of each of the components of a multi-faceted treatment. doi:10.1016/j.evalprogplan.2005.01.006

Unfortunately, learning to use SEM can challenge even the most mathematically skilled. Not only is the logic of SEM complex but its computational implementation can be a torture of trial and error. To make matters worse, there are usually no assurances that the method produces results that are relatively free from bias. And therein lies one of my primary complaints with the volume by Schumacker and Lomax. The most common use of SEM both in program evaluation and elsewhere is to assess causal linkages when the levels of the independent variable have not been assigned to research participants at random. When a researcher builds an SEM model under these common conditions, the estimates of treatment effects that are produced will be unbiased only if either (a) all the causal effects of the outcome variable are included in the model or (b) the effects of the causes that are omitted are uncorrelated (in the aggregate) with the causes that are included in the analysis. In practice it is impossible to satisfy either of these conditions, which results in the problem of ‘omitted variables’ with potentially devastating biases. Omitted variables can make a highly effective program look horrifically harmful or vice versa. Researchers who use SEM models need to understand the severe vulnerability of the method to omitted variables so that steps can be taken to reduce potential biases as much as possible and so that the results of SEM analyses can be interpreted in light of the biases that inevitably remain. For those who hold the belief that SEM is the devil’s work, it is the problem of omitted variables that is likely at the heart of their complaint. For my tastes, the problem of omitted variables receives far too little attention in the Schumacker and Lomax volume. To be fair to Schumacker and Lomax, most other books on SEM also fail to emphasize the problem of omitted variables. But that does not make the problem any less troublesome or important. I also have another reservation about the volume given its intended audience. The volume by Schumacker and Lomax covers all the traditional topics in SEM analysis beginning with data entry and ending with appendices on the representation of SEM models using matrix algebra. The volume’s presentation builds from simple regression models to path analysis and confirmatory factor models, before reaching latent variable structural models. The final two chapters address a full range of complex models including models for multi-trait multi-method matrices, latent growth curve models, dynamic factor models, multi-level models, and models with latent variable interactions, multiple groups, and

246

Book review / Evaluation and Program Planning 28 (2005) 245–246

structured means. Throughout the presentation, the volume explains the workings of three commonly used SEM computer programs (Amos, EQS, and LISREL). The volume also includes a CD containing data sets and worked examples. The drawback to such extensive coverage of models and software is that something else must necessarily receive relatively less attention. In the present case, I would have preferred less emphasis on encyclopedic coverage and more on pedagogical explications of the logic underlying the analyses. Although the title of the book targets ‘beginners’, I wonder if the book’s level of presentation is best suited for researchers who already have more ‘intermediate’ skills in SEM. Schumacker and Lomax obviously belong to the camp that believes SEM is more useful than misguided. The prototypical argument from that side of the debate acknowledges that SEM is imperfect but emphasizes that (a) random assignment to treatment conditions is often impossible, (b) it is nonetheless important to understand the causal impact of treatments under these circumstances, and (c) SEM is the best available method to assess cause and effect given these constraints. I sympathize with this perspective but only to the extent that its weaknesses are always kept firmly in mind when SEM analyses are conducted and its results are interpreted. It is becoming popular for qualitative researchers to argue that qualitative methods are superior to

quantitative approaches (such as SEM) for assessing cause and effect when randomization is not possible (and even when it is). Perhaps this is justified, perhaps not. But the problem of omitted variables (i.e. of not being able to assess all the causes of an observed outcome) cannot be avoided simply because the data are not numerical. Quantitative researchers need to remember the problem of omitted variables when they use SEM, lest their results be misleading. It would be a shame if qualitative researchers mislead themselves and their readers because they fail to learn this lesson from their quantitative brethren, who can teach it very clearly based on an understanding of SEM. In my view, the volume by Schumacker and Lomax doesn’t adequately emphasize the problem of omitted variables. In addition, the volume is directed toward a general social science audience more than focused on the uses of SEM that are most relevant to program evaluation. For these, as well as other reasons, the Schumacker and Lomax volume might be a better reference work than introductory guide for evaluators who are SEM novices. Charles S. Reichardt* Department of Psychology, University of Denver, 2155 S. Race, Denver, CO 80208, USA E-mail address: [email protected]

* Tel.: C1 303 871 3783; fax: C1 303 871 4747....