Lecture 6 - Instructor: Timothy Peterka PDF

Preview

Summary

Instructor: Timothy Peterka...

Description

FEB 1 Lecture 6 Measurement (cont.) Improving Measurement If unsure a single measure achieves 4 criteria (Operationalizations are not observable, 1 question, etc.) → Triangulation Can take multiple measurements/questions We can get closer to the “true” value Can raise content validity Multiple Questions or Multiple Measures “Multiple questions to create one measure” Want to better measure the construct In reality, each survey question is its own measure Just measures different components of that construct We use multiple measure to get a better overall measure of the construct of interest Multiple measures also applies outside of survey research Ex. Perez - Lost in Translation? Hypothesis: Expects that linguistic differences between items compromise their cross-language validity (He expects them to be different between English speaking people and Spanish speaking people) 4 “agree-disagree” questions about specific traits that an immigrant must hold in order to be considered part of the American society. (American Identity = )

-

-

Goal: to determine if the items (each question) work the same for both groups Would like the items to be “invariant” across groups Perez wants to see if each item for each construct is the same across groups Concept: American Identity Tested 5 Constructs

-

Survey Items (Questions) to measure American Identity “Do you identify yourself as an American?” On a scale from 1 (do not identify as an American) to 5 (definitely identify as an American), where would you place yourself? Problem w/these questions People don’t want to be seen in a negative light (over-reporting/stating) Scale is subjective Questions could be targeted at a certain group Too vague Not enough content

-

-

-

-

-

-

Translating Survey Items Standard method: Double Blind Back translation Write question in English → Translate into Spanish → Re-translate into English Implications Importance of functional equivalence = the idea that the survey items work the same across language groups If different We are measuring different things It’s not a valid measure of the construct Example shows multiple questions (measures) to better measure a construct Example of analyzing measure validity of survey items → thus the overall measure as well Growth of spanish-speaking population Used surveys to measure/analyze political attitudes all the time Why bother? Concerned about validity of measures Want to measure the same thing across groups Are questions comparable across languages? Depends on whether English or Spanish is the primary language Problem w/just 1 question Is it just one item that’s hard to translate or is it a problem w/a bunch of items? Findings Only 1 item (votes in elections) is invariant across both groups ¾ items are not functionally equivalent (comparable) across English-and Spanish-speaking Latinos Items don’t actually measure the construct of American Identity very well across groups If using these measures in analysis, we should be suspect of the results “Taken as a whole, this investigation reveals that insofar as our tests and constructs are concerned, item validity is weak in US bilingual political surveys” Confounding Variables? Differences in income? Differences in education? → Perez holds confounders constant → Compares English and Spanish speakers who have the same level of income and education Low education High education Low income High income → findings hold even after this Summary Concern = validity of survey measures Compared validity in survey items across English and Spanish speaking Latinos Finds differences across groups on ⅘ constructs → item validity is a problem Items don’t match up w/theoretical construct **Item validity is an ongoing problem in the world

Why do we not use multiple measure all the time? Expensive Labor intensive...