PSYC2001 - Chapter 6 Surveys and Observations PDF

Preview

Summary

Lecture notes, textbook notes, practice questions and quiz questions...

Description

Chapter 6 - Surveys and Observations Surveys and polls: a method of posing questions to people on the phone, in personal interviews, on written questionnaires, or online - If done well, there is evidence of construct validity and can support all three claims - depends on how well the questions are worded (based on wording people answer differently) Choosing questions format 1. Open-ended allows respondents to any way they like Ex: What do you like about this course? Ex: What does love mean to you? Ex: What goal are you pursuing over the next few months Advantage: rich and spontaneous data Disadvantage: responses need to be coded and categorized (time-consuming and difficult) 2. Forced choice people give their opinion by picking the best o two or more options Ex: Do you like this course (true/false) Ex: How often do you read the textbook materials for this course? (sometimes/never) Advantage: more focused than open-ended Disadvantage: option someone wants to choose might not be there ** options actually related to what the respondent will be thinking or feeling/include an “other” option 3. Likert rate a statement based on the degree of disagreement or agreement on a continuum

Advantage: gathering information about the extent of an attitude Disadvantage: people might have recollection bias / affected by differences in response styles 4. Semantic differential Rate a target object (person, idea, thing) using a numeric scale that is anchored with adjectives Ex: ratemyprofessor.com; Target = professor scale of easiness Easy 1 2 3 4 5 Hard Advantage: More descriptive and meaningful endpoints than a typical Likert scale ↳i.e., not just asking about the extent of agreement Disadvantage: Not all ratings have clear opposite adjective points Points/labels on the scale can affect ratings

Chapter 6 - Surveys and Observations Writing well-worded questions The way a question is worded can make a huge difference in how people answer it When interrogating a claim question how well were the variables measured? (construct validity) 1. Wording/Leading questions wording leads people to a particular response because it explains why someone should respond in certain way → questions should be as neutral as possible Ex: Do you agree that it is a terrible idea to legalize marijuana? ↳ Should we legalize marijuana? 2. Double-barreled questions ask two questions in one → questions should be as simple as possible (possible agreement to part of the question) Ex: Do you enjoy long walks and taking exams? ↳ Do you enjoy taking long walks? + Do you like taking exams? Ex: When I am hungry I often eat bad food and drink pop 3. Negatively worded questions question contains negative phrasing which causes confusion Ex: Does it seem possible or does it seem impossible to you that the Nazi extermination of the Jews never happened? - work working memory or motivation to pay attention rather than the variable Question order can influence the way people answer to later questions Solution: randomize question order ↳ make sure different sections of the survey have question in a different order Encouraging accurate responses - People do give meaningful responses - People accurately report their gender identity, socioeconomic status, ethnicity etc. - don’t need expensive tools to assess these Sometimes self-report is the only option - Dreams, feelings of anxiety, motivation..etc People sometimes use shortcuts… The response sets: adopting a consistent way of answering all the questions, especially at the end of a long questionnaire 1. Acquiescence (yea-saying) is saying “yes” or “strongly agree” to every item - Solution: reverse worded items (but this can also be confusing and negative, might be tapping into working mem instead) 2. Fence sitting is answering in the middle of a scale - Solution: remove the middle option or use forced-choice People try to look good… 1. Socially desirable responding (faking good): give answers that make them look better than they really are - Solution: - Make the survey anonymous (take it seriously) - Detection items (meant to capture this type of responding) - Ask friends or family to rate the participant - Evaluate implicit attitudes about sensitive topics - include filler items on the survey that mask the true purpose of a sensitive survey

Chapter 6 - Surveys and Observations 2. Faking bad: gives answers to make them look bad (more aggressive/deviant) → not common People self-report more than they know… Give justification for their answers even if they don’t really know why People self-report memories of events Memories aren’t as accurate as they think Confidence and accuracy are unrelated Making Reliable and Valid Observations Construct validity of observational data can be threatened by… 1. Observer bias is when observers’ expectations influence their interpretation of the participant’s behaviours or the outcome of the study - observer sees what they want to see Ex: Groups watched the same video of a man interacting with a professor. One group told he was a job applicant. Another group told him he was a patient - #1 described him as “attractive”, “candid”, and “innovative” - #2 said “tight, a defensive person” “frightened of his own aggressive impulses” 2. Observer effects (expectancy effects) when observers inadvertently change the behaviour of those they are observing Ex: Clever Hans 3. Reactivity is a change in behaviour when participants know the other person is watching - solution 1: blend in Unobtrusive observations: making yourself noticeable - solution 2: wait it out - solution 3: measure the behaviour’s results Ethical considerations Is it ethical to observe the behaviours of others without their permission? Public: people expect to be observed Ex: at a store Private: people are not expecting to be observed Ex: two-way mirror ACTIVITY Does a political psychologist wish to measure the percent of people who voted in the last election in a specific rural riding? Self-repot: people can accurately report if they voted or not Observation: voting records could also be an accurate option A psychology student wants to measure how often people sent texts while driving in a local neighbourhood Observing a sample of drivers Self-report would be particularly concerned with response bias (look good) A relationship researcher wants to measure how much jealousy women feel in their relationship with their partners Self-report because jealousy is a subjective feeling

Chapter 6 - Surveys and Observations PRACTICE QUIZ QUESTION Joseph wants to find out what customers at his restraint think about the food and the service. He creates a survey for diners to fill out at the end of their meal with three questions: 1) Please rate the food from 1–10, where 1 is yucky and 10 is delicious; 2) Please rate the service from 1–10 were 1 is unacceptable and 10 is outstanding. What kind of question format is Joseph using? semantic differential Which of the following describes a fence-sitting response to a survey? responding to a controversial question on a survey by selecting the response right in the middle Jenny recently learned of plans to cut down an old beautiful tree on her campus to make way for a new bike path. Jenny is opposed to cutting down the tree, and she decides to survey some students at her university to see if others also oppose cutting down the tree. She plans to share the results of her survey with the school administration to argue to keep the tree. One question on Jenny’s survey asks, “Would you be in favor of brutally cutting down this majestic tree to make way for a stupid bike path?” leading question Judy writes a survey to assess how much people worry. Her survey has 20 questions that people can rate their level of agreement to on a seven-point Likert scale. All of the questions are worded so that higher responses will indicate a higher level of worry. After having 100 people complete her survey, she finds that a lot of respondents often respond to all of the questions by choosing only the “strongly agree” option. How could she improve the construct validity of her survey? Include reverse-worded items The campus safety committee has asked Professor Ibrahim to study bicycling on his campus. He trains two observers and has them observe the number of cyclists and their safety at various points around campus. When the observations began, the observers noticed that bicyclists slowed down when they neared the observers. How could this reactivity be avoided? The observers could make unobtrusive observations by hiding Which of the following is NOT a way to control for socially desirable responses? remove the neutral option from the survey so that a person must choose one side or the other The following item appears on a survey: On a five-point scale, where 1 Strongly disagrees, 2 disagrees, 3 is Neither agree nor disagree, 4 is Agree, and 5 is Strongly agreed, rate the following statement:”I look forward to coming to class. What type of question format is being used? Likert scale The campus safety committee has asked Professor Ibrahim to study bicycling on his campus. He trains two observers and has them observe the number of cyclists and their safety at various points around campus. Initially, the observers can’t agree about what is dangerous behaviour. Professor Ibrahim refines his codebooks to clearly define the rating scales and retrains the observers. the reliability of the measurement Jenny recently learned of plans to cut down an old beautiful tree on her campus to make way for a new bike path. Jenny is opposed to cutting down the tree, and she decides to survey some students at her university to see if others also oppose cutting down the tree. She plans to share the results of her survey with the school administration to argue to keep the tree. One question on Jenny’s survey asks, “Do you oppose not cutting down this tree?” A problem with this question is that it is a negatively worded question The campus safety committee has asked Professor Ibrahim to study bicycling on his campus. He trains two observers and has them observe the number of cyclists and their safety at various points around campus. Both observers are very interested in the topic because they have been struck by bicycles. Although the interrater reliability is high, what threat to construct validity should concern Professor Ibrahim? observer bias QUIZ QUESTIONS

Chapter 6 - Surveys and Observations A study by Rosenthal and Jacobson (1968) involved telling teachers that some of their students were "bloomers" and would achieve rapid academic success within the next year. In fact, these students were no different than any of the other students in the class. At the end of the year, the "bloomers" showed more gains in IQ than the other students. It appeared that the teacher had unintentionally treated the "bloomers" in special ways. This is an example of which of the following? Observer effects If a question has response options such as strongly agree, agree, neither agree nor disagree, disagree and strongly disagree, this is known as a Likert scale Dr. Paul is concerned about a fence-sitting response set when he conducts his survey. Which of the following might you recommend to decrease fence-sitting? Using scales with an even number of response options How do reverse-worded items address shortcuts? They slow down readers, making them answer more carefully Another word for observer effects is expectancy effects Unobtrusive observation is done to counteract which of the following? Reactivity In developing a measure of "need for cognition" (the degree to which people like thinking and problem-solving), Dr. Jonason asks his participants to rate their agreement with the following statement: "I frequently solve and enjoy solving crossword puzzles and Sudoku puzzles." What is the problem with this question? It is a double-barreled question Which of the following is a disadvantage of using open-ended questions? The answers must be coded Dr. Gahan decides to create a questionnaire asking about people's attitudes toward immigration (a socially sensitive topic). He should be most concerned about which of the following? Fence sitting Faking good is also known as socially desirable responding

Dr. Ewell, a developmental psychologist, is planning on conducting a study that involves watching children play together to determine how sharing behaviour occurs in same-sex friend pairs compared to opposite-sex friend pairs. - Dr. Ewell is concerned that the children will behave differently because of the presence of research assistants. Which of the following could he do to address this concern? Hide a video camera in the daycare center and record the children playing without them knowing - Imagine that Dr. Ewell wants to videotape the children interacting in the park. Why would most psychologists have no problem with the ethics of Dr. Ewell's study? People in a public park do not expect their behaviour to be private - Dr. Ewell decides to collect his data at a neighbourhood park. He has his two research assistants pose as a married couple having a picnic. While having their picnic, they take detailed records of the sharing behaviour of the children and note whether the pairs are the same-sex or opposite sex. This technique is known as unobtrusive observation...