PSYCH 310 - Research Design & Analysis PDF

Preview

Summary

Download PSYCH 310 - Research Design & Analysis PDF

Description

PSYCH 310 - Research & Design Wednesday, January 8 What is science From the Latin scientia Meaning KNOWLEDGE! “The systematic knowledge of the physical or material world gained through observation and experimentation.” Where do we get knowledge? Tradition: I believe it is true because it has always been that way Intuition: I believe it is true because it is a gut feeling Authority: I believe it is true because an “expert says its true” Empiricism: I believe it is true because I measured it Reasoning: I believe it is true because it is logically derived Rational thought - thinking with reason Three parts: Major premise: all BYU students are smart Minor premise: Cosmo is a BYU student Conclusion: Cosmo is smart Science - a way of acquiring knowledge the continual interaction of empiricism and reasoning Reasoning - I believe it is true because it is logically derived Empiricism - I believe it is true because I measured it The Process of Using Science Objectives of science Tenets of science Objectives of science To Describe - description of the subject matter, i human development the subject matter is human or group behavior and mental processes To Explain - explain the trends that have been observed To Predict make predictions from the explanation If we can describe and explain something, then we can try to predict If predictions are not confirmed, then the explanation is considered faulty and must be revised To Control - attempt to control and apply the phenomena If we can describe the event, explain why that event happens, and reliability predict the occurence of an event, we can try to increase or decrease the likelihood of the event The Tenets of science

Determinism Belief that events have natural causes not predeterminism - predestination Empiricism Reliance on real evidence to confirm or refute claims Replicability - findings must be replicated before they are accepted Falsifiability - hypotheses and theories must be falsifiable/refutable through empirical research Parsimony - simplest explanation for a phenomena is usually the correct explanation Research! The diligent and systematic inquiry or investigation into a subject in order to discover or revise facts and theories The Scientific Method Assume a natural cause or the phenomenon Make an educated guess about the cause Test your guess Revise your hypothesis Re-test your guess Make a conclusion Theories, concepts, and hypotheses Theory - a formal statement of how concepts are related Data - sets of observations Hypotheses - a prediction of how concepts are related that is often deduced from a theory Friday, January 10 Review Quiz #2 Which part of a journal is a comprehensive summary of the article and describes what was done, to whom, and what was found? Abstract 3) C Critical Thinking Critical thinkers: Avoid oversimplifying Consider alternative explanations Tolerate uncertainty Maintain an air of skepticism but be open minded (i.e. not cynical) n of one fallacy - drawing conclusions/ generalizations from anecdotal evidence Why we do research To demonstrate a new technique

Determine if new ways of doing things is better Approaches to research Descriptive vs explanatory research Descriptive research involves trying to answer “what” questions such as “What is going on?” or “What does a population look like?” Explanatory research involves trying to answer “why” questions Descriptive Research Gallup Polling found that “In the US, Majority Overweight or Obese in all 50 States.” West Virginia 69.3% Mississippi 68.8% Kentucky 67.2% Iowa 66.6% North Dakota 66.5% Oklahoma 65.6% South Caroline 65.6% Arkansas 65.5% Alabama 65.3% Indiana 65.3% Approaches to research Quantitative vs qualitative research Quantitative research measures differences in amount of behavior Qualitative research describes differences in kind or quality of behavior Quantitative: When implementing an afterschool program for troubled youth, is there a decrease in their negative behavior? Qualitative: What types of behaviors do children in the foster system exhibit when they move to a new home? Basic vs Applied research In basic research the researcher is interested in answering a question that may not have immediate benefit to humankind In applied research the researcher is looking at applying the knowledge to somehow benefit humankind The Process of Becoming “Research Literature” Dissemination of findings Conferences Poster Presentation Talk Journals Peer Reviewed Journals Theoretical or Review Article

Empirical Article Books Once the manuscript is written Submit to a journal Editor sends it to 2-4 reviewers Reviewers read the manuscript and make comments in a “timely” fashion Scientific research is Peer reviewed - process whereby editor of a journal sends submitted manuscripts out for review by other researchers in same field of study Reviewed blind (?) - reviewers are not aware of the authors(s) they are reviewing and author does not know who the reviewers are This process helps to maintain a high standard of quality in research Editor reviews the comments and make decision Accept with no changes Accept with minor changes Revise and resubmit Reject Sends the decision to the author Searching the Literature Bibliographic databases Proquest ERIC Psych INFO - produced by the APA and widely used for searching psychological literature Monday, January 13 Personal Health Project Keep tack on the note sheet (or yout option) From TODAY through Monday, March 2nd Executive Summary paper due Friday, March 6th Worth 70 points

Types of Variables Measured - something that is observed or recorded Manipulated - something that the researcher controls Conceptual Variable - an abstract concept, hard to measure

Operational Variable - a variable that can be measured clearly Hypothesis A hypothesis is an educated guess Three levels of hypotheses Conceptual EX: Absence makes the heart grow fonder Research Statistical EX: State the expected relationship between or among summary values of populations, called parameters Wednesday, January 15 Challenging Validity Claims When you hear about a study, first questions should always be How did they measure…? What was the population? (n, demographies) What analysis did they use? Types of Claims Frequency Proportion or count E.g. how many students are international Association Variables are related, correlated, etc Positive Association - study time and test score Negative Association - party time and test score Zero Association - GPA and dance skills… or is it? Causal Variable A causes change in Variable B Validity Refers to the appropriateness of a conclusion or decision Has to do with being reasonable, accurate, justifiable Construct validity - are you measuring the real variable accurately? External validity - how well does it generalize out? Internal validity - the study’s method to ensure that A is related to B, and there is no other plausible alternative explanation Statistical validity - were the claims/results accurate Face validity - whether the measure seems to be a reasonable measure of a variable Content validity - the degree that a measure assesses all the dimensions of the construct

Content Validity Say we want to measure the quality of the recreational facilities on campus We assess the following Overall cleanliness Number of free weights Number of elliptical trainers May not have content validity! Location Number of pools Rock climbing wall Quality/upkeep of equipment Staff professionalism

ETC!

Inferential Errors:

Our prediction:

Groups different No difference

True State of Affairs: No difference Groups different Type 1 Error (false positive) Correct inference Correct inference Type 2 Error (false negative

Power and how to increase it Ways to increase power: Be careful about how you measure your variables Use more powerful statistical analyses Use designs that provide good control over extraneous variables Increase your sample size -> reduce error variance Maximize treatment manipulation Testing your hypothesis We quantify this “likelihood” through a p-value Generally we say that p< .05 is a significant difference Importance of precision in measurement Precision is the “Exactness” of your measurement In descriptive research, precise measurements can improve the accuracy of our description of a population When testing a research hypothesis, precision increases our chances of finding a statistically significant result Measures should be as precise as possible EXAMPLE: How much do you weigh? 150 lbs Is there a problem with setting it up this way? What should you ask?

Is there a better way of getting reliable data than asking? Validity Validity - Actually measuring what you’re intending to measure Remember internal validity? Friday, January 17 Ethics Because ethical decisions are rarely black and white, we need to carefully consider all the possible ethical problems that might arise from our research Institutional Review Board (IRB) - mandate is to apply guidelines to assess the ethics of research proposals General Principles Principle A: Beneficence and Nonmaleficence Maximize benefits and minimize harm to participants Principle B: Fidelity and Responsibility Be dependable, faithful to and honest with your colleagues, the community within which you conduct research, and the people you study Principle C: Integrity Strive to be truthful, accurate, objective and fair in all parts of the research “Doing what is right even when it is difficult” “Doing what is right even when nobody is watching” Principle D: Justice Ensure your limitations and biases do not hinder or cause others to hinder the rights of all people to be treated fairly and equally Principle E: Respect for People’s Rights and Dignity Ensure the safety and well-being of participants, particularly children and those with special needs Tuskegee Experiment Medical experiment conducted in the United States between 1932 and 1972 Approximately 400 black men with syphilis were recruited and they told they would receive “treatment” for bad blood The researchers (US Public Health Service) actually was looking at the long-term effect of syphilis on the body Men allowed to suffer through pain of prolonged syphilis even after penicillin became known as a treatment Friday, January 24 Research Assignment Examples

Research question: Do early public speaking opportunities relate to later increases in social confidence? IV: public speaking opportunities in childhood DV: social confidence in adulthood Research question: Does a smaller amount of sleep result in worse motor skills? IV: amount of sleep a person gets in one night DV: ability to trace a picture after one week of modified sleep schedule

Research question: If men knew about available resources, then would they be more likely to get help? Depression or eating disorder IV: amount of resources available DV: help received, if they seek help Monday, January 27 Ethical Standards 2. Boundaries of Competence With conducting research, you need familiarity with the area (through literature, coursework and/or consultation) If your topic involves participants from specific groups (e.g. Arab immigrants) you need training to understand these individuals/groups before you begin your research 3. Human Relations 3.01 Unfair Discrimination Should not treat people differently solely on the basis of some personal characteristic 3.02 Sexual Harassment Need to guard against abuses of the power you have as a researcher 3.03 Other Harassment Be careful in your actions toward and with your words to your participants and colleagues 3.04 Avoiding Harm Need to consider the relative cost versus the benefits of the research 3.05 Multiple Relationships Important to remain independent of participants 4. Privacy and Confidentiality 4.01 Maintaining Confidentiality Any information gathered in your research should remain confidential 4.02 Discussing the Limits of Confidentiality Need to clearly explain the limits of your confidentiality and under what conditions confidentiality can be broken Typical Limitations to Confidentiality

1. 2. 3. 4.

Suspected suicidal ideation Suspected homicidal ideation (or other significant intent to hurt someone) Potential abuse of children, the elderly, or disabled A court order by a judge

8. Research and Publication 8.01 Research and Publication Need to provide accurate information about your proposed research to the IRB/ERB 8.02 Informed Consent to Research Need to include an informed consent form that provides a brief description of the purpose of your study and what the participants will be expected to do 8.03 Informed Consent for Recording Voices and Images in Research Need to ask permission to record or photograph research participants 8.04 Client/Patient, Student, and Subordinate Research Participants Participation in research should be voluntary 8.06 Offering Inducements for Research Participation You my offer a small reward for participation but it is problematic if the incentive is too large (avoid coercion) 8.07 Description in Research Deception should be avoided if possible. If deception is used, be sure to reveal the purpose and importance of the deception when you debrief your participants 8.08 Debriefing After your research participants have completed the study, you must debrief them. Allows for the opportunity to explain details of the study not included in the informed consent and for participants to ask questions

Wednesday, January 29 Three R’s in Animal Studies Replacement - find alternatives to animals when possible (e.g., computer simulations) Refinement - modify experimental procedures to minimize or eliminate animal distress Reduction - use designs that require the fewest animals possible Ethical Standards 8.10 Reporting Research Results Need to report accurate data and results. If an error is discovered, the researcher has a responsibility to publish a correction

Internet Research Ethical guidelines have not been fully developed Problems particularly surround informed consent and debriefing (as you cannot ensure either has occurred) When doing Internet research follow the general principles and guidelines of the code as best as you can Friday, January 31 Practice/Review Question For a measure to truly be valid, it needs to be reliable - TRUE Reliability is how consistent the results of the measure are Validity concerns whether the operationalization is measuring what is supposed to measure Validity of measurement Face validity Whether the measure seems to be a reasonable measure of the variable Construct validity Whether the measure is measuring the underlying construct (operational definition of construct) Determined by how well the measure of a variable fits into a theory Content validity The degree that a measure assesses all the dimensions of the construct Criterion validity How well your measure is correlated with a relevant behavioral outcome Convergent and Discriminant Validity How similar or dissimilar measures are

Known Groups Paradigm One way to test for criterion validity Examining whether scores on the measure can discriminate among two or more groups whose behavior is already confirmed/diagnosed What examples can you think of? Using Reliability and Validity Developing tests, scales and inventories! Collecting data Surveys that have many “items” When you put multiple together that have the same content you can create a test or scale

Tests, scales, and inventories Test - refers to many procedures used to measure a variable Intelligence test, aptitude tests, achievement tests Inventory - used to describe interests or personality characteristics, but 47.3 doesn’t make sense Monday, February 3 Choosing Question Formats Open-ended questions Must be coded and categorized Not as efficient, but gives rich info Forced-choice questions Easier to get usable data But are you missing something? True/False or Yes/No Set/options Fixed Alternative (Multiple Choice) Rating or Likert Scale Scales Rating Scales - fixed alternative questions where the respondent indicates magnitude on a scale Likert Scales - statements in which the respondent is asked to indicate the degree he or she agrees or disagrees with the statement Writing Good Questions Question Wording Matters Leading Questions Double-Barreled Questions How satisfied are you and your spouse and marriage? (Scale of 1 to 7, very dissatisfied to very satisfied) Negative Wording Question Order Encouraging Accurate Responses Issues with Response Set Acquiescence (it’s all good!, or bad) Fence Sitting (I can’t decide… hmm…) Socially Desirable Responding Faking Good Faking Bad Accuracy of Self-Reporting

Self-Reporting More Than You Know Don’t set them up to have to answer more than they may know Self-Reporting Memories These can be very bias and are less accurate the older they are (like eyewitness testimony) Can still have a place in research, you just need to be aware of limitations Wednesday, February 5 Power and how to increase it Ways to increase power: Be careful about how you measure your variables Use more powerful statistical analyses Use designs that provide good control over extraneous variables Increase your sample size -> reduces error variance Accuracy and Biased Results Observations vs. Self-Report Observations are less bias to social desirability or participants influence (if done correctly) However, they open up other types of bias Observer Bias - Type of confirmation bias where the researcher’s expectations influence their interpretation of participants’ behavior Observer Effects - When researchers’ behaviors influence the behaviors’ of participants Encouraging Accurate Responses Masked Design (Blind Design) - participants don’t know purpose of the study, or they don’t know their group Grape juice study High protein weight loss study Reactivity - when participants know they are being observed and they change their behaviors Solutions Blend in (be unobtrusive) Wall it out (become normal background over time) Measure the behavior’s results (smudges) Wednesday, February 12 Sample Bias? Bias (unrepresentative) Easy to get Cheap, fast, convenient (Self-selection bias, etc)

Unbiased (representative Random Takes more work, time, and/or money Selecting research participants In research you select a sample from a population of potential participants Census - the whole population is researched Clinic settings - only individuals with a specific psychological disorder Schools - Could be at the macro school level (just looking at schools) or a more specific level of students Sampling frame - a list of the population from which the sample is drawn Sometimes we know the sampling frame, but other times we don’t Example: BMV/DMV What would be a sampling frame for this class? Sampling Methods Probability Sampling Allows us to make broader generalizations Random Sampling - a sample is drawn such that each member of the population has an equal probability of being included in the sample; uses a basic form of random selection from a population to get your participants Random sampling is different than random assignment! Sampling involves selection of overall participants Assignment involves selection to groups Systematic Sampling - the population size is divided by your sample size to provide you with a number , k, for example; then from a random starting point you select every kth individual

Non-probability Sampling Limited external validity, but can be used to test more specific theories It is impossible to specify the probability of selecting any one individual The sample may or may not be representative of the population Convenience sampling - using whatever participants are easily available Intro psychology class Local schools with all participants who consent Purposive-sampling - convenience sampling in which the goal is to select participants with particular characteristics Looking for only certain people due to your exact study parameters or goals Snowball sampling (or referral sampling) - involves including participants in the sample who have been referred by other participants Stratified Random Sampling Recruiting a certain percentage ...