Introduction to Research Methods Exam Revision PDF

Preview

Summary

Research Methods Exam ReviewThe scientific approach (ch2)  Main principles: o Determinism, discoverability, obtaining knowledge through systematic observation, making data-based conclusions, tentative conclusions, asking empirical questions, developing falsifiable explanations  Goals of research: ...

Description

Research Methods Exam Review The scientific approach (ch2)  Main principles: o Determinism, discoverability, obtaining knowledge through systematic observation, making data-based conclusions, tentative conclusions, asking empirical questions, developing falsifiable explanations  Goals of research: o Description, prediction, explanation, application  Qualities of good research: o Reliability, validity, being public, being cumulative, parsimonious  Theory as a basis: o Predictions are developed from theory and serve to provide support (or falsify) theories Experimental design (ch3,4)  Main features: o Manipulating IV, measuring DV, controlling extraneous variables o Between- and within-subjects designs  Experimental method allow us to make cause-and-effect inferences o IVs are manipulated and the effect on DV is observed and recorded  Quasi-experimental studies are more limited o IVs are not manipulated, so extraneous variables are not controlled Experimental control (ch 4)  Types of validity: o Statistical conclusion, construct, internal, external  Threats to internal validity: o History, maturation effects; regression to the mean; testing and instrumentation effects; selection effects, mortality; experimenter bias; reactivity Experimental control (ch3,4)  Control in between-subjects designs: o Random assignment, blocked random assignment, matching  Control in within-subjects design: o Counterbalancing (complete, partial, Latin square)  Controlling experimenter and subject bias: o Double-blind procedures, deception, placebo Ethics (ch14)  Main sources of risk to participants: o Stress, breaches of confidentiality, deception, invasive procedures  Key principles to deal with these: o Minimizing stress and other risks at the design stage, informed consent, right to withdraw, debriefing

Survey (correlational) method (ch5)  Causality in survey research: o Directionality and third variable o Longitudinal survey design; partial correlations  Sample representativeness  Sampling methods: o Simple random, stratified, cluster, purposive, quota, snowball, convenience Qualitative methods (ch12,13)  Main features: o Data are words, data-driven theory and analysis, understanding subjective meaning and experience (phenomenology), understanding phenomena in context,  Inductive vs deductive approach  Thematic Analysis as a specific example of a qualitative method: o Organising data into semantic and/or latent themes o Phases and pitfalls in TA ------------------------

The scientific approach (ch2) T1/01 The scientific approach (ch2) T1/01  Main principles: o Determinism, discoverability, obtaining knowledge through systematic observation, making data-based conclusions, tentative conclusions, asking empirical questions, developing falsifiable explanations  Goals of research: o Description, prediction, explanation, application  Qualities of good research: o Reliability, validity, being public, being cumulative, parsimonious  Theory as a basis: o Predictions are developed from theory and serve to provide support/falsify theories T1/01 Lecture + Revision Lecture - Main principles of scientific approach o 1 - Determinism and discoverability o 2 - Systematic observation o 3 - Producing public (verifiable) knowledge o 4 - Producing data-based conclusions o 5 - Producing tentative conclusions o 6 - Asking answerable (empirical) questions o 7 - Developing falsifiable explanations (theories) - 1 - Determinism and discoverability o Everything has causes, causes can be discovered o Probabilistic determinism – events can be predicted but not 100% certain - 2 - Systematic observation o Definitions, measuring tools, research methods, logical system to draw conclusions - 3 - Producing public (verifiable) knowledge o 2+ researches o Replicate - Clear procedures & terms so ppl can repeat it - 4 - Producing data-based conclusions o Data (not opinions) draws conclusion o Questioning data - 5 - Producing tentative conclusions 暫時性 o Open to alternative findings - 6 - Asking answerable (empirical) questions o √ “Does belief in God increase age?”; X “Does God exists?” o Operationalization 操作性 pp66 

-

The process of deciding how to measure/manipulate IV and and how to measure DV

7 - Developing falsifiable explanations (theories) o Hypothesis deduced from theories o Falsifiable theories – ones that can be supported or not supported by data 

Falsification = refutation – the process of rejecting the conclusions and theories on basis of evidence that is inconsistent with them.

-

Qualities of good research o Valid – show what it ought to show, correct interpretation (accurate results) o Reliable – can be replicated, not just by chance (consistently reproduced) o Public o Cumulative o Parsimonious – explains all phenomenon  Accounting for the maximum number of empirical findings in terms of the smallest number of theoretical principles pp26

-

Theories ≠ facts

-

o

Theory – a system of explanation to make sense of and integrate a number of empirical findings

o

Many theories ≠good! The major goal is to explain as many facts as possible with the smallest number of theoretical principles.

Other ways of developing research o Observation o Serendipity  Being in the right place at the right time o Everyday probs that need solution (applied psych) o Replication and extension  Replication crisis – fail to support earlier research o Developing from others’ research

Experimental design (ch3,4) T1/02 Experimental design (ch3,4) T1/02  Main features: o Manipulating IV, measuring DV, controlling extraneous variables o Between- and within-subjects designs  Experimental methods allow us to make cause-and-effect inferences o IVs are manipulated and the effect on DV is observed and recorded  Quasi-experimental studies are more limited o IVs are not manipulated, so extraneous variables are not controlled -

Manipulating independent variable (IV) o IV types  Situational variable  Task variable  Instructional variable o Ways  Control VS experimental group  Random assignment  Indirectly manipulate  Manipulation check  Manipulation check – a dependent measure used to check that manipulation of an IV has been successful

 -

E.g. ,

Measuring dependant variable (DV) o Pilot study (pp122) – a preliminary study designed to “road test” various design elements (e.g., IV, DV, procedure, …)  Ceiling effect (high scores, i.e. too easy)  Floor effect (low scores, i.e. too difficult) o

Relevance-sensitivity trade-off  DV that are more sensitive to IV can be less relevant to the real world phenomenon (external validity)  

-

Sensitive – detect effect of IV, relevant – study area of interest

 External validity – generalise across populations, times, places  Ecological validity – generalise to today’s real life settings

Controlling EV extraneous variable o Confounding variable – a variable that has unwanted impact to DV o Extraneous variable – any variables that has a potential influence to DV  o

Ask “what has the researcher manipulate other than the DV?”

Control by identifying and changing experimental design

-

Quasi experiments o Self-selected IV (e.g., gender) (i.e. no random assignment) o Cannot manipulate & rule out extraneous  X causal conclusions o Matching pp77  Control for extraneous variables in quasi-experiments  The process of attempting to remove systematic differences between experimental groups on extraneous variables that are likely to confound results

Experimental control (ch 4) T1/03 Experimental control (ch 4) T1/03  Types of validity: o Statistical conclusion, construct, internal, external  Threats to internal validity: o History, maturation effects; regression to the mean; testing and instrumentation effects; selection effects, mortality; experimenter bias; reactivity -

-

-

-

-

Statistical conclusions validity o the degree to which conclusions about the relationship among variables based on the data are correct o Use correct stats test Construct validity o the extent to which the measurements used, often questionnaires, actually test the hypothesis or theory they are measuring o Operationalization (correct manipulation of IV and measures of DV) Validity o Theoretical framework VS naïve empiricism  (Findings  theory  generalisation) External validity o The extent to which results can be generalised beyond the context of the study o Types:  populations validity  random sampling  gender bias, cultural bias, student samples  ecological validity (place)  mundane realism – settings are similar to the real world  experimental realism - settings feel real to the participant  temporal validity Internal validity o The extent to which the study tells us what we think it tells us Forms of threats to Internal Validity o When study extends over time…  History effects  An event (other than the manipulation) occurs between pretest and post-test and affects the study outcome  E.g. study attitudes on terrorism, terrorist attack occurred  Control group  Maturation effects  Participants naturally change  E.g. moral reasoning  Control group  Regression to mean (people have bad days)  if the population performs extremely poorly at a pretest, it is likely to improve at a post-test independently from treatment

o

-

 Control group  Testing effect  Improvements Control group  Fatigue Control group  alert to manipulation (e.g. deception) and thus pay attention  ask for feedback  Instrumentation effect  Different measurements are used  Control group Participants-related threats  Selection effects  No random assignment  Mortality  Participants of certain characteristics drop out

No experimental design can overcome dishonesty (researcher cheating)

Other vocabulary - Concealment pp90 o Similar to deception, but rather than misleading P, give them limited info - Cover story o Info about the supposed purposed of the research that conceals its actual purpose

Experimental control (ch3,4) T1/03 Experimental control (ch3,4) T1/03  Control in between-subjects designs: o Random assignment, blocked random assignment, matching  Control in within-subjects design: o Counterbalancing (complete, partial, Latin square)  Controlling experimenter and subject bias: o Double-blind procedures, deception, placebo -

Between- vs within-subjects/participants Between-subject/participants

When use?

-

within-subjects/participants

-

Independent/ nonrepeated measures (different P in each condition)

-

Repeated measures (same P in each condition)

-

Self-selecting Features make it impossible (deception)

-

participants fresh/naïve to hypothesis More P needed Pre-existing difference

-

Short condition, long preparation Small population of interest More data on each P ⇩ error variance Internal validity Order effect

advant age

-

disadva ntage

-

Control in between-subjects designs: o Equivalent groups, random assignment o Blocked random assignment  advantages  Ensure each condition has equal number of P (each condition has a P before repeating)  Each “block” are in all conditions in a randomised order  Need to choose and measure one extraneous variable (may lead to confound), then accommodate these in the participantto-condition assignment process  Steps  Get a score for the variable  Arrange scores in ascending order  Create P pairs (block) with adjacent scores  Randomly assign P in each pair (block) to conditions  Conditions/ constraints  Choose the right variable to match (can only choose one, an extraneous + confounding variable)  Logical measurements of the variable  Large enough sample size (if not, increase sample size and no matching)

-

Control in within-subjects design: o Error variance  Aka residual error; residual variance; unexplained variance.  Variability caused by extraneous variables o Order effect  Practice effect (improvements)  Fatigue effect (reduce performance)  Carryover effect (effects of 1 condition carry over to the next)   counterbalancing (more than one sequence of conditions)  Complete VS partial counterbalancing  Complete: use every possible sequence at least once  Partial: use some of the total number of sequences o Randomly arrange each P’s sequence; or o Sample from a list of all possible sequence

-

Controlling experimenter and subject bias o Experimenter bias  Know the treatments and thus behave differently in the different groups  Double-blind study  Automate procedure o Participant bias  Hawthorne effect  Participants change behaviour because they know they are being studied  Demand characteristics  Demand characteristics (pp91) o Cues that convey an experimental hypothesis to P o e.g. “do you really want to do that” may lead to a “no”  Being a “good participant” - if participants guess the hypothesis, they may try to help confirm it   neutral instructions  Evaluation apprehension  performance anxiety - worried of being judged when observed in a study, and thus behave in a way that can be judged favorably by the experimenter  Acquiescence effect  Tendency to agree rather than disagree   deception  Keeping participants naïve to the purpose of the study   placebo  Condition where participants believe they’re getting a treatment, but in fact they aren’t

Ethics (ch14) – T1/08 Ethics (ch14)  Main sources of risk to participants: o Stress, breaches of confidentiality, deception, invasive procedures  Key principles to deal with these: o Minimizing stress and other risks at the design stage, informed consent, right to withdraw, debriefing Useful Vocabulary from Textbook and Revision Lecture - Confidentiality = limit disclosure of patient’s info o a principle of professional ethics requiring providers of mental health care or medical care to limit the disclosure of a patient’s identity, his or her condition or treatment, and any data entrusted to professionals during assessment, diagnosis, and treatment. Similar protection is given to research participants and survey respondents against unauthorized access to information they reveal in confidence. - Confederate = fake participant o Someone who works for the experimenter, act in a certain way in an experiment and pretend to be a genuine participant - Discontinuing participation o Right to cease participation. No need to inform the experimenter - Invasive procedures o Research procedures that leads to changes in P Lecture T1/08 General Ethical Principles Ethical Implications and Development of Ethical Principles - Tuskegee syphilis study 1932-72 o Aim to investigate the progression of syphilis o soldiers but weren’t told that they were infected, not offered treatment, ended up with harmful consequences o  1974: National Research Act passed by US Congress - Nazi ‘medical’ experiments o Investigated effects of nonhumane treatments o Debate: but their research benefit us? o  1949: Nuremberg Code  Specified ethical principles for research on humans - Milgram’s ‘obedience’ studies o Emotional side effects o Cannot be run full today: partcially replicated in 2006 - Stanford Prison Study o Guards VS Prisoners o Cleared by APA investigation in 1973 o partially replicated in 2002 for the BBC Current Principles - APA

o

Principle A: Beneficence and Non-maleficence  Benefit + no harm to P 

Psychologists strive to benefit those with whom they work and take care to do no harm. In their professional actions, psychologists seek to safeguard the welfare and rights of those with whom they interact professionally and other affected persons and the welfare of animal subjects of research.

o Principle B: Fidelity and Responsibility  Aware of responsibilities to the ones they work with + professional and scientific responsibilities 

o

Psychologists establish relationships of trust with those with whom they work. They are aware of their professional and scientific responsibilities to society and to the specific communities in which they work.

Principle C: Integrity  accuracy, honesty and truthfulness 

Psychologists seek to promote accuracy, honesty and truthfulness in the science, teaching and practice of psychology.

o Principle D: Justice  equal access to the knowledge and equal opportunity to be benefited 

Psychologists recognize that fairness and justice entitle all persons to access to and benefit from the contributions of psychology and to equal quality in the processes, procedures and services being conducted by psychologists.

o Principle E: Respect for People's Rights and Dignity 

-

Psychologists respect the dignity and worth of all people, and the rights of individuals to privacy, confidentiality, and self-determination.

BPS o Respect – rights, privacy, and self-determination 

Psychologists value the dignity and worth of all persons… with particular regard to people’s rights including those of privacy and self-determination

o Competence – high standard (thus long career pathway) 

Psychologists value the continuing development and maintenance of high standards of competence in their professional work

o Responsibility – to clients, public, profession, and science 

Psychologists value their responsibilities to clients, to the general public, and to the profession and science of Psychology, including the avoidance of harm and the prevention of misuse or abuse of their contributions to society

o Integrity - accuracy, honesty and truthfulness 

Psychologists value honesty, accuracy, clarity, and fairness in their interactions with all persons



Researcher-Participant Relationship - EU General Data Protection Regulation o Protect P from physical and mental harm o No greater-than-day-to-day-lives harm o best judge of whether a study will cause offense: members of the population o specific standards for non-humans and vulnerable - Informed Consent - General Data Protection Regulations (GDPR) o Unambiguous o Freely given o Informed

Specific Demonstrable Explicit for special categories (e.g., health, racial, religious, political) No slice or opt-out  Cannot be unless u voice out, u will be involved - Deception o Withholding information or misleading participants o Only when necessary o Debriefing - Debriefing o Eliminate possible harm (research risk = possible harm to P) o Doesn’t justify unethical procedures o Written statement with research purpose and researcher’s contact information - Ethics approval o By Psychology Research Ethics Committee o 2 forms of applications:  Track A: non-interventionist methods (e.g., observational) and no ethical concerns  chair OR committee member  Track B: Ethically-sensitive methods and/or subject  chair AND committee member Other Forms of Ethical Concerns - Cases in which ethical issues aren’t obvious o E.g. beneficial treatment withhold from some P o Legal ≠ ethical - Handling data o Fabricated data (e.g., Diederik Stapel) o Questionable research practices  Not report all dependent measures  Collecting more data after seeing whether results were significant  Not report all conditions  Only report results that “work”  Stop recording data after achieving desired result  Excluding data after looking at the impact of doing so  Claiming to have predicted an unexpected finding  Falsely claiming that results are unaffected by demographics o Promote open and transparent research  Pre-registration in open science network...