PPA notes PDF

Preview

Summary

Exam notes from the whole year - lectures, tutorials, readings. Lecturer was Dr. Emily Castell ...

Description

PPA Semester notes Week 1 Readings The Griggs Vs. Duke Power decision - all testing used for selection or promotion of employees must have specific meaning, validity. Evidence for validity comes from showing the association between the test and other variables. Convincing truth is needed. Validity is the evidence for inferences about test scores, there are many subcategories.  Face validity: based on mere appearance, looks right  Content-related: represents the adequacy of the representation that the test is supposed to cover. Learn about first 6 chapter, the test should cover those chapters  Criterion-related: how well a test corresponds with a criterion. High correlations between well-defined criterion measures. Pre-marital test stands in for estimating marital happiness. Predictive: LSAT testing predicting performance for how they will go at succeeding. Concurrent: do 2 tests at the same time to describe what is happening NOW. Give employees a job task, for a mechanic; obtain samples of his mechanic work.  Construct: series of steps where the researcher simultaneously defines the construct and develops an instrument to test it. Assemble evidence for it.  Convergent: when the test measure correlated well with other tests measuring the same thing  Discriminant: proven to measure something different, doesn’t correlate with other tests A psychological test is a set of items that allows measurement of some attribute of a person, regarding thinking, behaving, feeling or traits.

Lecture

Test = an instrument, a tool Testing = the process of administering that tool Assessment = broad sense of using these tests, testing as broad and contextualised A psychological test = a tool which measures any psychological construct A good test = valid, reliable, psychometric properties Uses of a test = diagnosis, screening, effectiveness of treatment, current state, use for intervention planning, learning outcomes, eligibility for certain work & employment, diagnosis to get certain funding from the government (IQ, autism), research, clinically When we use a test, we assume: - Measures what its supposed to - Score remain stable of time with traits, but mood will have variation - Items are clear and interpreted the same by all - Individuals will report accurately and honestly o Note, we cant guarantee this, motivation, anxiety, current state, ulterior motive, fake good/fake bad, seeking another outcome, - The test score = true score + error (systematic and random error) Evaluating a psychological test: - Is it reliable - Is it valid - Practical, feasible? - Does it meet my objective? - Are there published guidelines for the test? - Cost-effective and suitable - Results be generalisable and support inferences? Know:  Understand the operationalised definition of the construct you’re using, before you use it  Tests are tools, they are only useful if you use the right tool for the job  You are in control of the tool, tailor your assessment to the referral/research question Steps in test construction/development: theory/rationale approach 1. Specify attribute 1. What’s known about it? 2. Check literature for existing test

3.

4.

5. 6.

7. 8. 9.

1. Is there already an existing measure? Is it suitable for my population? Age, culture, format Choose a measurement model 1. We need this measure, so then we choose a model (nominal = categories; ordinal = ranked ordered; interval = equal distance between points, ruler; ratio: a true zero) Write and edit items 1. Consult the literature, theoretical constructs. You can create an item pool. Avoid jargon, double negatives, leading, double-barrelled questions. 2. Item analysis: factor analysis, item-total correlations, item-external criterion correlation, item discrimination, qualitative analysis Administer and analyse responses with piloting Check reliability and validity 1. Reliability: the precision of an instrument, how accurate/precise is our measure? Internal consistency, inter-rater, split-half, test re-test, alternative forms 2. .7 .8 for research, .9 for clinical purposes as smaller group 3. Associated with your particular sample 4. Validity:  Face validity – do they appear correct?  Content – does it measure the construct comprehensively? assesses all dimension of the construct not just one aspect?  Construct (interpreted with coeffeicients)  Convergent – is it associated with other measures measuring a similar thing?  Discriminant – depression measure does not correlate with anxiety measure, or at least a smaller magnitude of a relationship, is it unique?  Criterion  Concurrent – correlated with established measure  Predictive – predicts performance on a criterion Norm if found – gives your test scores meaning, the average score Prepare test manual Publish test Assignment 1 Student Self-compassion scale using Neff’s definition We want to develop a reliable and valid scale to assess self-compassion in relation to academic performance in university students There is a domain general: self-compassion scale

Using a 5-point likert scale, ordinal using items Tutorial Theory driven, based on Nuff’s definition of self-compassion Student population Electronically scored We would want to generate Internal consistency General time-frame We want to Tutorial 1 – rubric run-through Convergent validity – strong positive, similar but distinct Discriminant validity – negative relationships, evidence that Against the validation scales, some will be used for convergence or discriminant Week 2 Readings History of assessment Chine 2200 BC, as a means for selecting those who would get government jobs Lecture Code of ethics APS (2007) – know for exam B.13. Psychological assessments When it comes to psychological assessment, the process involves:  We ask a question, from referral: does this person have memory loss from a brain injury? Clarify question  We collect data, from testing, checking records, observing, interviewing. Select test, administer, interpret  Answer question, by providing feedback For competent use of tests: - determine the need to use a test - choose an appropriate and technically sound instrument - organise and conduct a testing session - accurately score and use norms tables - interpret the results - communicate the results - make decision based on the basis of results

-

monitor the continuing effectiveness of the test over time appreciate the range of professional and ethical issues surrounding the use of tests

be reflective in practice, there is potential for - self=fullfillinf prophecies/expectancy effects - confirmatory bias - availablilty bias - halo effect/reverse halo effect - primary and regency effects - attribution bias - anchoring effects - failure to attend base rates types of tests: self report performance individual/group human/computer norm/criterion test-takers have a right of informed consent - reason - type - length - alternative procedures - limitations - what information obtained - to whom the results will be communicated What issues might arise in assessment involving: children and adolescents? Maturity to speak for themselves, meets their capabilities Referrals by third parties? May not receive all of the information Cases where assessment will determine eligibility for a service or program Test selection - does it suit my population - is training/experience necessary? - Will it contribute to answering my referral question - Are there guidelines? - Is it reliable and valid for my purpose - What inferences can be made from the score - Generalisbale?

Consider associations with other tests – would want the D component to load positively and srongly with BDI, and the Anxiety component to load positively with BAI, see below

Evaluating tests suitability: using the DAS for - 14 years old? No, only validated for 17+ - Diagnosing client with an anxiety disorder? Screening tool, but not to diagnose - Assessing exam anxiety in uni student? Its not specific to this context, be more specific test - Screening instrument for symptoms of depression? Could do, it can be used to see severity - Assessing stress in an asylum seeker? No, Only used with Australian population - Assessing change in stress over the course of a week? No Better to do no assessment than to do an assessment, if its not right, don’t use it Test administration  Self-administered  Group administered  Individuality  Computer-assisted – standardised scoring, did they complete the test themselves/alone? What aspects are standardised?  Instructions  Items  Order of itmes  Repetition/clarification  Feedback or reinforcement

 

Timing Audio/visual/physical test materials

It’s important to observe the test-taker, and their behaviour by attending to their behaviour presentation. Gives indications of outcomes, motivation, anxiety, apprehension in the environment, report on the observations to give weight to the inferences you draw from the results. If a child seems relaxed and confident, the test procedure was fine and we can rely on the results, if a child was anxious we can take that into account with the results Take note of their: Work habits Attention/concentration Behaviour Attitude Reaction to failure Reaction to praise Speech and language Visual motor behaviour Presentation? Ethical considerations  Accept responsibility  Know the proper use and limitations  Maintain standards  Keep records  Select appropriate test for purpose  Testing conditions to allow for optimal conditions  Follow admins and scoring procedures exactly  Don’t interpret beyond its use  Seek convergent data  Consider confidence intervals  Must interpret scores Tutorial Reliability Interpret internal consistency for overall scale = .911 For factor 1 = .909 For factor 2 = .761 Test re-test reliability for the entire scale = .80 For factor 1 = .83 For factor 2 = .78 Comments/interpretation: The overall scale has good internal consistency, even .7 for factor 2 is okay to use for basic research purposes, not for clinical use. Factor 1 is a lot better, Test re-test correlations are positive strong over all, factor 2 is lagging slightly We would use overall

If one factor is stronger, could be because of fewer items – talk in limitations/recommendations about adding more items, re-doing to process Validity If items are loading heavily onto one factor, it gives and indication that are measuring a similar construct. Tells us how many factors/content and construct validity The data is suitable for factor analysis: yes - Bartlett’s is significant - KMO is over .6 What amount of variance can be explained by each factor? Factor 1 = 42.649% Factor 2 = 6.620% Combination = 49.269% Comments/interpretations of this: Factor 1 is more effective for accounting for variance Factor 2 isn’t doing much Only accounting for nearly 50% of variance, is this what we wanted to achieve? Ideally want above 50% for an effective scale Evidence to suggest that those items are accounting for some variance, maybe add some more items. Is 6% better than 0%? Refine the scale regardless, because we can d obetter than 49% anyway Correlational analyses The PSS-S has/shows evidence of convergent validity with domain general perceived stress (.84 & .77) which is what we would expect, positive and stronger the better. And discriminant validity with self-esteem scale, as it’s a weak negative correlation (-.31 & -.43), which we expect. Address the literature – what does that say about what relationships should be? Regressions We can interpret regression analyses to support predictive utility/validity. For examples, can the PSS-s account for unique portion of variance in levels of self compassion among uni students? Week 3 Readings Covers the next 3 weeks – do at some point Lecture Intellectual functioning assessment Ethics of use:

Throughout history has bee used to marginalise and suppress individuals, dirty history, moving forward we need to be mindful of those issues It does not capture the full person; those who are divergent or creative people may be seen as a disadvantage Reducing someone’s ability to a profile For example: you want to do the test with your partner It wouldn’t be objective – bias toward the score/use cues, making the score not accurate You need to have a purpose Not qualified Taking the opportunity away from them to be later tested They might think they are ‘gifted’ The Wechsler Scales! 1. WPPSI-IV – 2.6-7.7 2. WISC-V – 6-16.11 3. WAIS-IV – 16-89 The ages overlap, for individual differences – some 6 year olds only just came to Australia, don’t speak English well, cultural differences, you would use the younger scale. Ceiling or floor effect Wechsler’s view:  Measures performance not capacity, the test may not capture your full ability, rather a sample at this point  How well someone uses their intelligence, doesn’t matter how much one has  Interpretation: performance relative to comparable others  Virtually every current intelligence test adopts this approach The global capacity of a person to act purposefully, to think rationally, and to deal effectively with his/her environment The tests are limited in predicting certain aspects of occupational success and nonacademic skills

Full scale IQ – each scale below has 3-5 subtests.  Verbal comprehension (similarities, vocab, information, comprehension)  Perceptual reasoning  Visual spatial index (block design, visual puzzles)  Fluid reasoning (matric reasoning, figure weights,  Working memory  Processing speed index Performance is affected by a multitude of factors No subscale is a pure measure of any cognitive ability The scales are useful for:  Assessing cognitive strengths/weaknesses  Providing clues for further assessment needed; see a speech path  Screening/assessment for diagnosis It doesn’t provide information about academic performance, and you can’t use it to diagnose learning difficulties or disabilities. Changes from WISC-IV to WISC-V  Eight new subtests  Electronic admin through ipad  Modified/new items – don’t use ‘balckboard’ anymore  Refined scoring criteria  Updated artwork  Changes to language Know which test measures what** Administration: Requires lots of coordination and multitasking Everything needs to be recorded in real time, whilst scoring, maintaining rapport and attending to behavioural responses But be a swan on the surface. Administration tasks - Maintaining in administration manual - Accurate timing - Record responses - Observe behaviour - Adhere to strict ‘start’ ‘discontinue’ rules - Modulating tone - Stay present - Have with you: record booklet, clipboard, pencils and pen, stop watch - Check your kit has everything in it - Complete demographics page of the record form – know the age of the person - Do all writing in ink, not pencil - Record client responses

-

Be familiar with the test materials, question wording and admin instructions prior to administration Inform the client about the nature of the assessment Don’t do small talk Don’t give feedback Record all responses and times, note relative behaviours You may repeat a question/instruction if requested, expect for digit span and letternumber sequencing

Rapport within a standardised test - Engage the client, be warm, smile, be natural - Smooth transitions - Be subtle with the stop watch - Use abbreviations with note taking - Praise their effort and encourage them throughout - Be lenient with time limits for the benefit of client motivation, but score correctly The WISC kit – worth about $2500 - Testing assessment manual o Advice on where to put the book, guidelines, rapport o Scoring instructions o The tests descriptions: tells you what materials, general instructions, - Test manual - 2 X stimulus books o Has pictures for certain subtests, types of block designs - Scoring templates - Stopwatch - blocks You need to bring, stopwatch, record sheet, pens etc. For each test, there are reverse rules and discontinuation rules Reverse: - determine an appropriate start point for the child. - If child does not obtain a perfect score on either of the first 2 items, administer previous items in reverse order until 2 consecutive perfect scores are achieved Discontinuation: - Determine when to cease subtest administration - Stop administering items after a client has scored the - Don’t have to give them all the questions Questioning responses: - For verbal items: the aim is to give the client the opportunity to clarify a partially correct response by asking them to provide more information - Some you can query or flag for later - Write down exactly their response - Do not question clearly incorrect responses - See guidelines in admin manual

Most common errors in administration - Fail to record responses - Deviate from instructions - Don’t establish a baselines or do the reverse rule - Ceiling level incorrect – too many or too few items administered - Incorrect scoring - Not querying responses - Inaccurate timing - Querying when not necessary As a novice – we make up to 66 mistakes in your first 10 assessments Errors seem to increase i.e. you need to practice Tutorial Played with the WISC-IV in class

Week 4 Readings Cover week 3, 4, 5, do at some point Tutorial Step 1 Age of client = 12 years, 6 months, 8 days Similarities raw scores of items 6-23 (as he is aged 12, we start with item 8, but he got 9 wrong, so we reverse scored back to 6). = 21 but Remember to add in the ‘perfect scores’ from item 1-5 too. = 31 for similarities. (32 for class purposes). Pretend all subtests are done….block design etc. 6=2 7=2 8 = 2 (started with this, correct) 9 = 0 (got this one wrong which means we use reverse rule, so we go to 7, then 6, both must be correct, then we go to 10 onwards) 10 = 1 11 = 2 12 = 2 13 = 2 14 = 1 15 = 2 16 = 2 17 = 1

18 = 0 19 = 0 20 = 1 21 = 0 (1) 22 = 1 23 = 0 Step 2 Search in the appendix the raw score for that subtest under the correct age (ages 12.412.7) under column SI, is in line with row 11 Lecture Scoring of intellectual functioning assessment WISC-V from now on = including the full scale and primary index scales We want to calculate their score But they mean nothing unless we standardise their score against age-related peers. Raw scores can’t be interpreted; it’s the scaled cores we want * first Calculate client test age using online Pearson age calculation 7 step for conducting primary analysis 1. Score individual subtest/Raw score for each subtest (block design = 30) a. Scoring insructions very for each test b. Typically the total score is the sum of all item scores c. Include scores for un-administered items prior to the start point d. Include reverse scored items, ones before that they would have gotten correct 2. Convert raw scores into a scaled score (compared to peer group) a. Use the appendix to find age and score lining up 3. Sum scaled scores for VCI, VSI, FRI, WMI, PSI, FSIQ a. Only add first 7 that aren’t in brackets 4. Look up composite score, percentile ranks and CI, plotting profile 5. overall IQ score 6. and subtest (SI) strengths and weaknesses – and can expect some scatter over scores y significant or meaningful s scors /10 becsue that’s how many items

Record ess

7. analysing index and subtest level pairwise differences – check the critical value and workout if its significant Week 5 – tuition free Week 6 Lecture Assessment 2  Reporting results and interpretations in a client-focused report  Score and interpret an IQ assessment  Answer the referral question, make appropriate recommendations  Describe the person and their behaviour (not just scores)  IQ isn’t fixed, expect fluctuation between one performance and the next  105-115 average  the Wechsler only assesses certain areas of ability  IQ score gives comparison to age0related peers Start with - referral question - data of test results – supported by behavioural observations - construct/skill – what is being measured, what do the scores indicate - conclusion – interpret clients ability in context of background and referral issues - recommendation – what action based on conclusions? - Use successive-level interpretation procedu...