PSYC3302 EXAM Notes PDF

Preview

Summary

University of Western Australia UWA ...

Description

PSYC3302(EXAM(NOTES( ( KEY(FINDINGS/TOPICS( ( PSYCHOMETRICS(AND(SCALING( ( Psychometrics,is,the,science,of,measuring,psychological,abilities,,attributes,and, characteristics, ( Psychometrics,is,concerned,with,evaluating,the,attributes,of,psychological,tests:, 1. The,types,of,information,(usually,test,scores),generated,by,tests, 2. The,reliability,of,data,from,tests, 3. The,validity,of,data,from,tests, , , Psychometric,soundness:,reliability+and,validity+ • The,reliability,of,a,measuring,tool,refers,to,its,consistency,–,the,precision,with, which,the,test,measures,,and,the,extent,to,which,error,is,present,in,the, measurements,, • The,validity,of,a,measuring,tool,,for,a,particular,purpose,,refers,to,the,degree,to, which,it,measures,what,it,purports,to,measure.,, , , A,psychological,test,‘is,a,systematic,procedure,for,comparing,the,behaviour,of,two,or, more,people’,, 1. Tests,involve,behavioural,samples, 2. The,behavioural,samples,are,collected,in,a,systematic,way, 3. The,purpose,of,the,test,is,to,compare,the,behaviour,of,two,or,more,people, , Psychological,Tests:,Criterion,Referenced,, • Are,used,to,evaluate,an,individual’s,test,score,with,reference,to,a,set,standard, • Are,typically,used,to,gauge,achievement,or,mastery,,so,they,are,sometimes, called,mastery,tasks.,, , Psychological,Tests:,Norm,Referenced, • Are,used,to,compare,a,person’s,test,score,with,scores,from,a,reference,sample, , , Psychological,tests,must,be,capable,of:,, 1. comparing,the,behaviour,of,different,people,–,known,as,interindividual+ differences, 2. comparing,the,behaviour,of,the,same,people,at,different,times,in,different, contexts,–,known,as,intra-individual+differences.,, , The,goal,of,psychological,measurement,is,to,quantify,interindividual,and/or,intraindividual,differences, , Scaling,,

•

Refers,to,how,numerical,values,are,assigned,to,psychological,attributes,

, Three,key,properties,of,numbers:, 1. Identity, 2. Order, 3. Quantity, , , Identity:,sorting,people,into,categories,based,on,similarity,of,psychological,features, ‘sameness’,vs,‘differetness’, Rules,for,sorting,individuals,into,categories:, 1. Individuals,within,a,category,must,satisfy,the,property,of,‘identity’, 2. The,categories,must,be,mutually,exclusive, 3. The,categories,must,be,exhaustive, , Order:,represents,information,about,the,relative,amount,of,an,attribute,people,possess., Indicates,the,rank,order,of,individuals,on,some,psychological,attribute.,, , Quantity:,conveys,information,about,the,magnitude,of,differences,between,individuals., Numerals,with,this,property,are,real,numbers:,, 1. The,number,1,defines,the,size,of,the,basic,unit,on,any,scale, 2. All,other,values,are,multiples,of,1,or,fractions,of,1, 3. Each,numeral,represents,a,count,of,basic,units, ( ( INDIVIDUAL(DIFFERENCES(&(CORRELATIONS( ( ( Factors,Effecting,the,Variance,and,Standard,Deviation,, 1. The,degree,to,which,scores,in,a,distribution,differ,from,each,other, 2. The,metric,of,the,scores,in,the,distribution, ( ( RELIABILITY( ( Classical,Test,Theory, , • CTT,assumes,that,each,respondent’s,observed+score,on,a,psychological,test, reflects,the,sum,of,two,components:( ( 1. Their,true+score,on,the,psychological,characteristic,being,measures, 2. Measurement,error, ( It,is,the,extent,to,which,differences,in,respondent’s,observed,scores,are,attributable,to, differences,in,their,true,scores,,as,opposed,to,measurement,error.,, ( Measurement,error,creates,inconsistency,between,observed,scores,and,true,scores., There,are,various,sources,of,measurement,error:, 1. Test,construction,

• • 2. • • • • 3. • •

Item,sampling,(variation,among,items,in,a,test), Content,sampling,(variation,among,items,between,tests), Test,administration, Test,environment,(temperature,,lighting,,noise), Events,o,the,day,(positive,vs.,negative,events, Test-taker,variables,(physical,discomfort,,lack,of,sleep), Examiner-related,variables,(physical,appearance,&,demeanour), Test,scoring,and,interpretation, Subjectivity,in,scoring,(grey,area,responses), Recording,errors,(technical,glitches),

, Reliability,depends,on,two,things:, 1. The,extent,to,which,differences,in,test,scores,can,be,attributes,to,real,individual, differences, 2. The,extent,to,which,differences,in,test,scores,are,due,to,error, , . A,person’s,observed,score,on,a,test,is,that,person’s,true,score,,plus,error,,which,can, be,expressed,as:,Xo = Xt + Xe,  Where,Xo,represents,a,person’s,observed,score,,Xt,represents,a,person’s,true,score,and, Xe,represents,error.,, , Several,key,assumptions,of,CTT,are,illustrated:,, 1. Observed,scores,on,a,psychological,measure,are,determined,by,a,respondent’s, true,scores,plus,measurement,error, 2. Measurement,error,is,random-it,is,just,as,likely,to,inflate,a,score,as,to,deflate,it, (a) Error,tends,to,cancel,itself,out,across,respondents,(Xe = 0), (b) Error,scores,are,uncorrelated,with,true,scores,(rte = 0) , , Reliability,reflects,the,degree,to,which,differences,in,observed,scores,are,consistent, with,differences,in,true,scores,,as,opposed,to,error., ,

( (

Parallel,Tests, To qualify as a parallel test, the previous assumptions of CTT must be satisfied for each test, and the following must be true: 1 participants true scores for one test must be exactly equal to their true scores on the other test—known as "tau equivalence"  2 the tests must have the same level of error variance  In other words, the observed scores on both tests should have the same mean and standard deviation 

(

(

( ( ( ( ( ( ( ( (

Test,Score,Interpretation, è A,point,estimate,(does,not,take,measurement,error,into,account), a) An,individual’s,observed,test,score, b) An,adjusted,true,score,estimate, è Confidence,Interval,(does,take,measurement,error,into,account), , Point,Estimate, ,

,,

, , , , , , ,

Confidence,Intervals, ( ,

( , (

, ( VALIDITY( ( Theoretical(Basis( Validity,is,a,term,used,to,refer,to,the,meaningfulness,of,a,test,score,–,what,the,test, score,truly,means.,It,is,how,well,a,test,measures,what,it,is,supposed,to,i.e.,claims,to, measure, , Validity,in,psychometrics:, 1. Validity,concerns,interpretations,and,uses,of,scores, 2. Validity,is,not,a,property,of,the,test,itself, 3. Validity,is,a,matter,of,degree, 4. Validity,is,based,on,theory,and,evidence,, , , ,

Validity,is,a,crucial,basis,for:,, 1. The,meaningful,interpretation,of,behavioural,research, 2. Making,sound,societal,decisions,based,on,such,research, 3. Making,informed,test-based,decisions,about,individuals, , , There,are,two,prominent,perspectives,in,psychology,and,education,for,conceiving, validity:,, 1. Trinitarian,view, 2. Unitary,view, , , Traditionally,,validity,has,been,conceptualise,din,terms,of,three,different,measures,–, the,so,called,Trinitarian+view:+ 1. Content(validity:(a,measure,based,on,an,evaluation,of,the,content,covered,by, items,in,a,test, 2. Criterion-related(validity:(a,measure,obtained,by,evaluating,the,relationship,of, scores,obtained,on,the,test,with,scores,on,other,tests( 3. Construct(validity:(a,measure,obtained,by,performing,an,analysis,of:,( a) how,scores,on,the,test,relate,to,other,test,scores,and,measures,,and, b) how,scores,on,the,test,can,be,understood,within,some,theoretical,framework, , , The,contemporary,perspective,on,validity,–,known,as,the,unitary+view+–,places, construct,validity,as,the,essential,concept,in,validity.,According,to,the,unitary,view,, there,are,five,types,of,evidence,relevant,to,establishing,the,construct,validity,of,test, score,interpretations.,

Test(Content,is,the,match,between,the,content,of,a,test,and,the,content,that,should,be, included,in,the,test., , 1. Content,validity, 2. Face,validity, , Content,validity,describes,a,judgement,of,how,representative,a,test’s,content,is,of,the, full,range,of,content,that,is,relevant,to,the,construct,being,measured.,, There,are,two,key,threats,to,content,validity:,, 1. Construct-irrelevant,content, A,test,should,not,include,content,that,is,irrelevant,to,the,construct,being,measured, 2. Construct,underrepresentation, A,test,should,include,the,full,range,of,content,relevant,to,the,construct,being, measured,,as,much,as,possible, , Face,validity,relates,to,what,a,test,appears,to,measure,to,the,person,being,tested,, rather,than,what,the,test,actually,measures, , Internal(Structure(is,the,way,the,parts,of,a,test,are,related,to,each,other:,, • Some,tests,include,items,that,are,highly,correlated,with,each,other,,forming,a, single,cluster, • Other,tests,include,items,that,fall,into,two,or,more,clusters, , Factorial,validity,concerns,the,match,between,the,actual,internal,structure,of,a,test,and, the,structure,the,test,should,possess, , Response(Processes,–,there,should,be,a,close,match,between,the,psychological, processes,that,the,respondents,actually,use,when,completing,a,measure,,and,the, process,that,they,should,use.,, , è think-,aloud,procedure, è cognitive,interviews, è focus,groups, è response,times, è eye,movements, , Associations(with(other(variables,emphasises,the,theoretical,understanding,of,the, construct,we,are,trying,to,measure., , • Convergent,evidence,–,also,known,as,convergent,validity,–,is,the,degree,to, which,test,scores,are,correlated,with,tests,of,related,constructs., • Discriminant,evidence,–,also,known,as,discriminant,validity,–,is,the,degree,to, which,test,scores,are,uncorrelated,with,tests,of,unrelated,constructs, • Concurrent,validity,evidence,refers,to,the,degree,to,which,test,scores,are, correlated,with,other,relevant,variables,that,are,measured,at,the,same,time,as, the,test,undergoing,validation, • Predictive,validity,evidence,refers,to,the,degree,to,which,scores,on,the,test, undergoing,validation,are,correlated,with,relevant,variables,that,are,measured, at,a,future,point,in,time,

Consequences(of(use,–,consequential,validity,–,refers,to,the,social,and,personal, consequences,associated,with,using,a,particular,test, , , • Criterion,Validity, • Induction-Construct,Development,Interplay, • Measurement,as,Theory, , , Empirical(Estimates( , • Nomological,Network, ( • Four,methods,for,evaluating,convergent,and,discriminant,validity,, 1. Focussed,associations, , • Factors,affecting,validity,coefficients, 1. True,associations, 2. Measurement,error, 3. Restricted,range, , 2. Sets,of,correlations, , 3. Multitrait-Multimehod,Matrices, , • Trait,variance, • Method,variance, , 4. Quantifying,Construct,Validity, , Advantages:, 1. It,forces,researchers,to,consider,carefully,the,expected,pattern,of,convergent, and,discriminant,associations,that,would,make,theoretical,sense, 2. It,forces,researchers,to,make,explicit,quantitative,predictions,about,the,pattern, of,associations, 3. It,provides,a,single,value,reflecting,the,overall,‘’goodness,of,fit’’,between,the, predicted,and,actual,associations, , • Guidelines,for,interpreting,validity,coefficients, , ,There,are,at,least,three,factors,that,affect,the,size,of,validity,coefficients:, 1. True,associations,between,constructs, 2. Measurement,error,and,reliability, 3. Restricted,range, , , , , ,

Factorial(Validity( ( Factorial,validity,is,relevant,to,determining,whether,the,number,and,nature,of, dimensions,associated,with,scores,from,a,test,corresponds,to,the,theorised,number,and, nature,of,dimensions,underlying,the,construct,of,interest.,Researchers,use,a,data, analytic,technique,known,as,factor,analysis,to,help,determine,the,number,and,nature,of, dimensions,associated,with,the,scores,derived,from,a,test.,, , , • Internal,structure,of,test,scores,and,is,the,way,that,the,parts,of,the,tests,are, related,to,each,other, • The,actual,structure,of,the,tests,should,match,the,theoretical,or,intended, structure,of,the,test, • Factorial,validity,helps,specify,what,the,test,measures,–,with,respect,to,the, number,of,dimensions,,and,the,definition,of,those,dimensions, • The,definitions,of,the,dimensions,are,determined,by,which,items,‘load’,onto, which,dimensions, , , Factor,analysis:, • Helps,up,clarify,the,number,of,factors,within,a,set,of,items, • Helps,us,determine,the,nature,of,the,associations,among,the,factors, • Helps,us,determine,which,items,are,linked,to,which,factor,,which,facilitates,the, interpretation,of,those,factors, , There,are,three,different,types,of,tests:, 1. Unidiemensional,test, Consists,of,items,which,all,measure,one,,single,factor, 2. Multidimensional,test,(correlated), Consists,of,items,which,measure,two,or,more,dimensions,which,are,correlated,with, each,other,(positively,or,negatively),, 3. Multidimensional,test,(uncorrelated),, Consists,of,items,which,measure,two,or,more,dimensions,which,are,unrelated,to,each, other, , , , , , , ( ( ( ( ( ( ( ( (

PSYCHOMETRIC(BARRIERS:(RESPONSE(BIASES( ( We,want,to,interpret,responses,on,psychological,measures,as,being,an,accurate, reflection,of,individuals’,true,psychological,characteristics.,Response,biases,can, diminish,test,reliability,and,the,validity,of,test,score,interpretations;,therefore,can, damage,our,ability,to,use,psychological,measures,in,a,meaningful,way.,, , Response,biases,are,affected,by:,, • The,testing,content, • The,testing,format, • The,testing,context, • Respondent’s,conscious,efforts,to,distort,‘reality’, • Unconscious,factors,that,bias,responses, , , Types,of,Response,bias:, • Acquiescence,bias, • Extreme,and,moderate,responding, • Social,desirability, • Malingering, • Random,responding, • Guessing, , , Acquiescence,bias,occurs,when,an,individual,agrees,with,statements,without,regard,for, the,meaning,of,those,statements., - Suprious,Correlations, - Nay-saying,vs,yay-saying, , Nonacquiescence,bias,–,nay,saying,,where,an,individual,disagrees,with,whatever,is, presented.,A,nay-saying,bias,can,have,similar,consequences,to,a,yay-saying,bias., Respondents,who,obtain,low,scores,on,one,test,will,obtain,low,scores,on,another,test,, causing,correlations,that,are,artificially,higher,than,they,should,be.,, , Factors,that,increase,the,acquiescence,bias,include:, 1. Ambiguous,items, 2. Long,items, 3. Large,number,of,items, , , Extreme,and,moderate,response,biases,refers,to,differences,in,the,tendency,to,use,or, avoid,extreme,response,options.,Some,people,are,more,likely,to,endorse,the,extreme,of, a,response,scale,(even,though,they,do,not,possess,the,attribute,to,that,extreme,degree), –,known,as,‘extreme,responding’.,Conversely,,some,people,will,choose,a,response, somewhere,in,the,middle,to,avoid,making,a,strong,claim.,This,is,known,as,‘moderate, responding’., , , ,

Extremity,bias,can:, - Generate,artificial,differences,amongst,respondents’,test,scores, - Can,obscure,true,differences,amongst,respondents’,construct,levels.,, , , We,want,test,takers,to,use,response,options,that,reflect,their,construct,levels,–,this, sometimes,requires,extreme,responding., It,becomes,a,problem,when:, 1. People,with,identical,construct,levels,differ,in,their,tendency,to,use,moderate, and,extreme,response,options, 2. When,people,with,different,construct,levels,do,not,differ,in,their,willingness,to, use,moderate,and,extreme,response,options, , , Social,desirability,bias,also,known,as,faking,good,,is,the,tendency,for,a,person,to, respond,in,a,way,that,seems,socially,appealing,,regardless,of,his,or,her,true, characteristics.,,Social,desirability,bias,does,indeed,compromise,the,validity,of,test, score,interpretations,however,it,is,unlikely,to,affected,reliability,,as,people,would,be, expected,to,engage,in,socially,desirable,responding,across,all,items,,consistently.,, , , Social,desirability,bias,can,be,affected,by,at,least,three,sources:, 1. Test,content,-,Some,psychological,constructs,have,greater,implications,for,social, appeal,than,do,others.,, 2. Test,context,–,social,desirable,responding,might,be,more,likely,to,occur,in, contexts,in,which,respondents,can,be,identified,than,in,contexts,in,which,they, are,anonymous., 3. Personality,of,the,respondent,–,some,people,have,a,personality,type,that, disposes,them,to,engaging,in,socially,desirable,responding.,E.g.,need,for, autonomy.,, , , Social,desirability,bias,has,implications,for:, 1. Applied,psychology,–,transparent,tests,make,it,easy,to,fake,having,desirable, qualities,, 2. Behavioural,research,-,,people,who,tend,to,engage,in,socially,desirable, responding,will,report,that,they,have,higher,levels,of,positive,affect,that,they,in, fact,do, , , There,may,be,multiple,forms,of,social,desirability,bias.,Del,Paulhus,proposed,two, processes,by,which,socially,desirable,responding,occurs:, 1. Impression,management,–,a,conscious,process,,where,test,takers,intentionally, attempt,to,appear,socially,desirable,i.e.,job,applicant.,State., 2. Self-deception,–,an,unconscious,process,,where,people,tend,to,actually,believe, their,overestimation,of,their,psychological,characteristics.,It,is,correlated, positively,with,narcissism.,Trait.,, , ,

Malingering,is,the,opposite,problem,to,social,desirability,where,respondents,might, exaggerate,their,psychological,problems.,It,is,particularly,problematic,in,testing, situations,where,individuals,stand,to,benefit,from,their,faking,behaviour.,, , , Careless,or,random,responding,refers,to,how,sometimes,,whether,due,to,carelessness, or,lack,of,motivation,,test,takers,respond,randomly,or,semi-randomly, , Guessing,refers,to,when,respondents,may,be,motivated,to,guess,on,tests,such,as, aptitude,and,achievement,tests,which,are,designed,so,that,specific,responses,are, correct,and,others,incorrect.,, , , Methods,for,copying,with,response,bias, There,are,at,least,three,general,kinds,of,strategies:, 1. Manage,the,testing,context,–,bogus,pipeline,technique,, 2. Manage,the,test,content,and/or,scoring,–,forced,choice,formats,to,minimise,the, existence,of,social,desirability,bias.,Create,balanced,scales,to,minimise,the, effects,of,response,bias,–,positively,and,negatively,keyed,items.,Use,validity, scales,to,detect,bias.,L,and,F,scale.,, 3. Use,specially,designed,‘’bias’’,tests, , Additionally,,there,are,at,least,three,goals,that,these,strategies,are,intended,to, accomplish:, 1. Minimise,the,existence,of,response,biases, 2. Minimise,the,effects,of,responses,biases, 3. Detect,bias,and,intervene,in,some,way, , , Response,Sets:,, •,Are,temporary,and,due,to,factors,such,as,, circumstances,surrounding,the,testing,or,the,test,itself,, , Response,Styles:,, •,Are,considered,relatively,stable,and,enduring,traits,that,, are,observed,across,tests,and,testing,situations,, •,Some,people,are,more,concerned,about,appearing,socially,desirable,than,others,, •,Response,biases,can,arise,from,response,sets,and,response,styles,, , , , , , , , , , , ,

PSYCHOMETRIC(BARRIERS:(TEST(BIASES( ( Test,bias,arises,when,there,is,a,factor,inherent,in,a,test,that,systematically,prevents, accurate,,impartial,measurement.,, , A,test,might,be,biased,in,one,of,these,ways:, • The,test,may,overestimate,the,true,score, • The,test,may,underestimate,the,true,score, , Two,types,of,test,bias,(WHICH,ARE,INDEPENDENT):, 1. Construct,bias:,biases,in,the,meaning,of,a,test, 2. Predictive,bias:,biases,in,the,use,of,a,test, , , Construct,bias,occurs,when,a,test,has,different,meaning,for,two,groups,in,terms,of,the, construct,it,measures.,It,concerns,the,relationship,of,observed,scores,to,true,scores,on, psychological,test.,,Construct,bias,can,lead,to,situations,in,which,two,groups,have,the, same,average,‘true,score’,but,different,averages,on,‘observed,score’,on,a,test,of,the, construct,., , Predictive,bias,occurs,when,a,test’s,use,has,different,implications,for,two,or,more, groups.,It,concerns,the,relationship,between,scores,on,two,different,tests.,, ( There,are,at,least,two,categories,of,procedures,that,can,be,used,to,identify,test,score, bias:,, 1. Internal,methods,to,identify,construct,bias, 2. External,methods,to,identify,predictive,bias, , , Detecting,construct(bias:,internal,evaluation,of,a,test, , Evaluating,construct,bias,typically,occurs,by,conducting,analyses,at,the,item,level., An,item,on,a,test,is,biased,if:,, 1. People,belonging,to,different,groups,respond,in,different,ways,to,the,item,,and, 2. If,it,could,be,shown,that,these,differing,responses,were,not,related,to,group, differences,associated,with,the,psychological,construct,measured,by,the,test,, , There,are,four,different,procedures,for,detecting,construct,bias:,, 1. Item,discrimination,index,–,reflects,the,degree,to,which...