Kaplan Summary Ch 7 - Psychological Testing: Principles Applications and Issues PDF

Preview

Summary

Chapter 7 Summary...

Description

Psychological Testing, Kaplan and Saccuzzo Chapter 7: Test Administration pp. 185 – 202 Chapter Objectives:      

Discuss how the relationship between the examiner and the test taker can affect test scores Explain how an expectancy effect might affect a test score Examine the relationship between test performance and reinforcing particular responses Outline some of the advantages of computer-assisted test administration List what characteristics of the state of the subject should be considered when a test score is evaluated Know what problems you would need to consider in training your observers if you were in charge of a behavioral observation study

The Relationship Between Examiner and Test Taker – pp. 186-187  





The behavior of the examiner and his/her relationship with the examinee can affect test scores Approving remarks from the examiner vs. disapproving remarks can affect test scores Examinees perform better under approving remarks. Also children tend to perform better on tests when they are familiar with the examiner Interviewer effects: respondents may give the response that they perceive to be expected by the interviewer. This is known as Response acquiescence and can cause systematic bias. Actually more common with experienced rather than inexperienced interviewers In most testing situations, examiners should be aware that their rapport with test takers can influence the results. Should keep in mind that rapport might be influenced by subtle processes such as the level of performance expected by the examiner

The Race of the Tester – pp. 187-189     

It has been found that the race of the examiner has nonsignificant effects on test performance for both African American and white children Because the procedures for properly administering an IQ test are so specific There may be biases in the way that some test items are presented The new versions of the Wechsler have introduced greater standardization and procedures for fair test administration This does not negate the fact that serious racial bias persists in our culture

Language of Test Taker – p. 189

   

The amount of linguistic demand can put non-English speakers at a disadvantage Translation may not be comparable to the English version The validity and reliability of tests for those who do not speak English is suspect Interpreters should be used only with great caution because test interpreters can introduce bias into the testing situation

Training of Test Administrators – p. 189 – 190 



There are no standardized protocols for training people to administer complicated tests such as the WAIS-R, although these tests are usually administered by licensed psychologists Test administration error goes down after approximately 10 administrations, suggesting that students need at least 10 practice sessions to begin gaining competence with the WAIS-R

Expectancy Effects – pp. 191-192 

    

Research has shown that data sometimes can be affected by what an experimenter expects to find. Called expectancy effects or Rosenthal effects after the researcher who studied this Results of these studies consistently demonstrated that subjects provide data that confirm the experimenter’s expectancies; however, the magnitude is small Exists in some but not all situations Expectancy effects were obtained when all of the experimenters followed a standardized script Rosenthal believes that this is due to subtle nonverbal communications between experimenter and subject Pay careful attention to the potentially biasing effect of expectancy

Effects of Reinforcing Responses – pp. 192 – 195         

Because reinforcement affects behavior, examiners should always administer tests under controlled conditions An inconsistent use of feedback can damage the reliability and validity of test scores Reward can significantly affect test performance Studies show that the type of praise is important (“you worked hard”) results in better performance than (“you are clever”) Random reinforcement destroys the accuracy of performance and decreases the motivation to respond Leads to learned helplessness A test manual should clearly spell out directions for administration A good test manual gives the examiner instructions that include the exact words to be read to the examinee The test may not remain reliable or valid if they deviate from the specified instructions

  

The validity is based on a standardized procedure so it is not clear that enhancing IQ scores with reinforcement would enhance the validity of the test Special occasions that require deviation from standardized testing: the blind and the aged To ensure tests are given under standardized conditions, may give instructions through a tape recorder or computer-assisted test administration

Computer-Assisted Test Administration – pp. 195 – 196 

     

Computer assisted testing offers: excellence of standardization, individually tailored sequential administration, precision of timing responses, release of human testers for other duties, patience (test taker not rushed), and control of bias Items can be given in any order Allows more experimental control Prevents test takers from looking ahead at other sections or going back to sections Comparisons of test scores have not tended to show large differences between computer-assisted and paper-and-pencil tests Elicits better responses; i.e., research showed that people may be more honest with their sensitive information when tested on a computer than in person Cons: computer-generated test reports in the hands of an inexperienced psychologist cannot replace clinical judgment; computerized scoring routines that have errors or are poorly validated; this is difficult to detect in the software; increase over the last 15 years in consumer liability cases involving software products; computerized reports might be based on an obsolete database

Mode of Administration – pp. 196 – 197 A. Self-administered measures vs. measures administered by a trained interviewer   

Studies show that measures administered by an interviewer more likely to show people in good health than are measures that are self-completed Measures administered via telephone yielded higher health scores than those requiring a fill-in questionnaire by the person Mode of administration should be constant within any evaluation of patients

Subject Variable – pp. 197-198      

The state of the subject may be a serious source of error Test anxiety can be debilitating Has 3 components: worry, emotionality, and lack of self-confidence Illness affects test scores The elderly may do better with individual testing sessions even when the test can be administered to a group Hormonal variations in men and women can affect test results





Women may perform better perceptually and in motor performance as a function of menstrual cycle; however they may perform poorly on tests of perceptual and spatial abilities mid-cycle than during menses Men also vary in test performance as a function of variations in sex hormones

Behavioral Assessment Methodology – p. 198   

Goes beyond testing also involves observing behavior The observer plays a more active role and may make errors Problems include: reactivity, drift and expectancies

Reactivity – p. 199     

Studies have shown that reliability and accuracy are higher when someone is checking on the observers to ascertain if they are recording properly This increase in reliability is called reactivity Accuracy and interrater agreement decrease when observers believe they are not being checked To correct for this problem, experimenter checks on the observer without his/her knowledge Always use caution in interpreting reports on interrater reliability

Drift – p. 199    

Observers have a tendency to drift away from strict rules after receiving extensive coaching Most common problem is contrast effect: the tendency to rate the same behavior differently when observations are repeated in the same context Results in biased ratings of behavior Observers should be periodically retrained to correct for drift

Expectancies – pp 199 – 200 As stated previously, expectancies of experimenters can affect the results of behavioral experiments  The impact of expectancy is subtle  Probably has some minor biasing effect on behavioral data  Expectancy produces a minor but potentially damaging effect  To correct for this issue, observers should not know what behavior to expect Deception – pp. 200 – 201 

 

Studies show that most people do a poor job in detecting a liar The effectiveness of integrity tests is at question

 

Integrity tests measure a broad construct relevant to job performance instead of providing a narrow measure of honesty Continued scrutiny of integrity tests is important

Statistical Control of Rating Errors – p. 201        

Many efforts to improve the accuracy of raters have produced discouraging results Extended training to increase rater reliability has been frustrating Training is expensive and time-consuming Halo-effect is the tendency to ascribe positive attributes independently of the observed behavior Psychologists argue that this effect can be controlled through partial correlation Rater characteristics may play an important role in the accuracy of evaluations Greater cognitive abilities, higher spatial aptitudes, and critical abilities are all associated with greater accuracy More research on factors associated with rater accuracy is needed because accurate performance evaluations provide the basis for employee selection and advancement...