Bias in psychological testing PDF

Preview

Summary

Download Bias in psychological testing PDF

Description

Bias in psychological testing 1. What is bias? Bias can be defined as an inclination towards or prejudice for or against a specific person or group of people. It is also defined as general pattern or tendency to think and behave in a specific way In psychological testing, bias refers to a factor that is inherent within a test that prevents valid, accurate, and impartial measurement. → systematic error in the measurement process that differentially influences scores for identified groups Bias occurs if score differences on the indicators of a particular construct (e.g., percentage of students knowing that War-saw is Poland’s capital) do not correspond to differences in the underlying trait or ability (e.g., geography knowledge). 2. Impact of bias Bias may lead to issues in fair testing. It will influence the validity and reliability of not only test scores, but of also the clinical interpretation of test scores and assessment observations, and thus influence the overall outcome of the psychological testing/assessment procedure in a negative and often discriminatory way, thus infringing upon ethical standards

Diminish or exaggerate the actual association, create association when there actually is none, or hides an association that actually exists 3. Ways to identify Bias

Bias in psychological testing has been examined using several different methods. The trinitarian model of validity (content validity, criterion-related validity, and construct validity) has been used to provide a framework to investigate bias in tests. Combination of methods including statistical techniques and judgmental review can be used to identify potential bias. -

content-related sources

-

response-related sources

-

differences in internal test structure

-

differential predication or selection across groups

In order to determine the source of bias on tests, psychometric or technical, quantitative approaches are used, particularly at the item or task level, to see whether predictions can be

made. Different techniques yield somewhat different findings, but the focus is on the responses of the student as scored by usual procedures.

4. Types of bias in psychological testing

Test Bias

Cultural Bias

Response Bias

-Test bias refers to the differential validity of test scores for groups (e.g., age, education, culture, race, sex). -Bias can be internal (psychometric properties, test structure) or external (differential prediction/selection) to the test. -What is considered wise in one society may not be considered wise in another -culture and background may lead to the inaccuracy of assessments

Cognitive biases in which the respondent feels compelled to respond in a certain way rather than reflect their true beliefs

the value and meaning of intelligence depend on cultural norm

Social desirability

Information processing bias – refers to the inappropriate interpretation of ambiguous stimuli, scenarios, observations or even test scores Interpreter Bias/Scoring Bias

bias may reside in the assessment process is in the method of scoring or how the scoring rules are applied to judge performance Confirmation Bias

Attention to information that confirms the examiner’s/psychologist’s beliefs/expectations, and ignoring other information Tendency to search for or

Looking for specific traits and states of a given psychopathology to confirm a premade diagnosis as opposed to taking to account any and all available information presented by the client.

Administration Bias

Outcome Bias

Situational bias

interpret information in a way that conforms one’s preconceptions Bias that arises due to the way in which tests and assessments are administered (verbally/nonverbally; computer vs pen and paper etc)

Communication problems between interviewers and interviewees can easily occur, especially, when they have different first languages and cultural backgrounds

Tendency to judge decisions by its eventual outcome instead of based on the quality of the decision at the time it was made

influences in the test situation, but independent of the test itself, that may bias test scores

characteristics of the examiner (e.g., age, race, or gender), the emotional atmosphere of the testing situation, the cooperativeness and/or motivation of the examinee, and characteristics of the test instructions

Test anxiety Halo effect

5. Types of Bias under Test Bias -

Construct-validity bias

Criterion-related validity bias

Occurs when the measure yields significantly different results for testtakers from the original culture for which the test was originally developed and the test-taker in a new culture in which the test was administered in - occurs if the construct measured is not identical across cultural groups concerned with differential predictive validity of a test that might indicate bias against a particular group. Many tests (particularly IQ, ability, and aptitude measures) are constructed to make predictions of

Western intelligence tests: emphasise reasoning, problem solving abilities, acquiring knowledge and memory Social aspects of intelligence are not emphasised → problematic as many different cultures regard this as an aspect of intelligence

Item bias

Item-selection bias

Method bias

Predictive-validity bias

performance on other relevant measures or future outcomes distortions at item level → Biased items have a different psychological meaning across cultures an item is taken to be biased if persons from different groups with the same score on the construct, commonly operationalized as the total score on the instrument, do not have the same expected score on the item Items-selected/used to formulate a test may be culturally-loaded and influenced by the culture in which the test has been developed in Method bias also refers to problems deriving from instrument characteristics (instrument bias) Response procedures (how to respond to a given test) creates method bias Occurs when associations with other variables (i.e., validity coefficients) significantly differ between groups or when a test systematically over or underestimates a criterion for a given group

refers to problems that occur with individual items on a test, such as problems related to item wording, grammar, and choice of cultural phrases.

stimulus familiarity

6. Cultural Bias Standardised tests are normed based on the knowledge and values of the majority groups, which can create bias against minority groups, including gender, race, community status, and persons with different language backgrounds, socioeconomic status, and culture. Culture bias refers to the cultural loading of test items which makes tests developed in one country/culture inappropriate to use in another country/culture due to cultural differences (e.g. language and customs) which will greatly impact the validity and reliability of the test. Not even that, but it will create significant differences amongst scores between scores from two individuals (1 =

from original culture; 2 = individual from a different culture), indicating favourable outcomes for one culture and ‘discriminating’/’being prejudice’ towards another culture. (E.g. utilising a French IQ test and administering it to South Africans).

cultural bias (1) A bias against subjects from certain sections of the population. (2) Using tests which presuppose a knowledge of a particular culture (typically, white middle class). Subjects from groups unfamiliar with this culture might perform badly on such tests, simply because they do not have the appropriate social background to understand the questions Cultural bias will also considers other factors that affect test scores and fairness of administration: 

socioeconomic status



race/ethnicity



language and language proficiency



gender



cultural customs and culture-specific behaviours

cultural bias can occur in: 

Interpretation of test results → occurs on two levels: o

From the interpreter’s perspective: the interpreter/test administer/examiner is susceptible to making clinical judgments, interpretations, and observations that are influenced by his/her culture or the dominant culture of a given country. This therefore, affects the validity and reliability of the clinical judgments, interpretations, and observations.

o

From the test-taker’s/patient’s perspective: Cultural and social norms affect how test-takers understand and interpret the wording of test questions. How they make sense of the test items can be influenced by their values, beliefs, experiences, communication patterns, teaching and learning styles, and epistemologies of their cultures and societies

o

Language: language presents as a big problem when it comes to the interpretation of test-items and test-scores, as well as, when it comes to forming a professional rapport with the test-administrator/test-taker, and the → item interpretation can be affected by test questions written in a language other than the native language of the test taker (how to avoid this issue: consider a non-native English speaker’s language proficiency before deciding whether to test her/him in English or the native language)

o

How to overcome: To overcome these testing biases, researchers suggest that acculturation should be measured in addition to psychometric tests



selection of test and test items → test-users/administers are susceptible to choosing and administering tests/test items that represent the dominant culture → this process is often influenced by their culture as well which is a source of bias.

How to avoid Cultural Bias -

Alternative assessments, in contrast to traditional or standardized tests, are considered more culturally fair

-

Power tests are considered more culturally fair than speed tests

-

Oral instructions are more culturally fair than written instructions,

-

non-verbal contents are more culturally fair than verbal contents, and familiar contents to the students are more culturally fair than unfamiliar contents

7. Response Bias

Response bias can affect both reliability and validity of the measurement. Response bias are systematic errors. Types of response bias: Acquiescence bias; Extreme/Moderate responding; Social desirability bias; Malingering; Random Responding; Guessing acquiescence bias: -

When an individual agrees with statements without regard for the statements meaning. Respondent has a tendency to agree with a statement when they are in doubt

-

Counter this by counterbalancing measure (phrase question negatively or positively)

When is acquiescent response most likely to occur? -

When respondent is in doubt

-

Items are too complex

-

The testing situation is distracting

-

The person has a hard time understanding material

What is social desirability response bias? -

The tendency for a person to respond in a way that seems socially appealing, regardless of their true characteristics.

-

Over-reporting of good behaviour and underreporting of negative behaviour.

Note: 

Equivalence of measures (or lack of bias) is a prerequisite for valid comparisons across cultural populations.



Statements about bias always refer to applications of an instrument/measure



that psychological assessment instruments, despite tremendous value for increasing an understanding of human nature, have the potential to be "used in harmful, inaccurate, and inappropriate ways due to the impact of bias in the administration and interpretation of assessments and tests

other factors that give rise to bias -

test is not demographically or culturally representative

-

Norm-referenced tests (or tests designed to compare and rank test takers in relation to one another) may be biased if the “norming process” does not include representative samples of all the tested subgroups

-

test formats may have an inherent bias toward some groups of students (e.g. med, multiplechoice tests may favour certain styles of thinking more characteristic of males than females, such as a willingness to risk guessing the right answer or questions that reflect black-andwhite logic rather than nuanced logic.)

-

Choice of language in test questions can introduce bias...