Final Evaluation PDF

Preview

Summary

Evaluation of tests, their scoring guides and effectiveness...

Description

1

Final Analysis of The Autism Spectrum Rating Scale Amanda Foust Capella University Dr. Sowa PSY7610

2 Introduction There are several assessments available that are designed to determine the likelihood an individual has autism spectrum disorder (ASD) which McClain et al., 2020 characterizes by deficits in social communication and interaction skills and the presence of restricted and repetitive patterns of behavior, interests, or activities, (McClain et al., 2020) and some even measure, to what level. In the essay we will evaluate one assessment that has, through research, proved itself the best available assessment among others that available: The Autism Spectrum Rating Scale (ASRS). The following sections will evaluate the ASRS in terms of its purpose, content, sampling procedures, materials, cost, and a few other sections. As with any test, there are positive and negative aspects to consider, we will delve a bit further into those as well. We will then compare this data with two different available assessment tools for ASD. Synthesizing and comparing this information will prove the ASRS an asset to determine if an individual has ASD. Relevancy to Specialization or a Related Profession With the goal of obtaining my master’s in clinical psychology with a specialization in Behavior Analysis, I plan to work in early intervention and within the pre-school/elementary school education system. Working in these environments will include many responsibilities, including assessing children for various intellectual disabilities, such as ASD. Research shows that early diagnosis of and interventions for autism are more likely to have major long-term positive effects on symptoms and later skills, (NIH, 2021) With this fact, having access to proper, valid, and reliable testing assessments in imperative. Through the years psychologists and researchers have developed testing instruments that were said to determine the level of ASD in an individual. With use some of these tests were found to not be effective or valid in that

3 attempt. On the other hand, there have been some seemingly successful creations, such as the ASRS. The Autism Spectrum Rating Scale (ASRS) is said to measure behaviors that are associated with ASD in individuals ages 2 – 18 years old. (Kluck & Shaw, 2014) With the use of five scores: social/communication, unusual behaviors, self-regulation, and short forms. (McClain et al., 2020) There are other assessments available, the Childhood Autism Rating Scale, Second Edition (CARS-2), and the Gilliam Autism Rating Scale, in its third edition (GARS-3) that are designed to test similar ages groups, CARS-2 assesses children ages 2 and up, (Malcolm et al., 2014) and the GARS-3 assess children aged 3 to 22. (Atlas et al., 2017). Further sections will describe why even though these three assessments are available, why I would choose the ASRS for my client’s assessment. Test Purpose and Content, Skills, and/or Constructs Assessed The ASRS, published in 2009, is the first nationally standardized, norm-referenced ASD Rating Scale. This assessment, a multi-informant measure, helps identify symptoms, behaviors, and associated features of ASD in children and adolescents aged 2 to 18 years. (Goldstein et al., 2021) This test contains five scores of measures: social/communication, unusual behaviors, selfregulation, and short forms. The purpose of these short forms is to assist with screening, treatment, and progress reporting. (McClain et al., 2020) The ASRS gives many treatment scores that include peer socialization, adult socialization, social/emotional reciprocity, atypical language, stereotype, behavioral rigidity, sensory, sensitivity, and attention/self-regulation. (Kluck & Shaw, 2014) Assessing all of these is crucial in determining an individual’s ASD status and beginning the process of individualizing a behavior and treatment plan, as early in the persons life as possible.

4 The CARS-2 is a little more in depth than the ASRS regarding what it assesses. The CARS-2 has a standard version and a high functioning version. The standard version of the test is used for identifying the presence of behavioral symptoms of autism in persons two to five years old, possibly older if the individual has an IQ under 79. The high-functioning version is for use in individuals aged 6 and older with an IQ of 80 and above with 16 different scales measured, Relating to People, Imitation, Emotional Response, Body Use, Object Use, Adaptation to Change, Visual Response, Listening Response, and more. (Malcolm & McLellan, 2014) The Gilliam Autism Rating Scale, in its third edition (GARS-3) is designed to identify individuals who may be showing indicators of ASD using seven scores. The scores are listed as Restricted/Repetitive Behaviors, Social Interaction, Social Communication, Emotional Responses, Cognitive Style, and Maladaptive Speech. (Atlas & Hutchins, 2017) The positives of the ASRS having less scores are that the test can be completed at a faster rate, depending on how cooperative the individual being assessed is which in this case would be young children, this could be a great factor. Also, the assessment is simple enough for a parent or teacher to complete with a certain amount of accuracy. The negatives behind the ASRS are that it does not go as in depth as the other assessments om terms of the scores it covers, and with the parents or teachers completing the assessment there could be a level of bias present. This will be discussed further later. Even with these negatives and the other assessments covering more scores, I still believe the ASRS is the best assessment tool to use in measuring ASD. The following sections will show why. Normative Sample, Sampling Procedures, and Intended Population Multi-Health Systems (MHS), the tests publisher, distinguished the ASRS as a normreferenced standardized test based on 2,560 non-clinical cases from across the United States.

5 (ASRS Autism Spectrum Rating Scales, 2021) In this case, the term norm-referenced testing means “a method of evaluation and a way of deriving meaning from test scores by evaluating an individual test takers scores and comparing it to the scores of others in the sample” (Cohen et al., 2021) The cases included 640 cases with 2 – 5 years old’s, 320 parents, 320 teachers, and 1,920 6 – 18 years old. The sample included diagnoses of ASD, Asperger's, Pervasive Developmental, Attention-Deficit/Hyperactivity Disorder, Mood and Anxiety Disorders, and Disruptive Behavior Disorders, just to name a few. (ASRS Autism Spectrum Rating Scales, 2021) For this sample, the administrators used three reports to effectively compare their data. The first report was the progress report which compared the results of administrators for the same individual to measure any changes over some time. The next report used was a comparative report, this combined the results of different test raters to provide an evaluation of an individual's scores from a multi-rater perspective. The purpose of this is to highlight potentially important inter-rater differences in scores. Finally, the test publisher used interpretive reports. These reports provided the administrators with information regarding a single individual's scores with the use of numbers and graphs. (ASRS Autism Spectrum Rating Scales, 2021) Both the CARS-2 and the GARS-3 are also norm samples. The CARS-2 included a clinical sample of 1,034 individuals who were diagnosed with ASD. To rate this test, the administrators used a four-point response scale. This scale is used to measure the number of behaviors present, their duration, intensity, and oddity. (CARS-2 Childhood Autism Rating Scale, Second Edition, 2021) The GARS-3 had a sample of 1,859 individuals. This scale has 56 items which describes the characteristic behaviors of persons with autism. The items are grouped into six subscales: Restrictive/Repetitive Behaviors, Social Interaction, Social Communication,

6 Emotional Responses, Cognitive Style, and Maladaptive Speech. (GARS-3: Gilliam Autism Rating Scale-Third Edition, 2021) In my opinion the sample used in the ASRS shows why it should be used over the CARS2 and the GARS-3. Yes, the other two scales may have a larger sample of participants, but the researchers involved in the study had a larger range of diagnoses involved in the study, which could help the researchers know how effective the assessment to a wider variety of test takers. Another positive of the ASRS sample is that the researchers did not only test the individuals once. Multiple researchers administered the assessment to the same individual and the scores were compared. This is beneficial because the researchers can try to determine why the individual may have scored different on the test, could there have been a problem that day before the test, was the individual more anxious than usual, did the individual not like one of the administrators. This allows for a clearer interpretation. Required Training, Knowledge, and Skills of Test Users In the creation of each assessment, it is determined who the test is meant for, who it is designed to measure and who/how the test is to be interpreted. As we now know, from the sections above, that the ASRS was designed to assess the level of ASD in individuals aged 2 to 18, and that once the assessment is properly and successfully completed, the test-user will be able to measure any behaviors that are associated with ASD. The ASRS allows for parents and teachers to be the administrators of the assessment via either pen and pencil or computer. To score the assessment, the parent or teacher needs to enter the information into a downloadable scoring software and the results of the test will be readily available. The test also provides alternate scoring methods for individuals taking the assessment who are unable to or have limited speaking abilities. (ASRS Autism Spectrum Rating Scales, 2021)

7 Comparatively, the CARS-2 assessment requires that the test administrator have a master’s degree in psychology, school counseling, occupational therapy, speech-language pathology, social work, education, special education, or a field similar. (CARS-2 Childhood Autism Rating Scale, Second Edition, 2021) Similarly, the GARS-3 also requires a master's degree in psychology, education, occupational therapy, social work, counseling, or in a field closely related to the intended use of the assessment, and formal training in the ethical administration, scoring, and interpretation of clinical assessments or by having certification by or full active membership in a professional organization that requires training and experience in the relevant area of assessment. (GARS-3: Gilliam Autism Rating Scale-Third Edition, 2021) In this respect the ASRS is the preferable tool due to its accessibility and ability to use. There may be some individuals and families who may not have reliable access to someone qualified to administer the CARS-2 or the GARS-3. The ASRS is the perfect tool to get a child assessed in the early stages, such as in early intervention, when one may be unsure if the child is exhibiting signs of ASD. The downside of a parent or teacher administering the assessment is again, a possible level of bias or lack of understanding of how to score or what the score means. For the benefit of the children who are possible subjects of an ASRS assessment, having access to the test, to even just to start the process of diagnosing an individual outweighs the cons. Test Technical Quality For the ASRS to be effective in early intervention, to help those whom it is designed to, and to be considered a ‘good test’, it will not only the instructions for administration and scoring, it will measure what it is said to measure (Cohen, 2022) A good test must also possess two key aspects, it must be reliable, and it must be valid. A good test will also prepare for any possible error. (Cohen, 2022) These facts will be the same for all assessments,

8 including the CARS-2 and GARS-3. To be effective these tests must contain a level of validity and reliability. A test is considered valid if it measures what it is designed to measure. (Cohen, 2022) The reliability of a test must meet certain criterion. This is the consistency of the measuring tool, the precision in which the test measures, and the extent to which errors are present in the measurement. (Cohen, 2022) Camodeca (2019) found that existing studies, prior to their research, showed the measure had strong internal consistency reliability. The test they reviewed also exhibited strong test-retest reliability for both the summary and treatment scales. Along with showing these forms of reliability, the study showed a higher level of criterion validity (Camodeca, 2019) In attempts to recreate and second the initial findings of the reliability of the ASRS, the test user evaluated a group of 422 children who had previously gone under evaluation for a possible diagnosis of ASD. The children were separated into two groups, identified with ASD, and identified without. (Camodeca, 2019) The current study investigated the criterion validity of the ASRS regarding an ASD diagnosis on large scale within a community. Within the study, it was founded that the strongest correlation within the test is the reflection of specific symptomology. This shows that the test has strong criterion validity. (Camodeca, 2019) In a validation and reliability study conducted by Moulton et al., (2019) 282 children on various degrees of the autism spectrum, ages ranging from 16 to 32 months of were assessed using the CARS-2. (Moulton et al., 2019) In these evaluations, it was founded that the assessment displays a high degree of consistency which would mean the possibility of a high replication rate but also, in using these methods, it was discovered different rates among the levels being tested by CARS-2. Still with the different rates it was determined that the assessment is appropriate to use among 2-year-old children, adding to the ages previously

9 approved. (Moulton et al., 2019) Sadly, when researching the GARS-3 there was no information available on its validity and reliability. In my research the GARS-2 had to be evaluated for both. In evaluating a test, I believe, the most important aspect in determining if a test is sufficient for use, is the level of reliability and validity the assessment has so I am sure the test is measuring what it says it is going to for the sake of the individual taking the assessment and that it will accurately calculate a proper diagnoses or lack thereof, which ever is true for a particular case. With the GARS-3 having no studies determining its level of reliability and validity, to me that makes the test not suitable for use. Yes, the CARS-2 has a decent level of validity and reliability, but the sample size used to determine these was much smaller and did not cover all the age ranges that the assessment is available for. Using all this information, I believe the ASRS is still the best option assessment for use in children with possible ASD. Test Materials, Cost, and Administration Times The ASRS can be administered by either paper-and-pencil or online and is used to evaluate the possibility an individual has ASD. Administration of the ASRS full form takes approximately 20 minutes to complete. The short form takes about 5 minutes and then scoring is completed either manually or on scoring software. Again, administration and scoring can be completed by the parent/guardian or teacher with this scale. (ASRS Autism Spectrum Rating Scales, 2021) The materials for the rating scale, both the long and short forms are available on the publisher’s website. The publisher offers all the material needed, in English and Spanish, and both individually and in all-inclusive kits. The individual forms and scoring guides are priced at $80. If the rater wanted to purchase a kit that includes everything they would need to administer and score the scale, it would cost $495. (ASRS Autism Spectrum Rating Scales, 2021)

10 The CARS-2 takes an estimated 5 to 10 minutes to complete, both the long and short form, with forms available in English, Bulgarian, and Italian. Materials range in price from $58 to $237, depending on what you are looking for. (CARS-2 Childhood Autism Rating Scale, Second Edition, 2021) The GARS-3 also takes approximately 10 minutes to complete. There are three items available as materials for the GARS-3. These include the GARS-3 complete kit, which costs a total of $175.50; summary response forms, $63.30, and finally the examiner's manual, which is $74.50. All materials are only available in English. (Gilliam Autism Rating Scale, Third Edition, 2021) The benefits of the ASRS in this section includes that administration of the assessment only requires 5 or so minutes. Also, a parent or teacher can administer and score the test. With these being the individuals that are closest with the test taker and are the witnesses of behavior they are the ones most able to answer the assessment, over someone who has had limit, if any contact, with the child. The negative of the assessment, compared to the CARS-2 is the price difference and the lack of availability in other languages. Translating the assessment to as many languages as possible, while remaining effective, reliable, and valid, would increase the range of reach for the test. Advances in Technology Scoring for the ASRS can occur in several ways and can be completed in settings such as at home or school, administered by parents, teachers, and/or childcare providers. (ASRS Autism Spectrum Rating Scales, 2021) The interpretation of the ASRS should be completed by an assessor who holds MHS B-level qualifications. The test can be scored either by hand, through ASRS scoring software, or the ASRS online assessment center. (Simek et al., 2011) The CARS-2 (CARS-2 Childhood Autism Rating Scale, Second Edition, 2021) and the GARS-3 (Gilliam

11 Autism Rating Scale – Third Edition, 2021) do not offer any technological advances to assist with the test administration and scoring. Both tests are available in paper-and-pencil format. Technological advances make accessibility to the rating scale wider for many individuals who would benefit from it. Also, there are some who are more at ease taking a test via pencil and paper and others who find ease in testing on computers. Having both options available to the test taker will assure the most accurate score possible. Therefore, the ASRS is the preferred test of used based on its available technology. APA Ethical Guidelines The American Psychological Association (APA) has set forth Ethical Standards, or enforceable rules of conduct for psychologists. The Ethics Code is intended to provide guidance for psychologists and standards of professional conduct that can be applied by the APA and by other bodies that choose to adopt them. This Ethics Code applies only to psychologists' activities that are part of their scientific, educational, or professional roles as psychologists. Areas covered include but are not limited to the clinical, counseling, development of assessments, and administration. (American Psychological Association, 2017) The Assessment section of the Ethical Principles of Psychologists and Code of Conduct includes 11 subsections, all just important as the last. We will look deeper into some of these sections and see how they relate to the ASRS. Section 9.01 of APA Ethical Principles, Bases for Assessment, states that “psychologists base the opinions contained in their recommendations, reports, and diagnostic or evaluative statements, including forensic testimony, on information and techniques sufficient to substantiate their findings.” (American Psychological Association, 2017) The researchers who created the ASRS and those who evaluated this scale did just that. They used the information available to

12 them at the time and used accepted techniques to ensure that this test not only measures what it says it is going to measure but it also is effective, and correct, in its results. To do this the use of sampling was implemented. The ASRS follows this standard. Another is, 9.02, Use of Assessments. This standard states: “Psychologists use assessment i...