Evaluation of test materials, scoring and technology PDF

Preview

Summary

Evaluation of scoring measurements...

Description

Evaluation of Test Materials, Scoring, and Technology

Evaluation of Test Materials, Scoring, and Technology Amanda Foust Capella University Dr. Sowa

1

Evaluation of Test Materials, Scoring, and Technology

2

Introduction Autism spectrum disorder (ASD) is characterized by deficits in social communication and interaction skills and the presence of restricted and repetitive patterns of behavior, interests, or activities. (McClain et al., 2020) To be sure an individual with ASD is provided with all the necessary tools to lead a full life it is imperative to obtain a diagnosis as early as possible. With the need for early intervention is the need for proper evaluation and diagnosis. (McClain et al., 2020) Three tests developed are the Autism Spectrum Ratings Scales (ASRS), Childhood Autism Rating Scales – 2nd Edition (CARS-2), and Gilliam Autism Rating Scales – 3rd Edition (GARS3). We will continue to evaluate these tests further as in previous papers. The ASRS are designed to measure behaviors associated with ASD for children aged 2 – 18 years. The test is offered in both a full length and a short form. These scales can help guide diagnostic decisions and can be used during treatment planning, ongoing monitoring of response to intervention, and program evaluation. (ASRS Autism Spectrum Rating Scales, 2021) The CARS-2 has become one of the most widely used and empirically validated autism measurement assessments. The purpose of the CARS-2 is to determine if a child, ages 6 and under, has autism and/or severe cognitive deficits. It also aims to distinguish between mild-to-moderate deficits through offering both standard and high-functioning versions (IQ above 80). (CARS-2 Childhood Autism Rating Scale, Second Edition, 2021) Finally, the GARS-3 assists parents and teachers in identifying the probability of autism in an individual and estimating its severity level. This test was developed to test individuals aged 3 to 22. (Gilliam Autism Rating Scale, Third Edition, 2021) In this paper, we will evaluate these three tests regarding their testing materials, scoring guidelines, and available technology.

Evaluation of Test Materials, Scoring, and Technology

3

Evaluation of Test Materials, Costs, and Administration Time The ASRS, published in 2009, can be administered either paper-and-pencil or online and is used to evaluate the possibility an individual has ASD. Administration of the ASRS full form takes approximately 20 minutes to complete. The short form takes about 5 minutes and then scoring is completed either manually or on scoring software. (ASRS Autism Spectrum Rating Scales, 2021) The materials for the rating scale, both the long and short forms are available on the publisher’s website. The publisher offers all the material needed individually but also in allinclusive kits. These materials are available in both English and Spanish. The individual forms and scoring guides are priced at $80. If the rater wanted to purchase a kit that includes everything they would need to administer and score the scale, it would cost them $495. (ASRS Autism Spectrum Rating Scales, 2021) The CARS-2, published in 2010, is a rating scale offered in both standard form and high functioning. It helps to identify children with autism and determine symptom severity through quantifiable ratings based on direct observation. Both forms of the CARS-2 take an estimated 5 to 10 minutes to complete, with forms available in English, Bulgarian, and Italian. (CARS-2 Childhood Autism Rating Scale, Second Edition, 2021) The materials on the publisher’s website for testing include the standard form, and the high-functioning form, both of which are $58. There is also the parent/caregiver questionnaire, $41, the materials manual, $65, and the CARS-2 manual, which is $109. The publisher does offer an all-inclusive kit so the rater can receive all the needed materials. This kit costs $237 with the publisher. (CARS-2 Childhood Autism Rating Scale, Second Edition, 2021) The GARS-3, published in 2013, assists teachers, parents, and clinicians in identifying autism in individuals and estimating its severity. This is a paper-and-pencil test that takes

Evaluation of Test Materials, Scoring, and Technology

4

approximately 5 to 10 minutes to complete, (Gilliam Autism Rating Scale, Third Edition, 2021) desirably, individually, by someone who has close contact with the individual. Once the test is completed, scoring is done manually. (Ziublsky et al., 2013) There are three items available as materials for the GARS-3. These include the GARS-3 complete kit, which costs a total of $175.50; summary response forms, $63.30, and finally the examiner's manual, which is $74.50. All materials are only available in English. (Gilliam Autism Rating Scale, Third Edition, 2021) All three of these scales offer the materials necessary to perform an accurate evaluation if the products are used properly. When working to diagnose an individual there are many aspects one will look at to see the best options for testing, length of test, the accuracy of test, reliability, cost, etc. In my opinion, the one that meets all of these is the CARS-2. The publisher of this scale not only suggests that the test be administered and scored by professionals, but they also offer detailed booklets and guides on how to use each version of the scale and how to properly interpret it. The test is also quick to administer and is available in a wider range of languages to cover more cultures. Evaluation of Scoring Procedures and Interpretation Guidelines Scoring for the ASRS can occur in several ways but first, let’s cover who can score the test. Both the full version and the short test of the ASRS can be completed in settings such as at home or school, administered by parents, teachers, and/or childcare providers. (ASRS Autism Spectrum Rating Scales, 2021) The interpretation of the ASRS should be completed by an assessor who holds MHS B-level qualifications. (Simek et al., 2011) The test can be scored either by hand, through ASRS scoring software, or the ASRS online assessment center. This scoring will offer the test administrator a T-score across the four scales of measurement, total score, ASRS, DSM-IV-TR, and treatment. The total score is considered the most accurate

Evaluation of Test Materials, Scoring, and Technology

5

determination of ASD in an individual on the ASRS. (Simek et al., 2011) The T-scores that are offered indicated a very elevated score (70 or higher), elevated score (65 to 69), slightly elevated score (60 to 64), average score (40 to 59), and a low score (40 and under). (Simek et al., 2011) If one uses the online assessment tool there will be three reports offered. First is an interpretive report that provides detailed results from one administration of the test. The comparative report provides the test administrator with a multi-rater perspective by combining results with up to five different raters. Finally, the progress monitoring report. This report provides an overview of change over time by combining the results of up to four administrations of the test to the same rater. (ASRS Autism Spectrum Rating Scales, 2021) Unlike the ASRS, the CARS-2 is administered by a clinician or a professional with training in ASD in a research or clinical setting. These same clinicians or professionals are to rate and interpret the scale as well. (CAR Autism Roadmap, 2020) The CARS-2 has three rating forms that consist of 15 items. Each item is then rated on a four-point rating scale. On this scale, “1” indicates that an individual’s behavior is at normal limits and “4” tells the interpreter that an individual’s behavior is highly abnormal or severe compared to peers in their same age group. (Van Bourgondien et al., 2010) CARS-2 does not have any subscales, only a total score. This means that the points from the four-point rating scale are then turned into a T-score and then into a percentile rank. This score and percentile will represent the individual’s symptom level. There is also a parent form, but this form remains unscored. (Van Bourgondien et al., 2010) Searching for scoring information on the GARS-3 lead to limited results. The 3rd edition of GARS returns standard scores and percentile ranks in attempts to answer the questions regarding the probability an individual is diagnosed with ASD and what level of severity. (Gilliam Autism Rating Scale – Third Edition, 2021) There was more information found

Evaluation of Test Materials, Scoring, and Technology

6

regarding the scoring on the GARS -2. The GARS-2 is a 42 item informant rating scale that is broken into three subscales: Stereotypes behavior, social interaction, and communication. It tells the interpreter to identify and potentially diagnose autism and provide information on the severity level. This test can be administered, interpreted, and scored by parents, teachers, and/or clinicians. (Volker et al., 2016) The 42 items from the subscales are rated on presence and frequency and then those raw scores are then converted into “percentile ranks and derived scores with a mean of 10 and a standard deviation of 3… The sum of these derived scores also produces an overall Autism Index that indicates the likelihood that an individual has this disorder” (Ziublsky et al., 2013) All three of these rating scales are desirable methods of measurement in behavior analysis. The earliest intervention possible is needed to see the largest outcome. Though all three of these scales seem to score similarly, I believe, based on scoring alone, that the CARS-2 may result in scores more accurate to determine if an individual does have ASD. Though the time to administer and interpret may take some more time, having a trained clinician perform the assessment will most likely lead to clearer answers and a greater understanding of the individual’s diagnosis. Evaluation of Test Technology The levels of technology for these assessments are all a bit different from one another. The ASRS is offered in both English and Spanish versions. The test itself is administered with a pen and paper or online. If the test taker uses the paper and pencil method to take the test the interpreter can then score the test one of three ways, manually, using scoring software, or online scoring. If the test is taken online, it is scored online. (ASRS Autism Spectrum Rating Scales, 2021) The CARS-2 (CARS-2 Childhood Autism Rating Scale, Second Edition, 2021) and the

Evaluation of Test Materials, Scoring, and Technology

7

GARS-3 (Gilliam Autism Rating Scale – Third Edition, 2021) do not offer any technological advances to assist with the test administration and scoring. Both tests are available in paper-andpencil format. The benefits that would come with including a technological option to the tests would be a wider reach to individuals with whom there is a probability of ASD. There would also be a lower chance of scoring error since the calculations would be done by a computer. The ASRS that is more effective with technology is the ASRS. This is because the test is offered not only in another language but there are opportunities for an individual to complete the assessment in a manner that is most comfortable to them, on paper or computer. The test also allows the interpreter to lower the chance for error by looking at the results on a computer. Advanced technology allows for clearer results and leads to a deeper understanding of the outcomes. Conclusion The Autism Spectrum Rating Scale, Childhood Autism Rating Scale – Second Edition, and Gilliam Autism Rating Scale – Third Edition are all used to measure and predict Autism Spectrum Disorder in children and estimate the level of severity. The publishers of these tests all offer testing materials, including the tests, scoring guides, and manuals to ensure the tests are being used properly and can accurately measure the risk of ASD in an individual. To help with the administration and scoring the ASRS offers online tools for the assessment. All the tests can be administered on paper to assess an individual. The scoring of these tests is of utmost importance because it could lead to a child getting the supports, he or she needs to be able to learn as much as they can in the best way they can, or it can mean a child is misdiagnosed and left in an environment in which they cannot flourish and intervention becomes even harder.

Evaluation of Test Materials, Scoring, and Technology

8

References ASRS Autism Spectrum Rating Scales. (2021) Retrieved August 27, 2021, from https://storefront.mhs.com/collections/asrs CAR Autism Roadmap. (2020) Retrieved August 28, 2021, from https://www.carautismroadmap.org/childhood-autism-rating-scale CARS-2 Childhood Autism Rating Scale, Second Edition. (2021) Retrieved August 27, 2021, from https://www.wpspublish.com/cars-2-childhood-autism-rating-scale-second-edition GARS-3: Gilliam Autism Rating Scale-Third Edition. (2021) Retrieved August 28, 2021, from https://www.pearsonassessments.com/store/usassessments/en/Store/ProfessionalAssessments/Behavior/Gilliam-Autism-Rating-Scale Simek, A. N., & Wahlberg, A. C. (2011). Test Review: Autism Spectrum Rating Scales. Journal of Psychoeducational Assessment, 29(2), 191- 195 https://doi.org/10.1177/0734282910375408 Vaughan, C. A. (2011). Test Review: E. Schopler, M. E. Van Bourgondien, G. J. Wellman, & S. R. Love Childhood Autism Rating Scale (2nd ed.). Los Angeles, CA: Western Psychological Services, 2010. Journal of Psychoeducational Assessment, 29(5), 489–493. https://doi.org/10.1177/0734282911400873 Volker, M. A., Dua, E. H., Lopata, C., Thomeer, M. L., Toomey, J. A., Smerbeck, A. M., Rodgers, J. D., Popkin, J. R., Nelson, A. T., & Lee, G. K. (2016). Factor Structure, Internal Consistency, and Screening Sensitivity of the GARS-2 in a Developmental Disabilities Sample. Autism research and treatment, 2016, 8243079. https://doi.org/10.1155/2016/8243079 Ziublsky, J., & Viezel, K. (2013) In Reynolds C. R., Vannest, K. J. and Fletcher-Janzen E.

Evaluation of Test Materials, Scoring, and Technology (Eds.), Gilliam autism rating scale – Second edition. John Wiley & Sons, Inc https://doi.org/10.1002/978660584.ese1021

9...