Exploring Reliability AND Validity 521 PDF

Preview

Summary

score 48 out of 50...

Description

EXPLORING RELIABILITY AND VALIDITY

Exploring Reliability and Validity

Cara Soto School of Behavioral Sciences in Mental Health Counseling, Liberty University

Author Note Cara Soto I have no known conflict of interest to disclose. Correspondence concerning this article should be addressed to Cara Soto Email: [email protected]

1

EXPLORING RELIABILITY AND VALIDITY

2

Exploring Reliability and Validity Assignment In this paper we will explore the different psychometric properties of the Values and Motives Questioner (VMQ) the technical manual (2020). The purpose of this paper is to compare the VMQ with acceptable measures to insure or question that validity of the assessment. Types of Reliability and Validity There are two types of reliability explained in this manual. However, the test that is used to measure the internal consistency is Cronbach’s Alpha. This test measures the internal correlation of the items on the scale and as explained, Cronbach’s Alpha does not measure if the assessment of values and motives are applicable to the test taker (Values and motives questionnaire the technical manual, 2020). The validity of a test for an assessment is preformed to demonstrate that the assessment is measuring the values and motives of a person. A validity test will demonstrate a correlation between the test results and the individual's values and motives. For the VMQ validation the use of construct validity was used. This measurement of the VMQ is consistent with other tests that measure similar values, the values and motives questionnaire the technical manual states (2020), that must be conducted after the reliability because it places an upper limit on the test’s validity. Reliability: Cronbach Alpha Coefficients. Interpretation of reliability coefficient

Finance Aesthetics Affection Security Altruistic Affiliative Traditional Ethical Moral

excellent r=.90 and up

good r=.80 – .89

adequate r=.70 – .79

may have limited applicability below r=.70

.83 .83 .79 .79 .74 .74 .70 .70 .68

EXPLORING RELIABILITY AND VALIDITY Independence S.D. Achievement Infrequency

3 .66 .64 .53 .52

The results of the Cronbach's alpha test show that only 15.4% of the test rates as good. 46.1% rates as being adequate, and the rest of the test falls below applicability at 38.5%. This scoring results in 61.5% of the assessment questions being homogeneous, while 38.5% have limited consistency. Sample Size and Nature of the Population a. Validity- to test the inter-correlations of the VMI assessment, a selection of 159 psychology and MBA students were chosen to participate. The relationship between the VMI and MAPP was tested with 59 psychology under-grad students. In testing the relationship between VQI and 16PF, 100 MBA students from a prestigious business school in London were chosen. For the last comparison I was a little confused because it says in the title that it is measuring OPP and VQI, however the sample states that 59 psychology undergrads were used to complete the VIM and MAPP (Values and motives questionnaire the technical manual, 2020). Then the paper goes on to explain the relationship between VMI and OPP. Therefore, I'm not certain of the sample group of the last test. The groups do have to be similar, therefore, I would like to know if the psychology students for the VQI assessment were grad students or under grads. I think this would make a difference based on the amount of education that they have. It is possible that this could reflect in the validity of the test. b. VMQ norming population- the assessment says that it can be used for counseling, coaching and vocational guidance with mature clients (Values and motives, 2020). It does not say

EXPLORING RELIABILITY AND VALIDITY

4

whether there are different cultures that are sampled. Genders are represented however, it is not an equal distribution. I do not believe that the group is broad enough to generalize to the greater population. I don't believe they sampled enough people from diverse backgrounds to capture the traits of the population, and also more information is needed about the level of education that the psychology students have is another concerning factor. Your Opinion of the VMQ The psychometric properties that were found for the reliability were the Cronbach’s alpha which in my eye's proved not to be as reliable as the authors would like it to be. Furthermore, 84% of the assessment is adequate or below adequate, with no percent being in the category of excellent. The authors state they are trying to approve and test more items to put on the assessment. This is a cause for concern according to Goeman & Jong (2018), adding more items will increase the percentage for the r because Cronbach's alpha is statistical, and the more items added to the assessment higher the percentage. The VMQ states that the inter correlations are modest. According to the numbers it seems as if the VMQ is measuring what it says it measures in the individuals who took the assessment. My opinion on the population is a true concern because the authors did not use a diverse population for their sample. To be able to generalize this assessment to the general population they would need a larger sampling and pool of diverse people.

EXPLORING RELIABILITY AND VALIDITY References Values and motives questionnaire the technical manual. (2020). https://www.psytech.co.za/images/PsytechSA/VMI/VMIMan.pdf Values and motives. (2020). https://www.psytech.co.za/measures/24-values-and-motives Goeman, J. J., & Jong, N. H. (2018). How well does the sum score summarize the test? summability as a measure of internal consistency. Educational Measurement, Issues and Practice, 37(2), 54-63. https://doi.org/10.1111/emip.12181

5...