Why Openness to Experience is not a Good Predictor of Job Performance PDF

Preview

Summary

INTERNATIONAL JOURNAL OF SELECTION AND ASSESSMENT VOLUME 12 NUMBER 3 SEPTEMBER 2004 Why Openness to Experience is not a Good Predictor of Job Performance Barbara Griffin and Beryl Hesketh * The University of Sydney Recent meta-analyses investigating the relationship between personality and job perfo...

Description

INTERNATIONAL JOURNAL OF SELECTION AND ASSESSMENT

VOLUME 12 NUMBER 3 SEPTEMBER 2004

Why Openness to Experience is not a Good Predictor of Job Performance Barbara Griffin and Beryl Hesketh * The University of Sydney

Recent meta-analyses investigating the relationship between personality and job performance have found that openness to experience is the least predictive of the Big Five factors. Unlike other research that has sought to explain the low criterion-validity with relation to job performance, this study explores the actual construct of openness to experience, suggesting that it consists of two dimensions that relate differentially to job performance thus reducing correlations between overall measures of openness to experience and performance criteria. Exploratory factor analysis of the six subdimensions, or facets, of the NEO PI-R (a popular measure of the Big Five factors) produced two factors of openness to experience corresponding to different areas to which people are open. A confirmatory factor analysis on a second set of data provided some support for this result. A pattern of differential relationships between the two factors and other variables including personality, biodata and supervisor-rated performance offered further support for the multidimensionality of openness to experience. The implications of these findings for future research in the selection context are discussed.

Introduction

T

he utility of using personality to predict job performance has been demonstrated through meta-analysis using the five-factor model of personality. Results have shown that one of the five factors, openness to experience, has yielded consistently low correlations with job performance (Barrick, Mount, & Judge, 2001). For example, Barrick and Mount (1991) reported a meta-analytically derived correlation of .02 between openness and job performance pooled across six occupational groups, and Salgado’s (1997) meta-analysis produced correlations of .00 and .06 using the criteria of job performance ratings and personnel data, respectively. A recent second-order meta-analysis (meta-analysis of a set of meta-analyses) indicated that of the five factors, openness to experience has the lowest true score correlation with performance across criteria and occupational groups (Barrick et al., 2001).

*Address for correspondence: Beryl Hesketh, The University of

Sydney, College of Sciences and Technology, Peter Nicol Russell (PNR) Building J02, NSW 2006, Australia. E-mail: B.Hesketh@ cst.usyd.edu.au

r Blackwell Publishing Ltd 2004, 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA.

The lack of criterion-related validity of openness to experience appears to be somewhat counterintuitive. After all, it seems quite reasonable to hypothesize that employees who are inquisitive when faced with novel situations (Judge, Thoresen, Pucik, & Welbourne, 1999), who readily adapt to change, and who creatively solve complex problems (LePine, Colquitt, & Erez, 2000) would be among an organization’s better performers. People who are high on openness to experience are also described as being imaginative, sensitive to aesthetics, independent thinkers, tolerant of ambiguity, and amenable to new ideas, experiences and perspectives (Costa & McCrae, 1992; McCrae & Costa, 1997). However, as McCrae (1993– 1994) observe, openness to experience is actually the most controversial, least understood, and least researched of the five factors. Several explanations have been posited to account for the disappointing criterion validity of the openness to experience factor. For example, openness may only be relevant in certain situations or occupational groups, such as those characterized by novelty or complexity. Studies by Bing and Lounsbury (2000), Marsh, Kiechel Koles, Boyce, and Zaccaro (2001) and George and Zhou (2001) offer some support for this suggestion. Alternatively, openness to experience might only be valid when conceptually relevant

243

BARBARA GRIFFIN AND BERYL HESKETH

244

criteria are chosen. The repeatedly found significant correlation between openness and performance in training courses (e.g., Salgado, 1997; Barrick & Mount, 1991) is one such example. A third explanation suggested by Burke and Witt (2002) is that other personality variables might moderate the openness to experience–job performance relationship. They found that low openness was detrimental to the performance of extraverts and those low in emotional stability. However, results from the Barrick et al. (2001) second-order meta-analysis led these authors to conclude that the presence of moderating variables is unlikely. These explanations involve factors other than the openness construct itself. In this paper, we propose an alternative proposition that turns the focus back to the measurement and components of the openness to experience factor. Specifically, we suggest that sub-dimensions within the domain relate differently to variables such as job performance, thus militating against finding a significant relationship between performance and broad measures of openness. Despite McCrae and Costa’s (1997, p. 828) assertion that openness to experience is ‘‘one of the broadest constructs in personality psychology,’’ few people employ measures of sub-dimensions when examining the relationship between openness to experience and job performance, using instead a single score to represent the full domain. The possibility that the predictive validity of openness to experience might be reduced by its multidimensionality is rarely considered within the field of I/O psychology. Notwithstanding the usefulness of the five-factor model of personality, there are increasing calls to investigate hierarchical taxonomies with a focus on the more specific, or lower-order, traits and not just the superordinate Big Five constructs (e.g. Barrick et al., 2001, Saucier & Ostendorf, 1999). Furthermore, some argue that the broad nature of the five factors masks important relationships when the goal of research is prediction rather than description (Hough, 1992). Even though there has been no consensus at this stage with regard to the specific sub-

dimensions of each of the Big Five factors (Saucier & Ostendorf, 1999), recent research on the factor of conscientiousness illustrates the benefit of examining subdimensions. Hough and colleagues (Hough, 1992; Hough & Ones, 2001; Hough & Paullin, 1994; Schneider, Hough, & Dunnette, 1996) argue that conscientiousness consists of two main sub-dimensions, achievement and dependability, and that although these sub-dimensions may relate to each other they have a distinctive pattern of differential relationships with other variables. In support, LePine, Colquitt, and Erez (2000) showed that achievement correlated positively with performance in the midst of change whereas dependability actually had a negative correlation in this situation. Likewise Hough (1992), Griffin and Hesketh (2001) and Hough, Ones, and Viswesvaran (1998) report that achievement and dependability relate differently to a range of criteria and other predictor variables. One of the measures of the Big Five factors most frequently used in research, the NEO PI-R (Costa & McCrae, 1992), was developed to measure six subdimensions (so-called ‘‘facets’’) for each of the five factors. Its openness to experience sub-dimensions are Fantasy, Aesthetics, Feelings, Actions, Ideas and Values. Table 1 provides descriptions of each of these and the corresponding dimensions from the International Personality Item Pool (IPIP) instrument (Goldberg, 2000). According to these definitions, it would appear plausible to hypothesize that some sub-dimensions (or facets), such as Actions, might be more likely to relate to job performance than other sub-dimensions, such as Fantasy. McCrae (1993– 1994) explains that ‘‘the facets correspond to specific areas to which individuals are relatively open or closed’’ (p. 41). However, as Saucier and Ostendorf (1999, p. 615) observe, ‘‘It would be quite astonishing if there were exactly six facets for each of the broad personality factors’’. Considering the definitions provided in Table 1, we suggest that conceptually the six facets might be grouped into

Table 1. NEO PI-R openness to experience facets: definitions and corresponding IPIP facets NEO PI-R facet

IPIP facet

R (rc)*

Definition of high scorers Have a vivid imagination & fantasy life which they believe enhances life Highly esteem and can be moved by art, music, poetry & beauty Are receptive to inner feelings, deeply experience emotions & see them as important Have a willingness to experience new activities, foods, places & prefer novelty to routine Open-mindedness & willingness to consider new ideas & pursue intellectual interests Willingness to re-examine social, political & religious values

Fantasy

Imagination

.74 (.90)

Aesthetics Feelings

Artistic Interests Emotionality

.80 (.95) .70 (.90)

Actions

Adventurousness

.71 (.99)

Ideas

Intellect

.80 (.95)

Values

Liberalism

.70 (.86)

Note: *r 5 correlation between the two scales, rc 5 correlation corrected for unreliability (Goldberg, 2000).

International Journal of Selection and Assessment

r Blackwell Publishing Ltd 2004

OPENNESS TO EXPERIENCE AND JOB PERFORMANCE

two sub-dimensions. Actions, Ideas and Values appear to describe areas external to the person, that is, they describe openness towards the external environment, whereas it appears that Fantasy and Feelings describe openness to areas internal, or inside, the person. We include Aesthetics within the latter sub-dimension because the experience appears to be internal, even though it is generated from the external environment. Although organizational research at a sub-dimension level is scant, empirical findings from across a range of psychological disciplines support the suggestion that the sub-dimensions of openness (as represented by the NEO PI-R facets) might cluster together to form two separate factors that are conceptually distinct. A study that examined the genetic influences on the NEO PI-R facets using two samples of Canadian and German twins found that openness to experience formed two separate genetic factors, one consisting of Fantasy, Aesthetics and Feelings and the other of Actions, Values and Ideas (Jang, Livesley, Angleitner, Riemann, & Vernon, 2002). The authors named the factors ‘‘Reflection’’ and ‘‘Curiosity’’, respectively, but noted that there was enough overlap to justify grouping them into a global measure. A Spanish study (Carrillo, Rojo, Sanchez-Bernardos, & Avia, 2001) examined the relationship between depression and these openness sub-dimensions. Fantasy (openness to internal experience) was associated with depression and neuroticism whereas Actions (openness to external experience) was negatively correlated with neuroticism and positively associated with well-being (or lack of depression). Similarly, Wolfenstein and Trull (1997) found that depressed subjects had higher scores on Aesthetics and Feelings (openness to internal experience) than non-depressed subjects. Watson (2001), investigating the antecedents of procrastination, showed that it was related to Fantasy but not to the other sub-dimensions. Although Colquitt, Hollenbeck, Ilgen, LePine, and Sheppard (2002) group the NEO sub-dimensions differently to that hypothesized in this study (i.e., ‘‘Intellect’’ 5 Actions, Ideas, and Fantasy; ‘‘Emotion’’ 5 Aesthetics, Feelings and Values), they also found differential relationships between sub-dimensions of openness and criteria. Finally, in a study that examined faking responses to items of the NEO PI-R (Griffin, Hesketh, & Grayson, 2004), it appeared that job applicants faked good by inflating their scores on Actions, Ideas and Values (openness to external experience components) and reducing their scores on Fantasy and Feelings and on some individual items within the facet of Aesthetics (openness to internal experience). The authors concluded that behaviors within the ‘‘openness to internal experience’’ subdimension are possibly perceived as being detrimental for a work context while those typifying ‘‘openness to external experience’’ are more likely to be perceived as beneficial. In this study, we examine measures of openness to experience sub-dimensions in samples of employees from three separate organizations and career areas. The research

r Blackwell Publishing Ltd 2004

245

was part of a larger series of studies examining adaptability (see Griffin & Hesketh, 2003; Griffin 2003), and therefore most of the measures had hypothesized links to the construct of adaptability. In that series, adaptability was operationalized in both the predictor and performance domains to provide a model for use in selection contexts. Within the predictor domain we proposed that adaptability consisted of four different categories of variables: trait (or personality) adaptability, cognitive flexibility, behavioral adaptability, and self-efficacy for behaving adaptively. Openness to experience was included as an indicator of trait adaptability. Adaptive performance was the criterion variable. Allworth and Hesketh (1999) were the first to propose that the task/contextual model of performance be extended to include adaptive performance, referring to those aspects of performance related to changing job requirements. They argued that task and contextual performance reflect ongoing behavior in a maintenance or steady state, while adaptive performance manifests itself in both task and contextual areas where job requirements change. Since then, Pulakos, Arad, Donovan, and Plamondon (2000) and Pulakos, Schmitt, Dorsey, Arad, Hedge, and Borman (2002) identified eight sub-dimensions of adaptive performance, namely dealing with emergencies or crisis situations, coping with work stress, solving problems creatively, coping with uncertain/unpredictable work situations, learning new work tasks, technologies and procedures, demonstrating interpersonal adaptability, demonstrating cultural adaptability, and demonstrating physically oriented adaptability. In the light of the above, we hypothesize that: 1. The domain of openness to experience consists of two sub-dimensions, one (openness to external experience) containing the NEO PI-R facets of Actions, Ideas and Values and the other (openness to internal experience) the NEO PI-R facets of Fantasy, Feelings and Aesthetics. 2. Only the openness to external experience sub-dimension is conceptually related to adaptability and therefore the construct validity of the two sub-dimensions will be evidenced by a differential relationship between them and self-reported measures of adaptability. 3. Openness to external experience will relate positively to supervisor ratings of job performance while openness to internal experience will be negatively related.

Method Participants In this study, we examine measures of the sub-dimensions of openness to experience in samples of employees from three different organizations and career areas. Sample 1. The first sample included 93 first-year medical interns working in the public hospital system.

Volume 12 Number 3 September 2004

BARBARA GRIFFIN AND BERYL HESKETH

246

Self-report data were collected during the participants’ final month as students, while performance data were collected 6 months later, during their first year of employment. Of the sample, 53% were male and 47% female. Their mean age was 28.06 years. Sample 2. The second sample consisted of 184 information technology employees of a multinational company. Of these, 67% were males and 33% females. Their average age was 40.81 years. Although 64% of the participants had worked with the company for less than 5 years, the mean tenure was 5.27 years (SD 5 5.60). Sample 3. The third sample consisted of 70 early entry employees in a large public service organization. Those who participated in the current study had an average age of 27.91 years, 30% were male and 70% were female, and all were in their seventh month of employment. They worked as financial officers throughout the organization.

Measures Data from Sample 1 were collected 18 months prior to those collected from Samples 2 and 3. During the intervening time the model of adaptability under study was further developed and therefore additional measures were used to measure the predictor domain for the latter two groups.

Openness to experience. Sample 1 participants completed the NEO PI-R measure of openness to experience (Costa & McCrae, 1992). The stability of the NEO PI-R’s factor structure has been tested in multiple cultural contexts, showing a consistent structure, albeit the openness to experience items have not always loaded consistently (Anderson & Ones, 2003). Construct, convergent, and divergent validity evidence of the scales has been collected through a series of studies showing that the NEO PI-R scales correlated with analogous scales from other instruments (Costa & McCrae, 1992). A score for the domain of openness to experience was calculated from 48 items measured on a five-point Likert scale that yielded a coefficient alpha for the current sample of .88. Facet scores were calculated by averaging the eight items for each facet. Coefficient alphas are reported in Table 2. Participants in Samples 2 and 3 completed the IPIP (Goldberg, 2000) measure of openness to experience that consisted of 48 items for the domain score and eight items for each facet score. The IPIP is a broad-bandwidth, public domain personality inventory that measures the five-factor model with scales developed to directly reflect several existing instruments including the NEO PI-R. The facet names, equivalent NEO PI-R facets, and definitions of the scales are presented in Table 1. Table 2 reports the coefficient alphas derived for each sample.

Table 2. Means, standard deviations, and coefficient alphas Sample 1 (N 5 93)

Sample 2 (N 5 184)

Mean

SD

Alpha

Mean

SD

Openness to experience Fantasy (imagination) Feelings (emotionality) Aesthetics (artistic interests) Actions (adventurousness) Ideas (intellect) Values (liberalism)

3.65 3.46 3.92 3.55 3.33 3.77 3.87

.36 .49 .47 .64 .50 .58 .58

.88 .63 .70 .78 .66 .77 .77

3.43 3.31 3.32 3.59 3.78 3.71 2.99

Self-report adaptability variables Variety seeking Conventional Change receptiveness Neuroticism Change biodata Self-efficacy Job tension

3.46 2.94 – – – – –

.58 .49 – – – – –

.69 .66 – – – – –

Performance Task performance Adaptive performance Interpersonal adaptability New learning Deals with crises Copes with stress

7.78 7.55 7.70 7.64 7.64 7.68

.85 .68 .91 .85 .79 .78

.86 .88 .89 .82 .84 .78

International Journal of Selection and Assessment

Sample 3 (N 5 70)

Alpha

Mean

SD

Alpha

.39 .58 .50 .69 .55 .59 .62

.87 .73 .65 .81 .73 .80 .68

3.55 3.43 3.64 3.67 3.56 3.91 3.08

.42 .57 .67 .83 .58 .63 .56

.89 .75 .79 .86 .78 .82 .75

– 2.95 3.40 2.58 3.13 3.81 2.69

– .46 .69 .65 .49 .60 .81

– .70 .77 .85 .77 .92 .75

3.53 – 3.55 – 3.13 3.87 –

.63 – .60 – .68 .52 –

.79 – .68 – .79 .85 –

6.55 6.05 – – – –

1.14 1.12 – – – –

.91 .97 – – – –

6.90 6.81 – – – –

.94 .94 – – – –

.91 .98 – – – –

r Blackwell Publishing Ltd 2004

OPENNESS TO EXPERIENCE AND JOB PERFORMANCE

Variety...