The Geneva Emotional Competence Test (GECo): An ability measure of workplace emotional intelligence.
Katja Schlegel and Marcello Mortillaro
Abstract:
Emotional intelligence (EI) has been frequently studied as a predictor of work criteria, but disparate approaches to defining and measuring EI have produced rather inconsistent findings. The conceptualization of EI as an ability to be measured with performance-based tests is by many considered the most appropriate approach, but only few tests developed in this tradition exist, and none of them is designed to specifically assess EI in the workplace. The present research introduces the Geneva Emotional Competence test (GECo)—a new ability EI test measuring emotion recognition (assessed using video clips of actors), emotion understanding, emotion regulation in oneself, and emotion management in others (all assessed with situational judgment items of work-related scenarios). For the situational judgment items, correct and incorrect response options were developed using established theories from the emotion and organizational field. Five studies (total N = 888) showed that all subtests had high measurement precision (as assessed with Item Response Theory), and correlated in expected ways with other EI tests, cognitive intelligence, personality, and demographic variables. Further, the GECo predicted performance in computerized assessment center tasks in a sample of professionals, and explained academic performance in students incrementally above another ability EI test. Because of its theory-based scoring, good psychometric properties, and focus on the workplace, the GECo represents a promising tool for studying the role of four major EI components in organizational outcomes.
Katja Schlegel and Marcello Mortillaro
Abstract:
Emotional intelligence (EI) has been frequently studied as a predictor of work criteria, but disparate approaches to defining and measuring EI have produced rather inconsistent findings. The conceptualization of EI as an ability to be measured with performance-based tests is by many considered the most appropriate approach, but only few tests developed in this tradition exist, and none of them is designed to specifically assess EI in the workplace. The present research introduces the Geneva Emotional Competence test (GECo)—a new ability EI test measuring emotion recognition (assessed using video clips of actors), emotion understanding, emotion regulation in oneself, and emotion management in others (all assessed with situational judgment items of work-related scenarios). For the situational judgment items, correct and incorrect response options were developed using established theories from the emotion and organizational field. Five studies (total N = 888) showed that all subtests had high measurement precision (as assessed with Item Response Theory), and correlated in expected ways with other EI tests, cognitive intelligence, personality, and demographic variables. Further, the GECo predicted performance in computerized assessment center tasks in a sample of professionals, and explained academic performance in students incrementally above another ability EI test. Because of its theory-based scoring, good psychometric properties, and focus on the workplace, the GECo represents a promising tool for studying the role of four major EI components in organizational outcomes.
Content validation guidelines: Evaluation criteria for definitional correspondence and definitional distinctiveness.
Jason A. Colquitt, Tyler B. Sabey, Jessica B. Rodell, and Edwyna T. Hill
Abstract:
Several reviews have been critical of the degree to which scales in industrial/organizational psychology and organizational behavior adequately reflect the content of their construct. One potential reason for that circumstance is a tendency for scholars to focus less on content validation than on other validation methods (e.g., establishing reliability, performing convergent, discriminant, and criterion-related validation, and examining factor structure). We provide clear evaluation criteria for 2 commonly used content validation approaches: Anderson and Gerbing (1991) and Hinkin and Tracey (1999). To create those guidelines, we gathered all new scales introduced in Journal of Applied Psychology, Academy of Management Journal, Personnel Psychology, and Organizational Behavior and Human Decision Processes from 2010 to 2016. We then subjected those 112 scales to Anderson and Gerbing’s (1991) and Hinkin and Tracey’s (1999) approaches using 6,240 participants from Amazon’s Mechanical Turk with detailed, transparent, and replicable instructions. For both approaches, our results provide evaluation criteria for definitional correspondence—the degree to which a scale’s items correspond to the construct’s definition—and definitional distinctiveness—the degree to which a scale’s items correspond more to the construct’s definition than to the definitions of other orbiting constructs.
Jason A. Colquitt, Tyler B. Sabey, Jessica B. Rodell, and Edwyna T. Hill
Abstract:
Several reviews have been critical of the degree to which scales in industrial/organizational psychology and organizational behavior adequately reflect the content of their construct. One potential reason for that circumstance is a tendency for scholars to focus less on content validation than on other validation methods (e.g., establishing reliability, performing convergent, discriminant, and criterion-related validation, and examining factor structure). We provide clear evaluation criteria for 2 commonly used content validation approaches: Anderson and Gerbing (1991) and Hinkin and Tracey (1999). To create those guidelines, we gathered all new scales introduced in Journal of Applied Psychology, Academy of Management Journal, Personnel Psychology, and Organizational Behavior and Human Decision Processes from 2010 to 2016. We then subjected those 112 scales to Anderson and Gerbing’s (1991) and Hinkin and Tracey’s (1999) approaches using 6,240 participants from Amazon’s Mechanical Turk with detailed, transparent, and replicable instructions. For both approaches, our results provide evaluation criteria for definitional correspondence—the degree to which a scale’s items correspond to the construct’s definition—and definitional distinctiveness—the degree to which a scale’s items correspond more to the construct’s definition than to the definitions of other orbiting constructs.
Generalized exchange orientation: Conceptualization and scale development.
Katsuhiko Yoshikawa, Chia-Huei Wu, Hyun-Jung Lee
Abstract:
We conceptualize generalized exchange orientation, and develop and validate a scale assessing individual orientations toward generalized exchange as well as reciprocal and negotiated exchange for offering a full set of measurements for social exchange orientation. Through 4 phases and using data from 1,408 participants, we established factorial, nomological, discriminant, and incremental validity of the social exchange orientation Scale and examined measurement invariance of the scale between samples from the United States and Japan. First, our findings indicate the newly developed scale for generalized exchange orientation (GEO) is distinct from measures of other forms of social exchange orientation and prosocial orientation. Second, the GEO Scale shows incremental validity as it predicts citizenship behaviors toward individuals above and beyond the key dispositional antecedents known in the literature. Implications and future research on generalized exchange are discussed.
Katsuhiko Yoshikawa, Chia-Huei Wu, Hyun-Jung Lee
Abstract:
We conceptualize generalized exchange orientation, and develop and validate a scale assessing individual orientations toward generalized exchange as well as reciprocal and negotiated exchange for offering a full set of measurements for social exchange orientation. Through 4 phases and using data from 1,408 participants, we established factorial, nomological, discriminant, and incremental validity of the social exchange orientation Scale and examined measurement invariance of the scale between samples from the United States and Japan. First, our findings indicate the newly developed scale for generalized exchange orientation (GEO) is distinct from measures of other forms of social exchange orientation and prosocial orientation. Second, the GEO Scale shows incremental validity as it predicts citizenship behaviors toward individuals above and beyond the key dispositional antecedents known in the literature. Implications and future research on generalized exchange are discussed.
The Multidimensional Workaholism Scale: Linking the conceptualization and measurement of workaholism.
Malissa A. Clark, Rachel W. Smith, Nicholas J. Haynes
Abstract:
Scholarly interest in workaholism has increased dramatically in recent years. This research has underscored the detrimental effects of workaholism for employees, their families, and the organizations that employ them. Despite drastic improvements in the quality of studies examining workaholism over the past several decades, researchers continue to almost exclusively rely on older measures of workaholism or new measures derived from these original measures. In the present study, we outline why a new measure is needed and propose a multidimensional conceptualization of workaholism that encompasses motivational, cognitive, emotional, and behavioral dimensions. We then develop and validate a new multidimensional measure of workaholism: the Multidimensional Workaholism Scale (MWS). Evidence from 5 samples representing individuals working in a wide variety of occupations and industries throughout the United States (total N = 1,252) provides support that (a) our proposed 4-factor structure replicates and fits better than alternative models; (b) the measure demonstrates high reliability and content validity; (c) the measure demonstrates evidence for convergent and discriminant validity with constructs in workaholism’s nomological network; (d) the measure demonstrates incremental validity in the prediction of important outcomes over and above prior measures of workaholism; and (e) the different dimensions demonstrate incremental validity in the prediction of specific outcomes over and above other dimensions of the MWS. Overall, results from the present study suggest that the MWS is a reliable and valid measure that can advance a more nuanced approach to research and practice relating to workaholism.
Malissa A. Clark, Rachel W. Smith, Nicholas J. Haynes
Abstract:
Scholarly interest in workaholism has increased dramatically in recent years. This research has underscored the detrimental effects of workaholism for employees, their families, and the organizations that employ them. Despite drastic improvements in the quality of studies examining workaholism over the past several decades, researchers continue to almost exclusively rely on older measures of workaholism or new measures derived from these original measures. In the present study, we outline why a new measure is needed and propose a multidimensional conceptualization of workaholism that encompasses motivational, cognitive, emotional, and behavioral dimensions. We then develop and validate a new multidimensional measure of workaholism: the Multidimensional Workaholism Scale (MWS). Evidence from 5 samples representing individuals working in a wide variety of occupations and industries throughout the United States (total N = 1,252) provides support that (a) our proposed 4-factor structure replicates and fits better than alternative models; (b) the measure demonstrates high reliability and content validity; (c) the measure demonstrates evidence for convergent and discriminant validity with constructs in workaholism’s nomological network; (d) the measure demonstrates incremental validity in the prediction of important outcomes over and above prior measures of workaholism; and (e) the different dimensions demonstrate incremental validity in the prediction of specific outcomes over and above other dimensions of the MWS. Overall, results from the present study suggest that the MWS is a reliable and valid measure that can advance a more nuanced approach to research and practice relating to workaholism.
From alpha to omega and beyond! A look at the past, present, and (possible) future of psychometric soundness in the Journal of Applied Psychology.
Jose M. Cortina, Zitong Sheng, Sheila K. Keener, Kathleen R. Keeler, Leah K. Grubb, Neal Schmitt, Scott Tonidandel, Karoline M. Summerville, Eric D. Heggestad, and George C. Banks
Abstract:
The psychometric soundness of measures has been a central concern of articles published in the Journal of Applied Psychology (JAP) since the inception of the journal. At the same time, it isn’t clear that investigators and reviewers prioritize psychometric soundness to a degree that would allow one to have sufficient confidence in conclusions regarding constructs. The purposes of the present article are to (a) examine current scale development and evaluation practices in JAP; (b) compare these practices to recommended practices, previous practices, and practices in other journals; and (c) use these comparisons to make recommendations for reviewers, editors, and investigators regarding the creation and evaluation of measures including Excel-based calculators for various indices. Finally, given that model complexity appears to have increased the need for short scales, we offer a user-friendly R Shiny app (https://orgscience.uncc.edu/about-us/resources) that identifies the subset of items that maximize a variety of psychometric criteria rather than merely maximizing alpha.
Jose M. Cortina, Zitong Sheng, Sheila K. Keener, Kathleen R. Keeler, Leah K. Grubb, Neal Schmitt, Scott Tonidandel, Karoline M. Summerville, Eric D. Heggestad, and George C. Banks
Abstract:
The psychometric soundness of measures has been a central concern of articles published in the Journal of Applied Psychology (JAP) since the inception of the journal. At the same time, it isn’t clear that investigators and reviewers prioritize psychometric soundness to a degree that would allow one to have sufficient confidence in conclusions regarding constructs. The purposes of the present article are to (a) examine current scale development and evaluation practices in JAP; (b) compare these practices to recommended practices, previous practices, and practices in other journals; and (c) use these comparisons to make recommendations for reviewers, editors, and investigators regarding the creation and evaluation of measures including Excel-based calculators for various indices. Finally, given that model complexity appears to have increased the need for short scales, we offer a user-friendly R Shiny app (https://orgscience.uncc.edu/about-us/resources) that identifies the subset of items that maximize a variety of psychometric criteria rather than merely maximizing alpha.