The method of score estimation does not affect main results on gender differences in a Big Five short scale
Anja Schwibbe & André Beauducel
Abstract | 
PDF of the full article
An investigation of the psychometric properties of a measure of teamwork among high school students
Brian F. French
Abstract | PDF of the full article
Differential Item Functioning on mathematics items using multilevel SIBTEST
Brian F. French, W. Holmes Finch & Juan Antonio Valdivia Vazquez
Abstract | PDF of the full article
On the hierarchical structure of the original Posner task
Karl Schweizer, Michael Altmeyer, Thomas Rammsayer & Stefan Troche
Abstract | PDF of the full article
Metric scales for emotion measurement
Martin Junge & Rainer Reisenzein
Abstract | PDF of the full article
A simulation study of person-fit in the Rasch model
Richard Artner
Abstract | PDF of the full article
The method of score estimation does not affect main results on gender differences in a Big Five short scale
Anja Schwibbe & André Beauducel
Abstract
We investigated the effect of score estimation method on the identification and quantification of gender differences in the Big Five. Group mean differences assessed with three different approach-es were tested for significance (unit-weighted sum scores, latent variables in a MGCFA model, and latent variable score predictors). We administered the BF-16, a 16 adjective measure of the Big Five proposed by Herzberg and Brähler (2006), in a sample of 300 men and 273 women. We demonstrated gender-based measurement invariance by means of multi-group confirmatory factor analysis. Outcomes showed significant gender differences in Neuroticism, Openness to Experience, and Agreeableness, with women scoring higher in these dimensions regardless of scoring method. However, effect sizes for differences on latent variable level were more pronounced.
Keywords: Big Five, gender differences, latent variable, measurement invariance, multi-group confirmatory factor analysis
Anja Schwibbe, PhD
University Medical Center Hamburg-Eppendorf
Department of Biochemistry and Molecular Cell Biology
Martinistr 52
20246 Hamburg, Germany
a.schwibbe@uke.de
An investigation of the psychometric properties of a measure of teamwork among high school students
Brian F. French, Chad M. Gotch, Jason C. Immekus & Jessica L. Beaver
Abstract
High school students need to possess a range of skills (e.g., reasoning) to function in a technologi-cally driven workforce age (21st Century Workforce Commission, 2000). Teamwork skills com-prise a foundational competency that addresses students ability to collaborate to attain a shared goal. Among high school students, there is a lack of theoretically and psychometrically sound instruments to measure teamwork (Wang, MacCann, Zhuang, Liu, & Roberts, 2009). The purpose of this study was to evaluate the psychometric properties of a teamwork measure for use among high school students. Data included students (N = 382) responses on a 26-item Teamwork scale and noncognitive and academic achievement measures. Confirmatory factor analytic results sup-ported conceptualizing the scale in terms of a bifactor model (Gibbons et al., 2007). Reliability estimates ranged from .76 to .92, respectively. Moderate to high positive correlations were found between Teamwork total and subscale scores and noncognitive and cognitive measures. Results suggest that the instrument may serve useful for the high school population, with implications for practice and research.
Key words: teamwork, validity, STEM, high school students
Brian F. French, PhD
Learning and Performance Research Center
Washington State University
Pullman, WA 99163, United States of America
frenchb@wsu.edu
Differential Item Functioning on mathematics items using multilevel SIBTEST
Brian F. French, W. Holmes Finch & Juan Antonio Valdivia Vazquez
Abstract
In many testing contexts, data are collected using a multilevel sampling design, in which clusters of individuals are sampled, and individuals within clusters are administered the assessment. Multilevel structured data lead to correlated item responses for individuals within the same cluster, in turn leading to model parameter estimation bias. Specifically, standard errors are biased which leads to increased Type I error rates. This has been shown with DIF detection analysis as well. In this study, a new multilevel version of SIBTEST (MSIBTEST) is demonstrated for DIF assessment in a multilevel data context. This study investigates DIF in the Brigance Comprehensive Inventory of Basic Skills-II (CIBS-II) mathematics assessment between girls and boys. We focus on sex differences given previous sex DIF findings in mathematics achievement relating to item components such as format. More importantly, we demonstrate a recently developed method that accounts for multilevel data structures with dichotomous items for DIF. As hypothesized, adjusting DIF statistics for clustered data resulted in fewer items flagged for DIF compared to no adjustment.
Keywords: SIBTEST, MSIBTEST, gender differences, mathematics
Brian F. French, PhD
Department of Educational Leadership
and Counseling Psychology
Cleveland Hall
Washington State University
Pullman, Washington, 99164, United States of America
frenchb@wsu.edu
On the hierarchical structure of the original Posner task
Karl Schweizer, Michael Altmeyer, Thomas Rammsayer & Stefan Troche
Abstract
The investigation of the reaction times obtained by means of the Posner task for measuring the speed of the retrieval of long-term memory contents revealed a hierarchical structure. This investigation also extended to the rule identity condition included in the original Posner task in addition to the commonly known and utilized physical and name identity conditions. Physical identity reaction times, name identity reaction times and rule identity reaction times constituted three first-order factors. The correlations among these first-order factors gave rise to the general second-order factor. Furthermore, there was an additional investigation of the causal effects among the first-order factors. The results suggested the processing stimulated by the physical identity condition as the basic source that contributes directly to the processing stimulated by the name identity condition and indirectly to the rule identity condition. Additionally there was a unique contribution of the name identity condition to the rule identity condition.
Keywords: speed of processing, retrieval of information, confirmatory factor analysis, hierarchical model
Karl Schweizer, PhD
Department of Psychology
Goethe University Frankfurt
Theodor-W.-Adorno-Platz 6
60323 Frankfurt a. M., Germany
K.Schweizer@psych.uni-frankfurt.de
Metric scales for emotion measurement
Martin Junge & Rainer Reisenzein
Abstract
The scale quality of indirect and direct scalings of the intensity of emotional experiences was investigated from the perspective of representational measurement theory. Study 1 focused on sensory pleasantness and disgust, Study 2 on surprise and amusement, and Study 3 on relief and disappointment. In each study, the emotion intensities elicited by a set of stimuli were estimated using Ordinal Difference Scaling, an indirect probabilistic scaling method based on graded pair comparisons. The obtained scale values were used to select test cases for the quadruple axiom, a central axiom of difference measurement. A parametric bootstrap test was used to decide whether the participants difference judgments systematically violated the axiom. Most participants passed this test. The indirect scalings of these participants were then linearly correlated with their direct emotion intensity ratings to determine whether they agreed with them up to measurement error, and hence might be metric as well. The majority of the participants did not pass this test. The findings suggest that Ordinal Difference Scaling allows to measure emotion intensity on a metric scale level for most participants. As a consequence, quantitative emotion theories become amenable to empirical test on the individual level using indirect measurements of emotional experience.
Keywords: Emotion intensity, measurement of emotional experience, representational measurement theory, test of measurement axioms, quadruple axiom, scaling methods, Ordinal Difference Scaling, bootstrap test
Martin Junge, PhD
Institute of Psychology
University of Greifswald
Franz-Mehring-Straße 47
17487 Greifswald, Germany
martin.junge@uni-greifswald.de
A simulation study of person-fit in the Rasch model
Richard Artner
Abstract
The validation of individual test scores in the Rasch model (1-PL model) is of primary importance, but the decision which person-fit index to choose is still not sufficiently answered. In this work, a simulation study was conducted in order to compare five well known person-fit indices in terms of specificity and sensitivity, under different testing conditions. Furthermore, this study analyzed the decrease in specificity of Andersen´s Likelihood-Ratio test in case of person-misfit, using the median of the raw score as an internal criterion, as well as the positive effect of removing suspicious respondents with the index C*. The three non-parametric indices Ht, C* and U3 performed slightly better than the parametric indices OUTFIT and INFIT. All indices performed better with a higher number of respondents and a higher number of items. Ht, OUTFIT, and INFIT showed huge deviations between nominal and actual specificity levels. The simulation revealed that person-misfit has a huge negative impact on the specificity of Andersen´s Likelihood-Ratio test. However, the removal of suspicious respondents with C* worked quite well and the nominal specificity can be almost respected if the specificity level of C* is set to 0.95.
Keywords: Rasch model, type-I- and type-II-risk, simulation study, person-fit, Andersens Likeli-hood-Ratio test
Richard Artner, PhD
University of Vienna, Austria
Richardartner@gmx.at
