Note: Machine Learning Modeling and Optimization Techniques in Psychological Assessment
David Goretzko & Markus Bühner
PDF of the full article

Using BIC and aBIC to develop Bayesian posterior probabilities for latent variable models
W. Holmes Finch
PDF of the full article

NICOTINE AND CAFFEINE USERS’ BLOOD PRESSURE REGULATION DURING SUSTAINED VISUAL ATTENTION
Alexandros M. Semertzi & Christiane Lange-Küttner
PDF of the full article

Comparison of Different Approaches to Dealing with Guessing in Rasch Modeling
Hong Jiao
PDF of the full article

Detecting Illusory Halo Effects in Rater-Mediated Assessment: A Mixture Rasch Facets Modeling Approach
Thomas Eckes & Kuan-Yu Jin
PDF of the full article

Alea iacta est - Development and evaluation of a manipulation approach of the item dif-ficulty in the distractor-based format of the cube construction test
Julie Levacher, Marco Koch, Maximilian Bungart, Frank M. Spinath & Nicolas Becker 
PDF of the full article

Note: Machine Learning Modeling and Optimization Techniques in Psychological Assessment
David Goretzko & Markus Bühner

Abstract
Recently, machine learning modeling has made its way into psychological research. While it is used mostly in regression or classification contexts to optimize the prediction of certain variables, its principles and techniques also have arrived in psychometrics and psychological assessment. In this paper, we present machine learning and optimization concepts that can be used for different aspects of questionnaire development and test construction focusing on four central issues – item development and item selection, dimensionality assessment in latent variable modeling, improving the generalizability of factor models as well as the evaluation of measurement invariance or differential item functioning. By introducing different machine learning techniques and newly developed methods, we want to encourage researchers to try out these tools and upgrade their psychometrics toolboxes.

Keywords: Psychometrics; Regularization; Machine Learning; Latent Variable Modeling; Recursive Partitioning
 

David Goretzko
LMU Munich Department of Psychology 
Leopoldstr. 13 
80802 Munich 
david.goretzko@psy.lmu.de

Using BIC and aBIC to develop Bayesian posterior probabilities for latent variable models
W. Holmes Finch

Abstract
Use of structural equation models (SEMs) to investigate relationships between latent variables has become increasingly widespread over the last 20 years. The use of SEM involves fitting multiple plausible models and selecting the one that provides optimal fit. A common approach for determining the optimal model involves use of information indices, which combine model misfit with a penalty for model complexity, with a minimum value being optimal. One concern regarding this strategy is that it does not acknowledge uncertainty inherent in the process. Wu, et al. (2020) described a Bayesian approach to quantifying this uncertainty and demonstrated its utility in the context of observed variable path models. This simulation study extends the work of Wu, et al. to the case of latent variable SEMs. Results demonstrate that the Bayesian approach does work well in the latent variable context. Implications for practice are discussed.

Keywords: Structural equation modeling; model selection; Bayesian information criterion

Holmes Finch
Department of Educational Psychology 
Muncie, IN, 47306. USA,
whfinch@bsu.edu

NICOTINE AND CAFFEINE USERS’ BLOOD PRESSURE REGULATION DURING SUSTAINED
VISUAL ATTENTION
Alexandros M. Semertzi & Christiane Lange-Küttner

Abstract
Smoking and caffeine improves some aspects of cognitive performance but it also brings with it some serious cardiovascular health risks. We investigated whether quiet and focused visual attention can reduce blood pressure in nicotine and coffee consumers. Participants either smoked (n=40), or drank coffee (n=40) on a daily basis. The control group neither smoked nor drank coffee (n=40), total n=120. We measured blood pressure before and after the Attentional Blink Task (ABT) which consists of a visual perception task that requires attention to very fast appearing and disappearing targets without and with delay due to distracters. Performance gains due to nicotine and caffeine were limited to immediate perception but were sensitive to delay. The nicotine group had a significantly higher systolic and the caffeine group a significantly higher diastolic blood pressure before the ABT, but both were significantly reduced afterwards showing a blood pressure regulation effect of sustained, focused visual attention.

Keywords: caffeine; nicotine; attentional blink task; sustained visual attention, blood pressure regulation

Prof Dr C. Lange-Küttner
orcid.org/0000-0001-6021-8931 ;
University of Greifswald 
Dept. of Psychology 
Franz-Mehring-Straße 47 
17489 Greifswald, Germany
christiane.lange-kuettner@uni-greifswald.de

Comparison of Different Approaches to Dealing with Guessing in Rasch Modeling
Hong Jiao

Abstract
This study compared three approaches to dealing with guessing in Rasch modeling: explicit modeling of guessing effects, correction of guessing effects, and the Rasch model which does not model guessing explicitly. The extended Rasch model explicitly includes a lower asymptote parameter in the Rasch model to account for guessing. Parameter estimation was explored using a Bayesian approach for the extended Rasch model with guessing. Further, model parameter estimates were compared with those from the Rasch model and the Rasch model with the correction procedure for guessing effects under different study conditions. The results indicated that the true model parameters could be well recovered by the Bayesian estimation method developed in OpenBUGS. Ignoring guessing in general led to the overestimation of test information, underestimation of item difficulty, and misrepresentation of the maximum test information
location.

Keywords: The Rasch model; guessing; test information; MCMC Bayesian estimation

Hong Jiao 
Measurement, Statistics and Evaluation 
Department of Human Development and Quantitative Methodology 
1230C Benjamin Building 
University of Maryland, College Park
MD 20742, USA 
hjiao@umd.edu

Detecting Illusory Halo Effects in Rater-Mediated Assessment: A Mixture Rasch Facets Modeling Approach
Thomas Eckes & Kuan-Yu Jin

Abstract
Halo effects refer to a common source of error in human judgment. In rater-mediated assessments where each rater assigns multiple scores to examinees, raters subject to halo tend to give similar scores on conceptually distinct traits, dimensions, or criteria. An intricate problem with empirically detecting halo effects concerns the separation between illusory halo due to judgmental biases or cognitive distortions, and true halo, due to actual overlap between the traits or criteria used for scoring examinee performances. The present research used the mixture Rasch facets model for halo effects (MRFM-H; Jin & Chiu, 2022) to detect illusory halo. In two separate studies, raters scored examinees' writing performances on a set of criteria using a fourcategory rating scale. Halo parameters were estimated building on Bayesian Markov chain Monte Carlo methods implemented in the freeware JAGS run within the R environment. The findings revealed that (a) the MRFM-H fit the data well but not better than the basic Rasch facets model (RFM), (b) in Study 2, we identified three raters that may have been subject to illusory halo effects. The discussion focuses on practical implications for ensuring high rating quality in performance assessments.

Keywords: rater effects, illusory halo, mixture Rasch model, Bayesian statistics, model fit

Thomas Eckes, PhD
TestDaF Institute
University of Bochum 
Universitätsstr. 134 
44799 Bochum, Germany
e-mail: thomas.eckes@gast.de 
orcid.org/0000-0002-8820-5902

Alea iacta est - Development and evaluation of a manipulation approach of the item dif-ficulty in the distractor-based format of the cube construction test
Julie Levacher, Marco Koch, Maximilian Bungart, Frank M. Spinath & Nicolas Becker

Abstract
Spatial reasoning is a component of intelligence with very high loadings on the g factor and which has high practical importance for personnel selection. To this extent, much research has been conducted regarding the measurement of spatial reasoning. Oftentimes, tests require par-ticipants to construct a mental representation of an object and to manipulate that object in order to solve an item. Past endeavours to develop distractor-based tests have resulted in rather nar-row distributions of item difficulties, reducing their usefulness for selection procedures. Thus, in the current study, one test of spatial reasoning has been administered to a sample of N = 116 psychology students with the aim of developing a manipulation approach to provide a broad range of item difficulties. The resulting items were of average difficulty (pi = .55) with a wide range (.18 ≤ pi ≤ .92). Compared to previous research, item parameters are in a range that is more useful for selection procedures. Albeit no predictor for item difficulty could be identified, an exploratory analysis suggested that this might be caused by non-overlapping items between the original cube construction task and the newly developed distractor based task. Nonetheless, this study provided a new approach to the manipulation of item difficulties in spatial reasoning tasks which can benefit practitioners when conducting personnel selection and researchers when developing new diagnostic instruments.

Keywords: spatial reasoning, mental rotation, test-development, distractor-based tests, item pa-rameter manipulation

Julie Levacher
Campus A1.3 
66123 Saarbrücken, Germany 
julie.levacher@uni-saarland.de

Psychological Test and Assessment Modeling
Volume 64 · 2022 · Issue 1
Pabst, 2022
ISSN 2190-0493 (Print)
ISSN 2190-0507 (Internet)