Applying multiple indices to monitor bullying longitudinally: A case of a Japanese junior high school
Tomoyuki Kanetsuna & Yuichi Toda
Abstract | 
PDF of the full article
Re-evaluating the psychometric properties of MicroFIN: A multidimensional measurement of complex problem solving or a unidimensional reasoning test?
André Kretzschmar, Liena Hacatrjana & Malgozata Rascevska
Abstract | PDF of the full article
Performances of LOO and WAIC as IRT model selection methods
Yong Luo & Khaleel Al-Harbi
Abstract | PDF of the full article
The effect of gender and academic background on cloze and reading comprehension performance using G-Theory
Hamdollah Ravand & Sahar A Sardari
Abstract | PDF of the full article
Test order effects in an online self-assessment: An experimental study
Lisbeth Weitensfelder
Abstract | PDF of the full article
Sequentially presented response options prevent the use of testwiseness cues in multiple-choice testing
Martin Papenberg, Sonja Willing & Jochen Musch
Abstract | PDF of the full article
Applying multiple indices to monitor bullying longitudinally: A case of a Japanese junior high school
Tomoyuki Kanetsuna & Yuichi Toda
Abstract
This article aimed to demonstrate the multiple ways to monitor the condition of bullying in a whole school level as well as year group and class levels by using traditional index of the change in number of reported bullying / victimization, in conjunction with new indices of the bully / victim ratio and the number of helpless victims. The study showed that along with the traditional index, by applying these new indices to monitor the prevalence and severity of bullying in different levels, we managed to have alternative views for the interpretation of the condition which enable schools and teachers to detect certain units of students that might face with a more serious situation than others, and this in turn enables them to detect and to intervene those silent helpless victims. Along with the monitoring of the condition of bullying in different levels, antibullying activities were implemented by the student committee to raise antibullying attitudes among students as well as teachers and other school staffs, and it was also examined how this affects the condition of bullying. Although the antibullying activities were conducted on a whole-school basis, there were considerable differences in its effects between classes and year groups , which may due to the time and effort students put into the activities. Some limitations and implications of the study are also discussed.
Keywords: monitoring, bully/victim ratio (BVR), number of helpless victims (NHV)
Tomoyuki Kanetsuna, PhD
Department of Psychology
Koshien University. 10-1
Momijigaoka, Takarazuka
Hyogo, JAPAN. p/c. 665-0006
kanetuna@koshien.ac.jp
Re-evaluating the psychometric properties of MicroFIN: A multidimensional measurement of complex problem solving or a unidimensional reasoning test?
André Kretzschmar, Liena Hacatrjana & Malgozata Rascevska
Abstract
The present study investigated the psychometric properties of an extended version of the MicroFIN test as the latest development in complex problem solving (CPS) assessments within the multiple complex systems (MCS) approach. Specifically, we examined factorial validity, reliability, and the relation to reasoning using data from 362 Latvian high school students (mean age 16.82 years). Results indicated that the commonly applied 2-dimensional measurement model for MCS-based CPS tests did not fit the data better than a 1-dimensional measurement model. Furthermore, the extended version of MicroFIN showed satisfactory reliability. With regard to the relation to reasoning, we found latent correlations ranging from .38 to .78 depending on the operationalization of reasoning. Explanations for the findings (e.g., the impact of the Brunswik symmetry principle) and implications for MicroFIN, the assessment of CPS in general, and future CPS research are discussed.
Keywords: complex problem solving, reasoning, factorial validity, reliability, Brunswik symmetry
André Kretzschmar, PhD
University of Tübingen
Hector Research Institute of
Education Sciences and Psychology
Europastraße 6
72072 Tübingen, Germany
kretzsch.andre@gmail.com
Performances of LOO and WAIC as IRT model selection methods
Yong Luo & Khaleel Al-Harbi
Abstract
The widely available information criterion (WAIC) and leave-one-out cross-validation (LOO) are considered fully Bayesian model selection methods due to their utilization of the whole posterior distribution other than the point estimates. Despite their theoretical advantage of being fully Bayesian, how such an advantage translates into practical performance remains unknown. In this paper, we conducted a simulation study to compare the performances of WAIC and LOO with other four commonly used methods, which are the likelihood ratio test (LRT), AIC, BIC, and DIC, in the context of dichotomous IRT model selection. We also used a real data set to illustrate that those six model selection methods can lead to different conclusions. The findings suggest that WAIC and LOO perform better than the other four methods, especially when the data were generated with 3PLM. In addition, it was found that AIC, one of the most widely used model selection method, can become inconsistent with different sample sizes and test lengths.
Key words: information criterion, model selection, IRT, WAIC, LOO
Yong Luo, PhD
National Center for Assessment
West Palm neighborhood j
King Khalid bin Abdul Aziz
P.O. Box 68566
Riyadh 11534, Saudi Arabia
jackyluoyong@gmail.com
The effect of gender and academic background on cloze and reading comprehension performance using G-Theory
Hamdollah Ravand & Sahar A Sardari
Abstract
This study employed generalizability theory to investigate the impact of academic background and gender on performance on cloze and reading comprehension performance. For this purpose, 5000 examinees who took the University Entrance Examination In Iran were studied. Results of the study showed that gender and academic background had negligible effects on reading and cloze test performance. However, the results showed a strong effect for the two-, three-, and four-way interactions of person with item, gender, and academic background. A notable finding in the present study was the highly similar effect of gender and academic background nd their interactions with item and person on both cloze and reading comprehension tests. The findings of the study may have implications for the construct equivalence of reading comprehension and cloze tests.
Keywords: generalizability theory, task type, cloze, reading comprehension, gender, academic background
Hamdollah Ravand, assistant professor
English Department
Faculty of Humanities
Vali-e-Asr University of Rafsanjan
Rafsanjan, Iran
ravand@vru.ac.ir
Test order effects in an online self-assessment: An experimental study
Lisbeth Weitensfelder
Abstract
While computer based online self-assessments become increasingly popular, not much research can be found yet on how such test batteries should be designed regarding test order, though effects on motivation and performance cant be ruled out. The given study examines in an experimental setting how test order influences dropout rates and fatigue in a computer based online self-assessment. Test order could at the most reduce overall dropouts to a limited degree - especially a combination of a warming up phase (e.g. a short questionnaire) with a subsequent high hurdle (e.g. a strenuous test) can be recommended. Contrary to studies that focused on high stakes situations, fatigue effects in strenuous tests could be found.
Key words: Context effects; self-assessment; test order; fatigue; dropout rates
Lisbeth Weitensfelder
doctoral student at the University of Vienna
Faculty of Psychology
Division of Psychological Assessment and Applied Psychometrics
Liebiggasse 5
1010 Vienna, Austria
lisbeth.weitensfelder@univie.ac.at
Sequentially presented response options prevent the use of testwiseness cues in multiple-choice testing
Martin Papenberg, Sonja Willing & Jochen Musch
Abstract
Testwiseness - the ability to find subtle cues to the solution by comparing all available response options-threatens the validity of multiple-choice (MC) tests. Discrete-option multiple-choice (DOMC) is an alternative testing format in which response options are presented sequentially rather than simultaneously. A test consisting of items that included cues to their solutions was constructed to test whether DOMC testing allows for a better control of testwiseness than MC testing. Although test items were generally more difficult in the DOMC than in the MC format, the availability of item cues led to an increase in test scores that was considerably larger in the MC condition. DOMC was thus shown to allow for a better control of testwiseness than MC. DOMC testing also reduced the number of response options that had to be presented. The DOMC format therefore seems to offer an interesting alternative to traditional MC testing.
Keywords: discrete-option multiple-choice, item cues, sequential item presentation, testwiseness, multiple-choice testing
Martin Papenberg, PhD
University of Düsseldorf
Department of Experimental Psychology
Universitätsstr. 1, Building 23.03
40225 Düsseldorf, Germany
martin.papenberg@uni-duesseldorf.de
