Contents, Volume 51, 2009, Issue 4
DAVID FOSTER & HAROLD L. MILLER
A new format for multiple-choice testing: Discrete-Option Multiple-Choice. Results from early studies
Abstract | 
PDF of the full article
KLAUS D. KUBINGER, DIETER RASCH & TAKUYA YANAGIDA
On designing data-sampling for Rasch model calibrating an achievement test
Abstract | PDF of the full article
Special topic: Working memory and intelligence
KARL SCHWEIZER
Editorial for special topic: Working memory and intelligence
NASH UNSWORTH, GREGORY J. SPILLERS & GENE A. BREWER
Examining the relations among working memory capacity, attention control, and fluid intelligence from a dual-component framework
Abstract | PDF of the full article
KERRY LEE, MADELINE LEE PE, SU YIN ANG & LAZAR STANKOV
Do measures of working memory predict academic proficiency better than measures of intelligence?
Abstract | PDF of the full article
SYBILLE ROCKSTROH & KARL SCHWEIZER
An investigation of the effect of retest practice on the relationship between speed and ability in attention, memory and working memory tasks
Abstract | PDF of the full article
STEFAN J. TROCHE, KARL SCHWEIZER & THOMAS H. RAMMSAYER
The relationship between attentional blink and psychometric intelligence: A fixed-links model approach
Abstract | PDF of the full article
DAMIAN P. BIRNEY & DAVID B. BOWMAN
An experimental-differential investigation of cognitive complexity
Abstract | PDF of the full article
MICHAEL ALTMEYER, MICHAEL SCHREINER & KARL SCHWEIZER
A process-based account of the speed-ability relationship for the Posner Task
Abstract | PDF of the full article
A new format for multiple-choice testing: Discrete-Option Multiple-Choice. Results from early studies
DAVID FOSTER & HAROLD L. MILLER JR.
Abstract
The standard multiple-choice format has remained relatively unchanged for nearly 100 years, even over the past 25 years as multiple-choice tests have been computerized. We introduce a unique version of the multiple-choice format that has the potential to improve a tests measurement and security properties, along with other advantages. We summarize our research with college students on course-level exams to demonstrate these benefits and to establish the Discrete-Option Multiple-Choice (DOMC) format as not only a viable way to measure skills and content knowledge, but an essential one.
Key words: Discrete-Option Multiple-Choice; multiple choice; test security; fairness; computerized testing
David Foster, PhD
148 S. 1410 E.
Lindon, Utah 84042, USA
E-Mail: dfoster@kryteriononline.com
On designing data-sampling for Rasch model calibrating an achievement test
KLAUS D. KUBINGER, DIETER RASCH & TAKUYA YANAGIDA
Abstract
In correspondence with pertinent statistical tests, it is of practical importance to design data-sampling when the Rasch model is used for calibrating an achievement test. That is, determining the sample size according to a given type-I- and type-II-risk, and according to a certain effect of model misfit which is of practical relevance is of interest. However, pertinent Rasch model tests use chi-squared distributed test-statistics, whose degrees of freedom do not depend on the sample size or the number of testees, but only on the number of estimated parameters. We therefore suggest a new approach using an F-distributed statistic as applied within analysis of variance, where the sample size directly affects the degrees of freedom. The Rasch models quality of specific objective measurement is in accordance with no interaction effect in a specific analysis of variance design. In analogy to Andersens approach in his Likelihood-Ratio test, the testees must be divided into at least two groups according to some criterion suspected of causing differential item functioning (DIF). Then a three-way analysis of variance design (A>B)xC with mixed classification is the result: There is a (fixed) group factor A, a (random) factor B of testees within A, and a (fixed) factor C of items cross-classified with A>B; obviously the factor B is nested within A. Yet the data are dichotomous (a testee either solves an item or fails to solve it) and only one observation per cell exists. The latter is not assumed to do harm, though the design is a mixed classification. But the former suggests the need to perform a simulation study in order to test whether the type-I-risk holds for the AxC interaction F-test - this interaction effect corresponds to Rasch models specific objectivity. If so, the critical number of testees is of interest for fulfilling the pertinent precision parameters. The simulation study (100 000 runs for each of several special cases) proved that the nominal type-I-risk holds as long as there is no significant group effect. Analysing a certain DIF, this F-test has fair power, consistently higher than Andersens test. Hence, we advise researchers to apply our approach as long as there is no significant group effect, and only to use other Rasch model tests if it is significant. Keep in mind that this is true only for some special cases and needs to be generalized in further research. Then a formula needs to be provided which will allow explicit calculation of the number of testees, given a type-I-, a type-II-risk, and a relevant effect as concerns Rasch model misfit.
Key words: Rasch model; sample size; type-I- and type-II-risk; analysis of variance; mixed model
Prof. Klaus D. Kubinger, Ph.D.
Head of the Division for Psychological Assessment and Applied Psychometrics
Faculty of Psychology
University of Vienna
Liebiggasse 5
A-1010 Vienna, Austria
E-Mail: klaus.kubinger@univie.ac.at
Examining the relations among working memory capacity, attention control, and fluid intelligence from a dual-component framework
NASH UNSWORTH, GREGORY J. SPILLERS & GENE A. BREWER
Abstract
The current study examined the relations between working memory capacity, attention control, and general fluid intelligence. Participants performed multiple measures of each construct and confirmatory factor analysis and structural equation modeling was used to analyze the data. The results suggested that attention control is an important component of the working memory and general fluid intelligence relation. Additionally, attention control accounted for unique variance in general fluid intelligence above and beyond working memory capacity. Consistent with the dual-component model of working memory, substantial independent variance was left over after accounting for attention controls role in this relation. Therefore, other important mediating variables need to be accounted for to fully appreciate working memorys ability to predict general fluid intelligence (e.g., retrieval from secondary memory).
Key words: working memory; attention control; fluid intelligence
Nash Unsworth, PhD
Department of Psychology
University of Georgia
Athens, GA 30602-3013, USA
E-Mail: nunswor@uga.edu
Do Measures of working memory predict academic proficiency better than measures of intelligence?
KERRY LEE, MADELINE LEE PE, SU YIN ANG & LAZAR STANKOV
Abstract
It is often asserted that working memory predicts more variance in academic proficiency than do measures of intelligence. We used data from three studies to show that the validity of this assertion is highly dependent on the method of analysis. Using the same measures of intelligence, but different measures of working memory and algebraic proficiency, we found working memory provided better explanatory power only when analysis was conducted on the observed variable level. When the same data were analysed using structural equation models, only measures of intelligence had a direct effect on algebraic proficiency. From a theoretical viewpoint, our findings are consistent with a claim that working memory is a constituent component of (fluid) intelligence.
Key words: short term memory; intelligence measures; quantitative methods; statistical regression
Kerry Lee, PhD
National Institute of Education
1 Nanyang Walk
Singapore 637616
E-Mail: kerry.lee@nie.edu.sg
An investigation of the effect of retest practice on the relationship between speed and ability in attention, memory and working memory tasks
SYBILLE ROCKSTROH & KARL SCHWEIZER
Abstract
The research work reported in this paper addresses the question whether retest practice changes the speed-ability relationship. In three successive test sessions several cognitive tests of attention, memory and working memory were administered: Attention Switching Task, Continuous Attention Task, Memory Scanning Task, Letter Comparison Task, and Maintenance Summation Task. Retest practice led to substantial decreases of mean reaction time in the memory and working memory tasks while in attention tasks the effect was small or even insignificant. Intercepts, slopes and components representing sources with a constant, increasing and decreasing influence on responding were computed and correlated with ability. The results revealed that the correlation of the Letter Comparison task was large independently of retest practice. In contrast, for the Maintenance Summation Task the highest correlation was observed for the component representing the source with an increasing influence. Some retest practice seems to be necessary in order to establish the relationship of working memory and ability.
Key words: Retest practice; attention; working memory; reaction time; ability; trend analysis
Karl Schweizer, PhD
Department of Psychology
Goethe University Frankfurt
Mertonstr. 17
60054 Frankfurt a.M., Germany
E-Mail: k.schweizer@psych.uni-frankfurt.de
The relationship between attentional blink and psychometric intelligence: A fixed-links model approach
STEFAN J. TROCHE, KARL SCHWEIZER & THOMAS H. RAMMSAYER
Abstract
Attentional blink (AB) denotes the impairment in identifying a target when this target follows a preceding target after about 150 to 500 msec. Several models explain AB and some studies suggest that more processes than only one are involved in AB. Therefore, confounding effects of these underlying processes might be the reason why previous studies could not observe a relationship between AB and psychometric intelligence. In the present study, fixed-links models were used to disentangle the processes underlying the performance of 66 female and 52 male volunteers on an AB task. In accordance with theoretical explanations of AB, three latent variables with loadings describing a linearly increasing, a linearly decreasing and a u-shaped trend described the data well. Psychometric intelligence was related to the latent variables reflected by the u-shaped (β = .30; p < .05) and the linearly increasing trends (β = .23; p < .05) but not to the latent variable reflected by the linearly decreasing trend (β = .10; n.s.). These results support the assumptions that more processes than only one are involved in AB. Decomposition of the underlying processes seems to be promising to investigate intelligence-related individual differences on this early level of information processing.
Key words: attentional blink; intelligence; fixed-links models
Stefan Troche, PhD
Department of Psychology
University of Bern
Muesmattstr. 45
CH-3000 Bern 9, Switzerland
E-Mail: stefan.troche@psy.unibe.ch
An experimental-differential investigation of cognitive complexity
DAMIAN P. BIRNEY & DAVID B. BOWMAN
Abstract
Cognitive complexity as defined by differential and experimental traditions was explored to investigate the theoretical advantage and utility of relational complexity (RC) theory as a common framework for studying fluid cognitive functions. RC theory provides a domain general account of processing demand as a function of task complexity. In total, 142 participants completed two tasks in which RC was manipulated, and two tasks entailing manipulations of complexity derived from the differential psychology literature. A series of analyses indicated that, as expected, task manipulations influenced item difficulty. However, comparable changes in a psychometric index of complexity were not consistently observed. Active maintenance of information across multiple steps of the problem solving process, which entails strategic coordination of storage and processing that cannot be modelled under the RC framework was found to be an important component of cognitive complexity.
Key words: Complexity; relational complexity; fluid intelligence
Damian P. Birney, PhD
AGSM Building (G27)
Accelerated Learning Laboratory
University of New South Wales
NSW, Australia, 2052
E-Mail: d.birney@unsw.edu.au
A process-based account of the speed-ability relationship for the Posner Task
MICHAEL ALTMEYER, MICHAEL SCHREINER & KARL SCHWEIZER
Abstract
A process-based approach is selected for studying the relationship between intelligence and the speed of information processing on the basis of the Posner Task. By applying meta-analytic methods, several studies involving this letter-matching task were used for gaining a large data set. Within the Posner Task, participants had to react to stimuli according to two different instructions. The first instruction required mainly perceptual processes, whereas the second involved mostly memory processes. Using a model with fixed links between manifest and latent variables, and a step-wise procedure to determine parameters, enabled us to study the influences these processes have on intelligence individually. The results of our study show a much stronger influence of memory processes than of perceptual processes. Also our study illustrates the advantages a fixed-links model can have in studying separate influences of processes, as opposed to a standard structural equation model.
Key words: Posner Task; fixed-links model; elementary cognitive tasks; intelligence
Michael Altmeyer, PhD
Department of Psychology
Goethe University
Mertonstraße 16
60325 Frankfurt, Germany
E-Mail: m.altmeyer@psych.uni-frankfurt.de
