CONTENTS
Editorial: Promising Reasoning Test Ideas not yet published
Klaus D. Kubinger
DOI: https://doi.org/10.2440/001-0001
Conceptualization of a new “Two-way Figural Reasoning-Test”
Larissa Bartok & Klaus D. Kubinger
DOI: https://doi.org/10.2440/001-0002
Conceptualization of the Reasoning-Test “(Mathematical) Equations”
Klaus D. Kubinger & Claudia Gamsjäger
DOI: https://doi.org/10.2440/001-0003
Conceptualization of the Reasoning-Test “Numerical Topologies”
Klaus D. Kubinger & Nina Heuberger
DOI: https://doi.org/10.2440/001-0004
Conceptualization of the Reasoning-Test “Culture-referenced Pictographic Analogies”
Klaus D. Kubinger, Serra Ünal & Bernadette Schnait
DOI: https://doi.org/10.2440/001-0005
Conceptualization of the Reasoning-Test “Reality-contradicting Syllogisms”
Sarah Treiber & Klaus D. Kubinger
DOI: https://doi.org/10.2440/001-0006
Conceptualization of the “Family Relations Reasoning Test” (FRRT)
Herbert Poinstingl & Jörn R. Sparfeldt
DOI: https://doi.org/10.2440/001-0007
On the Dimensionality of Reasoning
Klaus D. Kubinger
DOI: https://doi.org/10.2440/001-0008
Editorial: Promising Reasoning Test Ideas not yet published
Klaus D. Kubinger University of Vienna
Preamble:
Over a quarter of a century ago, when I focused my research work on a renaissance of Raymond B. Cattell’s so-called Objective Personality tests (cf. Kubinger, 1997, 2006, 2009), there was not only a (German-edited) test-battery on the market (though hardly in practical use; Häcker, Schmidt, Schwenkmezger, & Utz, 1975), which offered about 50 test concepts (indicating by numeration that at least 197 such concepts were developed in the original English version); but there was also the famous compendium by Cattell and Warburton (1967), which summarized some empirical evidence of those test concepts. Although most of them were not examined according to relevant quality criteria of psychological test construction, and most notably, the test-scores were never standardized based on a representative sample, there was a seemingly endless quantum of promising test concepts at an interested researcher’s disposal. A similar situation arises for interested researchers in a research area from which I am withdrawing, now: the dimensionality of reasoning – though not with respect to the incomparable number of Cat-tell’s test concepts. It seems worthwhile to issue some conceptualized reasoning tests that have partly demonstrated their psychometric quality, particularly with respect to their empirical ad-equacy of measurement. That is, there are several tests with publication potential, which, how-ever, need further in-depth research investment. This special issue may serve as a stimulation for this purpose.
Prof. Klaus D. Kubinger, PhD, MSc. c/o University of Vienna, Faculty of Psychology. klaus.kubinger@univie.ac.at
Conceptualization of a new “Two-way Figural Reasoning-Test”
Larissa Bartok & Klaus D. Kubinger
Abstract: A concept for a so-called matrices test (a missing figure has to be deduced by reasoning, according to row- and/or column-wise logically proceeding figures in a matrix) is introduced which is new in the following ways: 1) Instead of using a typical 3 x 3 matrix design, a 5 x 5 matrix design is used, 2) in contrast to common matrices, highly redundant cells are emptied, and 3) the questioned cell is located rather at random instead of in the last row and the last column. Due to the wellknown problems of a multiple-choice response format (i.e. recognizing the solution instead of producing; the phenomenon of lucky guessing; construction of plausible distractors), a free response format is used: The testee has to draw the solution by hand, whereby the necessary drawing ability is extremely low. A first draft of the Two-way Figural Reasoning-Test essentially measures in accordance with the Rasch model. However, testing whether the given item generating rules sufficiently explain the resulting item difficulty parameters by means of the LLTM (linear logistic test model) disclosed that some up to now nonaware (cognitive) operation components interfere with the items’ difficulty. Above all, fur-ther research is needed in order to remove any not reasoning-based radicals of item difficulty – some suggestions of such radicals are given.
Keywords: matrices test, multiple-choice response format, Rasch model, item generating rules, LLTM
Mag.a Larissa Bartok, BSc, University of Vienna, Center for Teaching and Learning. larissa.bartok@univie.ac.at
Prof. Klaus D. Kubinger, PhD, MSc. c/o University of Vienna, Faculty of Psychology. klaus.kubinger@univie.ac.at
Conceptualization of the Reasoning-Test “(Mathematical) Equations”
Klaus D. Kubinger & Claudia Gamsjäger
Abstract: Due to the six categories of reasoning tests established by Kubinger (2023a) a test concept is suggested which refers to the crystallized facet, concerning numerical contents. Each item presents a mathematical equation with two unknowns and the task is to find values for both of them which fulfill the given equation. Thereby a special multiple choice answer format is used: “2 x (1 out of 4)“, that is for each unknown four options are offered. A first draft of such a test (mathematical) Equations with 44 items has been psychometrically analyzed according to the Rasch model. The result shows that five items have to be deleted in order to achieve a-posteriori model conformity. The deleted items are extensively discussed in particular with respect to the given (arrangement of the) distractors. In doing so some unintended variables could be suspected of affecting the items’ difficulties: All above different strategies of pro-cessing the task seem to cause the model misfit. Consequently, some means are proposed to prevent a testee from applying the strategy of trial and error by simply checking step by step each combination of the two times four answer options until the solution is obtained.
Keywords: Reasoning, multiple-choice response format, process strategy, distractor, Rasch model
Prof. Klaus D. Kubinger, PhD, MSc. c/o University of Vienna, Faculty of Psychology. klaus.kubinger@univie.ac.at
Conceptualization of the Reasoning-Test “Culture-referenced Pictographic Analogies”
Klaus D. Kubinger, Serra Ünal & Bernadette Schnait
Abstract: In this paper a test is proposed which, following the well-known tests asking for lexical analogies, uses a non-verbal analogon with pictures of everyday objects. By this means one out of the six categories of reasoning tests as suggested by Kubinger (2023a) can be real-ized: the crystallized facet with figural contents. In two studies a first draft of such a test with 27 items has been psychometrically analyzed according to the Rasch model in two studies. Five items resulted as definitely not fitting the model. It is shown, retrospectively, that this is most likely caused by the items’ topics and contents: either they refer to highly specialized knowledge, measuring rather the level of general education than deductive thinking (reasoning) by using objects that are commonly present nowadays; or/and the items’ task does not focus on everyday objects’ common functions but is rather reduced to associating one object to another. As a consequence an operational definition is given of that ability which the items of a future test Culture-referenced Pictographic Analogies should measure.
Keywords: Reasoning, everyday knowledge, pictograph, multiple-choice response format, Rasch model
Prof. Klaus D. Kubinger, PhD, MSc. c/o University of Vienna, Faculty of Psychology. klaus.kubinger@univie.ac.at
Conceptualization of the Reasoning-Test “Reality-contradicting Syllogisms”
Sarah Treiber & Klaus D. Kubinger
Abstract: Considering so-called reasoning tests, almost only test concepts with figural item contents are practically in use – this being in Raymond B. Cattell’s tradition of aiming for cul-ture-fair tests. However, as Kubinger (2023a) recently suggested there are six categories of reasoning tests, i.e. a two times three classification of fluid vs. crystallized facets and lexical vs. numerical vs. figural contents. And especially the combination of fluid facets with lexical con-tents is hardly available to a practitioner. Though, there is at least the approach to use the for-mal-logical principle of a “syllogism”, which will be re-activated in this paper. In contrast to a lot of Srp’s items of her test (Srp, 1994) now each item is constructed in such manner that at least one premise as well as the conclusion contradict the actual facts (the material truth) or, alternatively, it has no reference to reality by using meaningless, freely invented “words” as acting terms. A first draft of such a test Reality-contradicting Syllogisms with 20 items has been psychometrically analyzed according to the Rasch model. Although only two of the items have to be deleted in order to achieve a-posteriori model conformity, no insight could be gained about which specific components in the composition of an item are problematic so that it cap-tures something different from the other items. For now, these items’ non-conformity with the Rasch model is simply be attributed to chance. But for deeper understanding the challenge of syllogisms as a psychological test’s task, further research is needed. Some hints for doing so are given in this paper.
Keywords: Reasoning, syllogism, fluid vs. crystallized facets, multiple-choice response for-mat, Rasch model
Prof. Klaus D. Kubinger, PhD, MSc. c/o University of Vienna, Faculty of Psychology. klaus.kubinger@univie.ac.at
Conceptualization of the “Family Relations Reasoning Test” (FRRT)
Herbert Poinstingl & Jörn R. Sparfeldt
Abstract: To assess the important intelligence factor verbal reasoning, a new task type was developed. After reading a short description introducing the relations of different family mem-bers, test takers needed to identify the relation between two target family members. Based on item generation rules, 60 items of the corresponding Family Relations Reasoning Test (FRRT) were constructed. In study 1, n = 225 Austrian university students worked on one of four sets consisting of 18 German items linked with bridge items. Item Response Theory- (IRT-) anal-yses showed that the Rasch model held for most items, Conditional Likelihood Ratio Tests revealed strong model validity. No Differential Item Functioning (DIF) was detected, and misfit was shown for only one item. Reliability estimates and correlations with syllogism items (up to medium effect size) were partially convincing. In study 2, 60 English items were administered to n = 113 Californian college students. IRT analyses showed mostly strong validity (only five items depicted DIF and six items revealed misfit). Reliability indicators were at least acceptable, and correlations with syllogism items were of small to medium effect size. Finally, we discuss these mostly promising results suggesting the usefulness to construct rule-based items related to family relations and to assess verbal reasoning with these items.
Keywords: Family Relations Reasoning Test, verbal reasoning, crystallized intelligence, IRT analyses
Mag. Herbert Poinstingl, Paderborn University, Faculty of Arts and Humanities
E-Mail: herbert.poinstingl@uni-paderborn.de
Prof. Dr. Jörn R. Sparfeldt, Saarland University, Department of Educational Science
E-Mail: j.sparfeldt@mx.uni-saarland.de
On the dimensionality of Reasoning
Klaus D. Kubinger
Abstract: According to the categorization of reasoning tests by Kubinger (2023) in this special issue, corresponding conceptualizations of six tests were in detail also presented there.For all of them the Rasch model’s validness could be fundamentally proven. And each of them seems potentially usefull for practical counseling. This paper now analyzes the dimensionality of the abilities being measured with these tests. The question is if each indeed measures a specific intelligence factor but is not covered by other factors or even by a general reasoning factor – both the latter cases means that at least one of the tests is redundant and therefore is not neces-sarily further to discuss. Consequently, the hypothesis is: these tests are correlated only in a practical negligible extent. Consequently, the hypothesis is: these tests are correlated only in a practical negligible extent. Because expecting a six-factor solution by applying factor analysis to six variables is hardly realistic, an analysis with multiple correlations has been tried. The question is whether the correlation coefficient between any two tests (significantly) increases if some further tests were taken into account. The pairwise correlation coefficients of eight studies were on the author’s disposal (n = 2047) which all proved to be significantly non-zero, however the largest one is (only) .430, meaning a determination coefficient of 18.5% at most. When the correlation coefficient of each test as the regressand and another test as the regressor is com-pared with the multiple correlation coefficient using additionally one or two more tests as re-gressors, this always resulted in non-significance (a = .01) except on one occasion – however even there the correlation with a coefficient of .456 is anything but impressive for practice. That is, each of the tests is neither covered by another factor nor by a general reasoning factor.
Keywords: Reasoning, crystallized vs. fluid intelligence, Jäger’s contents, dimensionality, multiple correlation
Prof. Klaus D. Kubinger, PhD, MSc. c/o University of Vienna, Faculty of Psychology. klaus.kubinger@univie.ac.at
Psychological Test and Assessment Modeling
Volume 65 · 2023 · Issue 3
Pabst, 2023
ISSN 2190-0493 (Print)
ISSN 2190-0507 (Internet)
