Article metrics

  • citations in SCindeks: [1]
  • citations in CrossRef:0
  • citations in Google Scholar:[=>]
  • visits in previous 30 days:7
  • full-text downloads in 30 days:1
article: 3 from 6  
Back back to result list
2018, vol. 51, iss. 3, pp. 351-375
article language: English
document type: Original Paper
published on: 27/09/2018
doi: 10.2298/PSI161031003Z
Creative Commons License 4.0
Multiple solutions test: Part I: Development and psychometric evaluation
aUniversity of Belgrade, Faculty of Philosophy, Department of Psychology
bUniversity of Belgrade, Institute for Medical Research



Identification, measurement and development of the cognitive and emotional competences important for a Europe-oriented society (MESTD - 179018)
Noninvasive modulation of cortical excitability and plasticity - Noninvasive neuromodulation of the CNS in the study of physiological mechanisms, diagnosis and treatment (MESTD - 175012)


As people outside the context of testing seldom find themselves in situations where they are presented with limited options and a single correct answer, with all others being equally wrong, a modification of traditional intelligence tests (in terms of increasing its flexibility), can potentially provide a more comprehensive and a more valid measure of intelligence. Therefore, the aim of this study is the development and psychometric evaluation of the figural reasoning test in the form of matrices with multiple solutions. Unlike conventional intelligence tests, in this test the subjects are faced with more than one task, i.e., to detect: 1) the best solution - a figure that completes a given matrix best; 2) the second-best solution - a figure that would complete the matrix in the best way if the best answer was absent; 3) the least accurate option - a figure that completes the given matrix in the least accurate way. In the process of test development, an initial set of 80 items was designed and administrated to a sample of 41 participants, with the goal of gaining insight into the quality and the need for adjustments of the initial item pool. Psychometric characteristics of the instrument consisting of 74 items with three types of tasks have been evaluated on a sample of 263 participants, after which the short version of the instrument is proposed. All three tasks within the test and test as a whole have shown good internal psychometric properties (α the best = .92, α the second-best = .90, α the least accurate = .87; α full-scale = .95) offering a possibility of reliable measurement of intelligence with a brother scope.



Ackerman, P.L., Beier, M.E., Boyle, M.O. (2005) Working memory and intelligence: The same or different constructs?. Psychological Bulletin, 131(1): 30-60
Ackerman, P.L., Heggestad, E.D. (1997) Intelligence, personality, and interests: Evidence for overlapping traits. Psychological Bulletin, 121(2): 219-245
Babcock, R.L. (1994) Analysis of adult age differences on the Raven's Advanced Progressive Matrices Test. Psychology and Aging, 9(2): 303-314
Bujas, Z. (1966) Modifikacija Ravenovih Progresivnih Matrica. Zagreb: Odsjek za psihologiju
Bujas, Z., Bartolović, B., Vodanović, M. (1967) Test višestrukih rješenja. Zagreb: Odsjek za psihologiju
Carpenter, P.A., Just, M.A., Shell, P. (1990) What one intelligence test measures: A theoretical account of the processing in the Raven Progressive Matrices Test. Psychological Review, 97(3): 404-431
Carroll, J.B. (1993) Human cognitive abilities: A survey of factor-analytic studies. Cambridge, itd: Cambridge University Press / CUP
Carroll, J.B. (1997) The three-stratum theory of cognitive abilities. in: Flanagan D.P., J.L.Genshaft, P.L.Harrison [ed.] Contemporary intellectual assessment. Theories, tests, and issues, New York: The Guilford Press
Carroll, J.B. (2005) The three-stratum theory of cognitive abilities. in: Flanagan P.D., Harrison P.L. [ed.] Contemporary intellectual assessment: Theories, tests and issues, New York: Guilford, pp. 69-76; 2nd edition
Cattell, R.B. (1987) Intelligence: Its structure, growth and action. New York: North-Holland
Ceci, S.J. (1990) On intelligence... more or less: A bioecological treatise on intellectual development. Englewood Cliffs: Prentice Hall
Chuderski, A., Taraday, M., Nęcka, E., Smoleń, T. (2012) Storage capacity explains fluid intelligence but executive control does not. Intelligence, 40(3): 278-295
Colom, R. (2004) Working memory is (almost) perfectly predicted by g. Intelligence, 32(3): 277-296
Colom, R., Abad, F.J., Rebollo, I., Chun, S.P. (2005) Memory span and general intelligence: A latent-variable approach. Intelligence, 33(6): 623-642
Conway, A.R., Cowan, N., Bunting, M.F., Therriault, D.J., Minkoff, S.R. (2002) A latent variable analysis of working memory capacity, short-term memory capacity, processing speed, and general fluid intelligence. Intelligence, 30(2): 163-183
Denis, P.L., Gilbert, F. (2012) The effect of time constraints and personality facets on general cognitive ability (GCA) assessment. Personality and Individual Differences, 52(4): 541-545
Diamond, A. (2013) Executive Functions. Annual Review of Psychology, 64(1): 135-168
Domino, G., Domino, M.L. (2009) Psychological Testing. Cambridge: Cambridge University Press (CUP)
Dragićević, Č., Momirović, K. (1969) Standardizacije NSI i RBM na teritoriji SR Srbije [Standardization of NSI and RBM in Serbia]. Beograd: Institut za psihologiju
Embretson, S.E., Reise, S.P. (2000) Item response theory for psychologists. Hillsdale, NJ, itd: Erlbaum
Engle, R.W., Tuholski, S.W., Laughlin, J.E., Conway, A.R. (1999) Working memory, short-term memory, and general fluid intelligence: A latent-variable approach. J Exp Psychol Gen, 128(3): 309-31
Friedman, N.P., Miyake, A., Corley, R.P., Young, S.E., Defries, J.C., Hewitt, J.K. (2006) Not all executive functions are related to intelligence. Psychological Science, (17): 172-179
Gardner, H. (1993) Frames of mind: The Theory of multiple intelligences. New York: Basic Books
Gignac, G.E. (2015) Raven's is not a pure measure of general intelligence: Implications for g factor theory and the brief measurement of g. Intelligence, 52: 71-79
Gottfredson, L.S. (1997) Why g matters: The complexity of everyday life. Intelligence, 24(1): 79-132
Gustafsson, J.E. (1984) A unifying model for the structure of intellectual abilities. Intelligence, vol. 8, str. 179-203
Gustafsson, J.E. (1988) Hierarchical models of individual differences in cognitive abilities. in: Sternberg R.J. [ed.] Advances in the psychology of human intelligence, Hillsdale, New Jersey: Lawrence Erlbaum Associates, Inc, Vol. 4, (pp. 35-71)
Gustafsson, J.E. (2002) Measurement from a hierarchical point of view. in: Braun H.I., Jackson D.N., Wiley D.E. [ed.] The role of constructs in psychological and educational measurement, London: Lawrence Erlbaum Associates Publishers, pp. 73-95
Gustafsson, J. (1994) Hierarchical Models of Intelligence and Educational Achievement. in: Intelligence, mind and reasoning: Structure and development, Elsevier BV, str. 45-73
Horn, J. (1988) Thinking about Human Abilities. in: Nesselroade, John R.; Cattell, Raymond B. [ed.] Handbook of Multivariate Experimental Psychology, Boston, MA: Springer Nature America, Inc, str. 645-685
Horn, J.L., Cattell, R.B. (1966) Refinement and test of the theory of fluid and crystallized general intelligences. Journal of Educational Psychology, 57(5): 253-270
Horn, J.L. (1979) The rise and fall of human abilities. Journal of Research and Development in Education, vol. 12, str. 59-78
Jensen, A.R. (1982) Reaction time and psychometric G. in: Eysenck Hans J. [ed.] A model for inteligence, Berlin, itd: Springer Verlag
Jensen, A.R. (1998) The g factor: The science of mental ability. Connecticut: Praeger Publishers
Kane, M.J., Hambrick, D.Z., Conway, A.R.A. (2005) Working Memory Capacity and Fluid Intelligence Are Strongly Related Constructs: Comment on Ackerman, Beier, and Boyle (2005). Psychological Bulletin, 131(1): 66-71
Kane, M.J., Hambrick, D.Z., Tuholski, S.W., Wilhelm, O., Payne, T.W., Engle, R.W. (2004) The generality of working memory capacity: A latent-variable approach to verbal and visuospatial memory span and reasoning. J Exp Psychol Gen, 133(2): 189-217
Knežević, G.D., Momirović, K. (1996) RTT9G i RTT10G - dva programa za analizu metrijskih karakteristika kompozitnih mernih instrumenata. in: Kostić Petar [ed.] Merenje u psihologiji, Beograd: Institut za kriminološka i sociološka istraživanja, br. 2, str. 37-56
Kuncel, N.R., Hezlett, S.A., Ones, D.S. (2004) Academic Performance, Career Potential, Creativity, and Job Performance: Can One Construct Predict Them All?. Journal of Personality and Social Psychology, 86(1): 148-161
Kyllonen, P.C., Christal, R.E. (1990) Reasoning abillity is (little more than) working memory capacity. Intelligence, vol. 14, str. 389-433
Lu, Y., Sireci, S.G. (2007) Validity Issues in Test Speededness. Educational Measurement: Issues and Practice, 26(4): 29-37
Mackintosh, N.J. (1998) IQ and Human Intelligence. UK: Oxford University Press
Martínez, K., Burgaleta, M., Román, F.J., Escorial, S., Shih, P.C., Quiroga, M., Colom, R. (2011) Can fluid intelligence be reduced to 'simple’ short-term storage?. Intelligence, 39(6): 473-480
McGrew, K.S. (2009) CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research. Intelligence, 37(1): 1-10
McGrew, K.S., Wendling, B.J. (2010) Cattell-Horn-Carroll cognitive-achievement relations: What we have learned from the past 20 years of research. Psychology in the Schools, str. n/a-n/a
Miyake, A., Friedman, N.P., Emerson, M.J., Witzki, A.H., Howerter, A., Wager, T.D. (2000) The unity and diversity of executive functions and their contributions to complex 'Frontal Lobe' tasks: a latent variable analysis. Cognitive psychology, 41(1): 49-100
Miyake, A., Friedman, N.P. (2012) The Nature and Organization of Individual Differences in Executive Functions. Current Directions in Psychological Science, 21(1): 8-14
Oberauer, K., Süβ, H., Wilhelm, O., Wittmann, W.W. (2008) Which working memory functions predict intelligence?. Intelligence, 36(6): 641-652
Osteen, P. (2010) An Introduction to Using Multidimensional Item Response Theory to Assess Latent Factor Structures. Journal of the Society for Social Work and Research, 1(2): 66-82
Penrose, L.S., Raven, J.C. (2011) A new series of perceptual tests: preliminary communication. British Journal of Medical Psychology, 16(2): 97-104
Raven, J. (2000) The Raven's progressive matrices: Change and stability over culture and time. Cognit Psychol, 41(1): 1-48
Raven, J., Raven, J.C., Court, J.H. (1998) Raven manual: Section 4. Advanced progressive matrices. Oxford: Oxford Psychologist Press
Raven, J., Raven, J.C., Court, J.H. (2000) Raven manual: Section 3. Standard progressive matrices. Oxford: Oxford Psychologist Press
Raven, J.C., Court, J.H. (1998) Raven's progressive matrices and vocabulary scales. Oxford Psychologists Press
Raven, J.C. (1938) Progressive matrices: A perceptual test of intelligence. London: H.K. Lewis
Salgado, J.F., Anderson, N., Moscoso, S., Bertua, C., de Fruyt, F., Rolland, J.P. (2003) A Meta-Analytic Study of General Mental Ability Validity for Different Occupations in the European Community. Journal of Applied Psychology, 88(6): 1068-1081
Sattler, J.M., Ryan, J.J. (2009) Assessment with the WAIS-IV. San Diego, CA: Jerome M. Sattler, Publisher
Schmidt, F.L., Hunter, J.E. (1998) The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2): 262-274
Snow, R.E., Kyllonan.P.C., Marshalek, B. (1984) The topography of ability and learning correlations. in: Sternberg R.J. [ed.] Advances in the psychology of human intelligence, Hillsdale, NJ, itd: Erlbaum, vol. 2, str. 47-103
Spearman, C. (2011) Theory of general factor. British Journal of Psychology. General Section, 36(3): 117-131
Spearman, C.E. (1927) The abilities of man: Their nature and measurement. London-New York, itd: Macmillan Publishing
Spearman, C.E. (1904) General intelligence, objectively determined and measured. American Journal of Psychology, vol. 15, str. 201-293
Sternberg, R.J. (1985) Beyond IQ: A triarchic theory of human intelligence. Cambridge, itd: Cambridge University Press
Sternberg, R.J., Wagner, R.K. (1986) Practical Intelligence: Nature and Origins of competence in the everyday world. New York: Cambridge University Press
Sternberg, R.J. (1999) The theory of successful intelligence. Review of General Psychology, 3(4): 292-316
Sternberg, R.J., Wagner, R.K. (1993) The g -ocentric View of Intelligence and Job Performance Is Wrong. Current Directions in Psychological Science, 2(1): 1-4
Süß, H., Beauducel, A. (2015) Modeling the construct validity of the Berlin Intelligence Structure Model. Estudos de Psicologia (Campinas), 32(1): 13-25
Vernon, P.E., Parry, J.B. (1949) Personnel selection in the British Forces. London: University of London Press
Wechsler, D. (2008) Wechsler adult intelligence scale: Technical and interpretive manual. San Antonio, TX: Pearson Assessment, Fourth Edition
Živanović, M., Bjekić, J., Opačić, G. (2018) Multiple solutions test: Part II: Evidence on construct and predictive validity. Psihologija, vol. 51, br. 3, str. 377-396