Article metrics

  • citations in SCindeks: [1]
  • citations in CrossRef:0
  • citations in Google Scholar:[=>]
  • visits in previous 30 days:10
  • full-text downloads in 30 days:1
article: 4 from 6  
Back back to result list
2018, vol. 51, iss. 3, pp. 377-396
article language: English
document type: Original Paper
published on: 27/09/2018
doi: 10.2298/PSI170205004Z
Creative Commons License 4.0
Multiple solutions test: Part II: Evidence on construct and predictive validity
aUniversity of Belgrade, Faculty of Philosophy, Department of Psychology
bUniversity of Belgrade, Institute for Medical Research



Identification, measurement and development of the cognitive and emotional competences important for a Europe-oriented society (MESTD - 179018)
Noninvasive modulation of cortical excitability and plasticity - Noninvasive neuromodulation of the CNS in the study of physiological mechanisms, diagnosis and treatment (MESTD - 175012)


In this study, we complement data on Multiple solutions test (MST) by examining its construct and predictive validity. Unlike conventional matrices where a single solution is required, MST sets three types of problems before the participants, requiring them to solve matrices for the best, the second-best, and the least accurate solution. A total sample of 235 individuals (age M = 22.65, SD = 3.33, 199 females) participated in the study. Construct validity of each task within MST was tested in relation to the KOG9 battery of intellectual abilities (N = 156), while predictive value of individual tasks and full-scale performance was tested in relation to scholastic achievement measured by GPA (N = 235). The results have shown high between-task correlations, but also pointed to the specificities of each of them. Additionally, differential difficulties among the tasks were found with the least accurate task being the most difficult, followed by the second-best, and the best one. The test showed satisfactory convergent validity in relation to Gf/Gv test markers within KOG9 battery. Furthermore, MST has shown predictive validity, along with the incremental value of alternative tasks above the standard one (when the scholastic achievement was taken as a criterion), as well as incremental validity in predicting GPA above the KOG9 battery. In general, MST has shown to be a valid instrument for an intelligence assessment, and its alternative tasks have a potential to be a useful addition to standard matrices with one type of solution.



Ackerman, P.L., Beier, M.E., Boyle, M.O. (2005) Working memory and intelligence: The same or different constructs?. Psychological Bulletin, 131(1): 30-60
Allix, N.M. (2000) The Theory of Multiple Intelligences: A Case of Missing Cognitive Matter. Australian Journal of Education, 44(3): 272-288
Bele-Potočnik, Ž. (1983) Test Domino 'D48' - priručnik. Ljubljana: Zavod SR Slovenije za produktivnost dela - Centar za psihodijagnostička sredstva
Brody, N. (2003) What Sternberg should have concluded. Intelligence, 31(4): 339-342
Bujas, Z., Bartolović, B., Vodanović, M. (1967) Test višestrukih rješenja. Zagreb: Odsjek za psihologiju
Bujas, Z. (1966) Modifikacija Ravenovih Progresivnih Matrica. Zagreb: Odsjek za psihologiju
Cabrera, M.A., Nguyen, N.T. (2001) Situational Judgment Tests: A Review of Practice and Constructs Assessed. International Journal of Selection and Assessment, 9(1&2): 103-113
Carpenter, P.A., Just, M.A., Shell, P. (1990) What one intelligence test measures: A theoretical account of the processing in the Raven Progressive Matrices Test. Psychological Review, 97(3): 404-431
Carroll, J.B. (1997) The three-stratum theory of cognitive abilities. in: Flanagan D.P., J.L.Genshaft, P.L.Harrison [ed.] Contemporary intellectual assessment. Theories, tests, and issues, New York: The Guilford Press
Carroll, J.B. (1993) Human cognitive abilities: A survey of factor-analytic studies. Cambridge, itd: Cambridge University Press / CUP
Carroll, J.B. (2005) The three-stratum theory of cognitive abilities. in: Flanagan P.D., Harrison P.L. [ed.] Contemporary intellectual assessment: Theories, tests and issues, New York: Guilford, pp. 69-76; 2nd edition
Ceci, S.J. (1990) On intelligence... more or less: A bioecological treatise on intellectual development. Englewood Cliffs: Prentice Hall
Chuderski, A., Taraday, M., Nęcka, E., Smoleń, T. (2012) Storage capacity explains fluid intelligence but executive control does not. Intelligence, 40(3): 278-295
Colom, R. (2004) Working memory is (almost) perfectly predicted by g. Intelligence, 32(3): 277-296
Colom, R., Abad, F.J., Rebollo, I., Chun, S.P. (2005) Memory span and general intelligence: A latent-variable approach. Intelligence, 33(6): 623-642
Conway, A.R., Cowan, N., Bunting, M.F., Therriault, D.J., Minkoff, S.R. (2002) A latent variable analysis of working memory capacity, short-term memory capacity, processing speed, and general fluid intelligence. Intelligence, 30(2): 163-183
Diamond, A. (2013) Executive Functions. Annual Review of Psychology, 64(1): 135-168
Domino, G. (2001) The D-48 application in Mexican-American children of a culture fair test. School Psychology International, 22(3): 253
Engle, R.W., Tuholski, S.W., Laughlin, J.E., Conway, A.R. (1999) Working memory, short-term memory, and general fluid intelligence: A latent-variable approach. J Exp Psychol Gen, 128(3): 309-31
Frensch, P.A., Funke, J. (1995) Complex problem solving: The European perspective. Hillsdale-London, itd: Lawrence Erlbaum Associates
Friedman, N.P., Miyake, A., Corley, R.P., Young, S.E., Defries, J.C., Hewitt, J.K. (2006) Not all executive functions are related to intelligence. Psychological Science, (17): 172-179
Funke, J. (2010) Complex problem solving: a case for complex cognition?. Cognitive Processing, 11(2): 133-142
Gardner, H. (1993) Frames of mind: The Theory of multiple intelligences. New York: Basic Books
Gonzalez, C., Vanyukov, P., Martin, M.K. (2005) The use of microworlds to study dynamic decision making. Computers in Human Behavior, 21(2): 273-286
Gottfredson, L.S. (1997) Why g matters: The complexity of everyday life. Intelligence, 24(1): 79-132
Gottfredson, L.S. (2003) Dissecting practical intelligence theory. Intelligence, 31(4): 343-397
Gustafsson, J.E., Undheim, J.O. (1996) Individual differences in cognitive functions. in: Berliner D.C., Calfee R.C. [ed.] Handbook of educational psychology, New York: Prentice Hall International, pp. 186-242
Horn, J.L. (1979) The rise and fall of human abilities. Journal of Research and Development in Education, vol. 12, str. 59-78
Jensen, A.R. (1998) The g factor: The science of mental ability. Connecticut: Praeger Publishers
Kane, M.J., Hambrick, D.Z., Conway, A.R.A. (2005) Working Memory Capacity and Fluid Intelligence Are Strongly Related Constructs: Comment on Ackerman, Beier, and Boyle (2005). Psychological Bulletin, 131(1): 66-71
Kane, M.J., Hambrick, D.Z., Tuholski, S.W., Wilhelm, O., Payne, T.W., Engle, R.W. (2004) The generality of working memory capacity: A latent-variable approach to verbal and visuospatial memory span and reasoning. J Exp Psychol Gen, 133(2): 189-217
Kretzschmar, A., Neubert, J.C., Wüstenberg, S., Greiff, S. (2016) Construct validity of complex problem solving: A comprehensive view on different facets of intelligence and school grades. Intelligence, 54: 55-69
Kuncel, N.R., Hezlett, S.A., Ones, D.S. (2004) Academic Performance, Career Potential, Creativity, and Job Performance: Can One Construct Predict Them All?. Journal of Personality and Social Psychology, 86(1): 148-161
Kyllonen, P.C., Christal, R.E. (1990) Reasoning abillity is (little more than) working memory capacity. Intelligence, vol. 14, str. 389-433
Lazarević, L.B., Knežević, G. (2008) Factor structure of intelligence test battery KOG9. Psihologija, vol. 41, br. 4, str. 489-505
Martínez, K., Burgaleta, M., Román, F.J., Escorial, S., Shih, P.C., Quiroga, M., Colom, R. (2011) Can fluid intelligence be reduced to 'simple’ short-term storage?. Intelligence, 39(6): 473-480
Mcdaniel, M.A., Whetzel, D.L. (2003) The emperor's new clothes: Additional critiques of sternberg's practical intelligence theory. in: Fourth Annual Conference of the International Society for Intelligence Research, Newport Beach, CA
McGrew, K.S. (2009) CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research. Intelligence, 37(1): 1-10
McGrew, K.S., Wendling, B.J. (2010) Cattell-Horn-Carroll cognitive-achievement relations: What we have learned from the past 20 years of research. Psychology in the Schools, str. n/a-n/a
Miyake, A., Friedman, N.P., Emerson, M.J., Witzki, A.H., Howerter, A., Wager, T.D. (2000) The unity and diversity of executive functions and their contributions to complex 'Frontal Lobe' tasks: a latent variable analysis. Cognitive psychology, 41(1): 49-100
Miyake, A., Friedman, N.P. (2012) The Nature and Organization of Individual Differences in Executive Functions. Current Directions in Psychological Science, 21(1): 8-14
Momirović, K., Bosnar, K., Horga, S. (1982) Kibernetički model kognitivnog funkcionisanja: Pokušaj sinteze nekih teorija o strukturi kognitivnih sposobnosti. Kineziologija, 63-82; 14
Oberauer, K., Süβ, H., Wilhelm, O., Wittmann, W.W. (2008) Which working memory functions predict intelligence?. Intelligence, 36(6): 641-652
Rohde, T.E., Thompson, L.A. (2007) Predicting academic achievement with cognitive ability. Intelligence, 35(1): 83-92
Salgado, J.F., Anderson, N., Moscoso, S., Bertua, C., de Fruyt, F., Rolland, J.P. (2003) A Meta-Analytic Study of General Mental Ability Validity for Different Occupations in the European Community. Journal of Applied Psychology, 88(6): 1068-1081
Schmidt, F.L., Hunter, J.E. (1998) The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2): 262-274
Snow, R.E., Kyllonan.P.C., Marshalek, B. (1984) The topography of ability and learning correlations. in: Sternberg R.J. [ed.] Advances in the psychology of human intelligence, Hillsdale, NJ, itd: Erlbaum, vol. 2, str. 47-103
Soloman, S.R., Sawilowsky, S.S. (2009) Impact of Rank-Based Normalizing Transformations on the Accuracy of Test Scores. Journal of Modern Applied Statistical Methods, 8(2): 448-462
Spearman, C. (2011) Theory of general factor. British Journal of Psychology. General Section, 36(3): 117-131
Sternberg, R.J., Forsythe, G.B., Hedlund, J., Horvath, J.A., Wagner, R.K., Williams, W.M., Snook, S.A., Grigorenko, E.L. (2000) Practical intelligence in everyday life. New York: Cambridge University Press
Sternberg, R.J., Wagner, R.K., ur. (1986) Practical Intelligence: Nature and origins of competence in the everyday world. New York: Cambridge University Press
Sternberg, R.J. (1985) Beyond IQ: A triarchic theory of human intelligence. Cambridge, itd: Cambridge University Press
Sternberg, R.J. (2003) Our research program validating the triarchic theory of successful intelligence: reply to Gottfredson. Intelligence, 31(4): 399-413
Sternberg, R.J., Wagner, R.K. (1993) The g -ocentric View of Intelligence and Job Performance Is Wrong. Current Directions in Psychological Science, 2(1): 1-4
Sternberg, R.J. (1999) The theory of successful intelligence. Review of General Psychology, 3(4): 292-316
Vernon, P.E., Parry, J.B. (1949) Personnel selection in the British Forces. London: University of London Press
Waterhouse, L. (2006) Multiple Intelligences, the Mozart Effect, and Emotional Intelligence: A Critical Review. Educational Psychologist, 41(4): 207-225
Wolf, B., Momirović, K., Džamonja, Z. (1992) Baterija testova inteligencije KOG3. Beograd: Centar za primenjenu psihologiju
Živanović, M., Bjekić, J., Opačić, G. (2018) Multiple solutions test: Part I: Development and psychometric evaluation. Psihologija, vol. 51, br. 3, str. 351-375