Multiple solutions test: Part I: Development and psychometric evaluation

Abstract

As people outside the context of testing seldom find themselves in situations where they are presented with limited options and a single correct answer, with all others being equally wrong, a modification of traditional intelligence tests (in terms of increasing its flexibility), can potentially provide a more comprehensive and a more valid measure of intelligence. Therefore, the aim of this study is the development and psychometric evaluation of the figural reasoning test in the form of matrices with multiple solutions. Unlike conventional intelligence tests, in this test the subjects are faced with more than one task, i.e., to detect: 1) the best solution - a figure that completes a given matrix best; 2) the second-best solution - a figure that would complete the matrix in the best way if the best answer was absent; 3) the least accurate option - a figure that completes the given matrix in the least accurate way. In the process of test development, an initial set of 80 items was designed and administrated to a sample of 41 participants, with the goal of gaining insight into the quality and the need for adjustments of the initial item pool. Psychometric characteristics of the instrument consisting of 74 items with three types of tasks have been evaluated on a sample of 263 participants, after which the short version of the instrument is proposed. All three tasks within the test and test as a whole have shown good internal psychometric properties (α the best = .92, α the second-best = .90, α the least accurate = .87; α full-scale = .95) offering a possibility of reliable measurement of intelligence with a brother scope.

Kako se ljudi van testovnog konteksta retko nalaze u situacijama gde su suočeni sa ograničenim opcijama i samo jednim tačnim odgovorom, pri čemu su svi ostali odgovori podjednako pogrešni, izmena tradicionalnih testova inteligencije (u smislu povećanja fleksibilnosti) može potencijalno da obezbedi obuhvatniju i validniju meru inteligencije. Stoga, cilj ove studije je razvoj i psihometrijska evaluacija testa figuralnog rezonovanja u formi matrica sa višestrukim rešenjima. Za razliku od konvencionalnih testova inteligencije, u ovom testu ispitanici su suočeni sa više od jednog zadatka, tj. oni treba da otkriju: 1) najbolje rešenje - figuru koja najbolje kompletira datu matricu; 2) drugo najbolje rešenje - figuru koja bi najbolje upotpunila matricu ukoliko ne bi bilo prvog, najboljeg odgovora; 3) najmanje tačnu opciju - figuru koja upotpunjava datu matricu na najmanje tačan način. U procesu razvoja testa, konstruisan je početni skup od 80 stavki i zadat uzorku od 41 ispitanika, sa ciljem sticanja uvida u kvalitet i potrebu za prilagođavanjem početnog skupa stavki. Psihometrijske karakteristike instrumenta koji se sastoji od 74 stavke sa tri tipa zadataka proverene su na uzorku od 263 ispitanika, nakon čega je predložena kratka verzija instrumenta. Sva tri zadatka unutar testa, kao i test u celini pokazali su dobra interna psihometrijska svojstva (α najbolje rešenje = .92, α drugo najbolje rešenje = .90, α najmanje tačno rešenje = .87; α ukupni skor = .95) nudeći mogućnost pouzdanog i obuhvatnijeg merenja inteligencije.