Balancing Between Categorical and Dimensional Assessment in Short-Scale Construction Using Ant Colony Optimization

Achaa-Amankwaa, Priscilla
Trautwein, Tim
Lenhard, Wolfgang
Schroeders, Ulrich

Department of Psychology, University of Kassel, Germany
Department of Psychology IV, University of Würzburg, Germany

Publication Ethics Informed consent was obtained from all participants included in the study.

Authorship Priscilla Achaa-Amankwaa, Ulrich Schroeders, Tim Trautwein, Conceptualization, methodology; T. Trautwein, P. Achaa-Amankwaa, Formal analysis; Wolfgang Lenhard, Data curation; P. Achaa-Amankwaa, Writing – original draft; P. Achaa-Amankwaa., U. Schroeders, T. Trautwein, W. Lenhard, Writing – review & editing.

All authors approved the final version of the article.

Open Science Open Analytic Code: We confirm that all the scripts, code, and outputs needed to reproduce the results are provided (see OSF and GitHub repositories, Achaa-Amankwaa et al., 2025a; Achaa-Amankwaa et al., 2025b).

Open Data: We confirm that there is sufficient information for an independent researcher to reproduce all of the reported results, including a codebook if relevant, available via an OSF repository (https://osf.io/pe5sy/, Achaa-Amankwaa et al., 2025b).

Open Materials: We confirm that there is sufficient information for an independent researcher to reproduce all of the reported methodology, available via a GitHub repository (https://github.com/timtrautwein/ACO-Cat-vs.-ACO-Dim/tree/main; Achaa-Amankwaa et al., 2025b).

Preregistration and Analysis Plans: This study was not preregistered.

Funding: Open access publication enabled by University of Kassel, Germany.

Priscilla Achaa-Amankwaa Department of Psychology University of Kassel Holländische Straße 36-38 34127 Kassel Germany [email protected]

Received May 4, 2024

revised October 29, 2024

Accepted December 27, 2024

European Journal of Psychological Assessment Publish Ahead of Print, May 13, 2025. | DOI: 10.1027/1015-5759/a000892

Language proficiency assessment poses particular challenges for test developers in selecting items that allow for a clear assignment of individuals to language proficiency levels (categorical assessment), while at the same time providing a reliable and comprehensive dimensional assessment of language proficiency. We show how Ant Colony Optimization (ACO) can be used to achieve a balance between these measurement goals, using a German entry-level language assessment as a working example. We tailored competing ACO algorithms to develop short scales of different lengths that met several pre-specified criteria, including model fit, composite reliability, and criterion validity. In optimizing the short scales, we favored either accurate dimensional assessment (model fit and composite reliability), between-category classification accuracy (a high polychoric correlation between model-predicted and independently assessed proficiency levels), or a balance of both. We argue that scale optimization strategies such as ACO are essential for balancing conflicting measurement goals such as optimizing between categorical and dimensional assessment.

Distributed as a Hogrefe OpenMind article under the license CC BY 4.0 (https://creativecommons.org/licenses/by/4.0)