Assessing accuracy for multi-class classification when subclasses are involved

Nan, Nan
Tian, Lili

¹Department of Biostatistics, University at Buffalo, Buffalo, NY, USA.

Corresponding author: Lili Tian, Department of Biostatistics, 717 Kimball Tower, 3435 Main Street, Buffalo, NY 14214, USA. Email: [email protected]

Statistical Methods in Medical Research 34(7):p 1480-1503, July 2025. | DOI: 10.1177/09622802251343600

Abstract

Classifications that involve subclasses are common in many applied fields. “Compound multi-class classification” refers to the settings which involve three or more main classes and at least one of the main classes has multiple subclasses. In this paper, we propose an accuracy metric proper for “compound M-class classification,” namely “hypervolume under compound Symbol manifold Symbol.” The proposed HUM_C,M evaluates the overall accuracy of a biomarker measured on continuous scale correctly identifying M main classes without requiring specification of an ordering in terms of marker values for subclasses relative to each other within each main class. The probabilistic interpretation of HUM_C,M is analytically derived. A network-based computing algorithm which enables efficient computation of the empirical estimate of HUM_C,M is developed. Non-parametric bootstrap percentile confidence intervals of HUM_C,M are assessed through extensive simulation studies. Lastly, a real data example is included to illustrate the usage of our proposed method.