A novel accuracy metric for general M-class classification when subclasses are involved
Principal Investigator
Name
Nan Nan
Degrees
M.A.
Institution
State University of New York at Buffalo
Position Title
PhD student
Email
About this CDAS Project
Study
PLCO
(Learn more about this study)
Project ID
PLCO-1531
Initial CDAS Request Approval
Apr 29, 2024
Title
A novel accuracy metric for general M-class classification when subclasses are involved
Summary
In this project, we are introducing a novel accuracy metric proper for general compound M-class classification. "Compound multi-class classification'' refers to the setting where three or more main classes are involved and at least one of the main classes has multiple subclasses.
In this paper, we first propose a new accuracy measure proper for ``compound multi-class classification'' with general M main classes where M >= 3, namely ``hypervolume under compound ROC manifold HUMc".
The proposed HUMc evaluates the accuracy of a biomarker appropriately by identifying main classes without requiring the specification of an ordering for marker values of subclasses within each main class.
For confidence interval estimation of ${VUS_C}$, an efficient computing algorithm estimating its empirical estimator is discussed alongside the non-parametric bootstrap confidence interval, and simulation studies are carried out to assess coverage probabilities and interval lengths. Lastly, a real data example concerning cancer is included to illustrate the usage of our proposed method.
In this paper, we first propose a new accuracy measure proper for ``compound multi-class classification'' with general M main classes where M >= 3, namely ``hypervolume under compound ROC manifold HUMc".
The proposed HUMc evaluates the accuracy of a biomarker appropriately by identifying main classes without requiring the specification of an ordering for marker values of subclasses within each main class.
For confidence interval estimation of ${VUS_C}$, an efficient computing algorithm estimating its empirical estimator is discussed alongside the non-parametric bootstrap confidence interval, and simulation studies are carried out to assess coverage probabilities and interval lengths. Lastly, a real data example concerning cancer is included to illustrate the usage of our proposed method.
Aims
To propose a new accuracy metric for compound multi-class classification
Adapt and propose an efficient computing algorithm
Collaborators
Dr. Lili Tian, Department of Biostatistics, State University of New York at Buffalo