OPLS-DA: confidence interval for accuracy metric

0

Entering edit mode

Eddie • 0

@f2dcf917

Last seen 2.3 years ago

Belgium

I fitted an OPLS-DA and obtained my confusion matrix for testing dataset. My testing dataset is balanced ( equal numbers for the two groups being compared) so it is good to use accuracy as a metric for comparing models. I can easily calculate the accuracy from the confusions matrix but I need to provide a confidence interval for it. Do you know how to determine that using confusion matrix. Thank you.

Metabolomics • 1.1k views

ADD COMMENT • link 2.3 years ago Eddie • 0

0

Entering edit mode

The bootstrapping approach can be used to compute the CIs: see here for an example: Mendez et al. (2019). « A comparative evaluation of the generalised predictive ability of eight machine learning algorithms across ten clinical metabolomics data sets for binary classification », Metabolomics, vol. 15, nᵒ 12, p. 150, doi: 10.1007/s11306-019-1612-4
The pROC package may be useful to compute such metrics: see for instance: https://stackoverflow.com/questions/63218559/how-to-obtain-the-95-confidence-interval-of-accuracy-of-roc