Monday, June 9, 2014

Confidence limits, error bars and method comparison in molecular modeling. Part 1: The calculation of confidence intervals

Anthony Nicholls Journal of Computer-Aided Molecular Design 2014 (Open Access)
Contributed by +Jan Jensen

Almost every computed number we report has an uncertainty and ...
... without an assessment of this uncertainty, or a description of how to estimate it, what we have really delivered is a report, not a prediction; “we did X, followed by Y, and got Z”. 
Of course we all know this and we faithfully report RMSD values, Pearson's correlation coefficient ($r$) and other measures of uncertainty.  However, when was the last time you saw an uncertainty attached to these quantities? In others words, how likely is it that a future study would compute the same RMSD value for a different set of experimental values using my method? Or, my $r$ value looks great but do I have enough data points?

This wonderful and very readable paper tells you how to compute the uncertainty in your uncertainties and what they mean. There will be a follow-up paper that will describe how meaningfully compare quantities for which such uncertainties have been computed. I can't wait.

This work is licensed under a Creative Commons Attribution 4.0 International License.