Sunday, March 10, 2013

Are Protein Force Fields Getting Better? A Systematic Benchmark on 524 Diverse NMR Measurements

Kyle A. Beauchamp, Yu-Shan Lin, Rhiju Das, and Vijay S. Pande, J. Chem. Theory Comput. 2012 vol. 8 (4) pp. 1409-1414

Contributed by Ric Baron

Beauchamp and co-workers answer: YES! Proteins force fields are indeed getting better. After systematic analysis of 11 recent force field parameter sets, in combination with 5 popular fixed-charge water models and implicit solvent schemes, the authors conclude that two force fields in the pool (ff99sb-ildn- phi, ff99sb-ildn-NMR) that combine recent side chain and backbone torsion modifications achieved high accuracy. A simple metric to compare force field quality was employed and allows easy identificaiton of optimal force field/water model combinations (Figure 1). The benchmark set considered included dipeptides, tripeptides, tetra-alanine, and ubiquitin, and was validated against an extensive set of 524 NMR measurements. Expectedly, explicit solvent models outperform implicit solvent approaches.

Figure 1 in J. Chem. Theory Comput. 2012 vol. 8 (4) pp. 1409-1414
Figure 1. Quality of different protein force fields when combined with different treatement of the solvent. This is an unofficial adaptation of Figure 1 that appeared in an ACS publication, J. Chem. Theory Comput. 2012 vol. 8 (4) pp. 1409-1414 . ACS has not endorsed the content of this adaptation or the context of its use

I believe this study is of particular interest for the biomolecular simulation community due to the extensive and rigorous force field comparison presented. It is a well-known challenge to define metrics that fairly capture force field quality. It is also problematic in practice to perform apples-to-apples force field comparison studies. Among the chief obstacles, few software engines enable simulations with alternative force fields using (i) identical treatment of bonded and non-bonded interactions among the force fields compared, and/or (ii) underlying algorithms and energy terms identical to those employed for force field parametrization. Beauchamp et al. employ an extensive benchmark set and reach a compromise following solution (i), i.e. by considering some of the force fields that can be evaluated by simulations in GROMACS.

An interesting implication of Beauchamp’s thorough force-field comparison study is that - for the two optimal force fields in the group considered (ff99sb-ildn- phi, ff99sb-ildn-NMR) – the calculation error is comparable to the uncertainty in the experimental comparison. This observation has remarkable implications for the comparison of molecular simulations with experiments. Therefore, while protein force fields steadily improve, extracting additional force field improvements from NMR data “may require in the future increased accuracy in J coupling and chemical shift prediction.”

At this stage, this work focused on peptide conformational sampling. As the authors note in their concluding remarks, “other experimental data, such as solvation free energies, may be critical for evaluating other aspects of force field performance.”