Stochastic tensor contraction for quantum chemistry

Jiace Suna and Garnet Kin-Lic Chan (2026)
Highlighted by Jan Jensen

What this paper lacks in terms of punchy title, it makes up for in content. I guess I would have gone with something like "Monte Carlo Meets Coupled Cluster: Slashing the Cost of CCSD(T)" or "Stochastic Tensor Contraction Pushes CCSD(T) Toward Mean-Field Cost". 

Anyway, tensor contraction is the algebraic core of much of quantum chemistry: large multidimensional arrays representing amplitudes and integrals are multiplied and summed over shared indices to produce energies and intermediates. It matters because these contractions set the scaling wall for methods like CCSD(T), where the formal cost rises far faster than Hartree–Fock. 

This study uses importance samplling to evaluate the tensor contraction, Importance sampling means drawing the most important terms in a sum more often than the unimportant ones, while reweighting so the final estimator stays unbiased. Here, Sun and Chan use it to evaluate high-order tensor contractions stochastically.

The headline result is that stochastic tensor contraction (STC) drives the scaling of CCSD(T) down dramatically: from the usual O(N^6) and O(N^7) down to O(N^4). In practice, water-cluster tests show very large FLOP reductions and wall-time crossovers at surprisingly small sizes. 

Figure 7 in the paper is the real selling point, because it compares against the incumbent approximate workhorse, DLPNO-CCSD(T), on 20 realistic molecules. STC is faster than DLPNO for every system in the set, with speedups ranging from 2.5× to 32×, while also delivering smaller errors than all DLPNO/Normal results and 15 of 20 DLPNO/Tight results. Just as importantly, the STC errors stay tightly clustered around the chosen target of 0.2 kcal/mol, whereas DLPNO errors vary much more from system to system. That makes STC look not just fast, but controllable. 

Table 3 sharpens that message. Averaged over the benchmark set, STC has a mean absolute error of 0.2 kcal/mol at a geometric mean runtime of 10.7 min, compared with 3.00 kcal/mol / 58 min for DLPNO/Normal, 0.70 kcal/mol / 159 min for DLPNO/Tight, and 773 min for exact CCSD(T). So the paper’s central claim is not merely better asymptotic scaling, but a roughly order-of-magnitude win in both time and error relative to state-of-the-art local correlation in this benchmark. 

One caveat: while the speed-up is undeniably impressive, another likely limiting factor is memory. The paper notes the use of density fitting “to reduce memory requirements,” but does not really quantify memory use or memory scaling in the same systematic way as FLOPs and wall time. Given that modern CC implementations are often limited as much by storage and movement of intermediates as by raw arithmetic, that omission stands out. 

Overall, this is prototype code, but very exciting prototype code. It will be very interesting to see whether this stochastic route can mature into something that genuinely displaces DLPNO-CCSD(T) as the default reduced-cost gold-standard method. Code: GitHub repository

This work is licensed under a Creative Commons Attribution 4.0 International License.

Error Estimates for Solid-State Density-Functional Theory Predictions: An Overview by Means of the Ground-State Elemental Crystals

K. Lejaeghere, V. Van Speybroeck, G. Van Oost & S. Cottenier Critical Reviews in Solid State and Materials Sciences 2014, 39, 1-24
Contributed by David Bowler
Reposted from Atomistic Computer Simulations with permission

The question of how to characterise the accuracy of a computer code is a difficult one, and I have touched on these issues before (here, here and here for instance). However, given the large number of codes available, it should be possible to compare them to each other and to experiment (or higher level calculations) to test them. This is a well-established process in the quantum chemistry community, where there are various test sets for different properties, including enthalpies of formation(G97/2)[1], weak bonding (S22)[2] (and a vast database of different properties[3]).

A recent paper[4] and associated web site[5] offers a first approach for solid state codes, with the comparison based on the differences between all-electron and pseudopotential calculations. The presumption here is that all-electron calculations are the touchstone (though the website notes that the all-electron results have been refined to use extremely accurate tolerances and small muffin-tin radii, so there is clearly room for improvement in any method).

The idea of the comparison is to calculate binding energy curves for most elements in the periodic table, and from these curves to derive a single number which characterises the deviation from all-electron results. The deviation per element can be viewed, as can the deviation from experiment for the all-electron calculations. The deviation, delta, is defined by an integral over all calculated values of volume, and thus includes implicitly both the lattice constant and the bulk modulus, though not the cohesive energy. Most results shown on the website are for plane-wave codes, which generally perform rather well (the old norm-conserving FHI pseudopotentials are less accurate, though, and should be treated with care).

The approach is a good one, though a little heavy on the tester, and scripts to perform the necessary calculations are made freely available. However, the choice of the elemental form makes the tests rather restricted: there is no way to examine different types of bonding or different oxidation state, for instance. It is quite easy to imagine developing a set of these test suites for different purposes in solid state codes, just as there are different test sets in quantum chemistry.

The accuracy tests seem to be most illuminating for the pseudopotentials, rather than the codes themselves, and I think that it would be of immense value to the community if the pseudopotential generation details were made available. This should not just include the core radii and the reference configurations, but also a clear description of the pseudopotential algorithm (or an appropriate reference along with details).

There is something of a danger of using codes and supplied pseudopotential libraries as black boxes: there is a need to test parameters, though it’s rare (and I am happy to acknowledge that I don’t do it as much as I should). This paper and associated developments should go some way to standardising plane wave codes, and giving quantitative information on their reliability.

Note As I was writing this, a new paper in Science has just been published which takes a different view, and compares results from different functionals; it’s worth a read[6]

[1] J. Chem. Phys. 106, 1063 (1997) DOI:10.1063/1.473182
[2] Phys. Chem. Chem. Phys. 8, 1985 (2006) DOI: 10.1039/B600027D
[3] http://t1.chem.umn.edu/db/ and see arXiv http://arxiv.org/abs/1212.0944
[4] Crit. Rev. Sol. Stat. Mat. Sci. 39, 1–24. DOI:10.1080/10408436.2013.772503
[5] http://molmod.ugent.be/deltacodesdft
[6] Science 345, 197 (2014) DOI:10.1126/science.1253486

Confidence limits, error bars and method comparison in molecular modeling. Part 1: The calculation of confidence intervals

Anthony Nicholls Journal of Computer-Aided Molecular Design 2014 (Open Access)
Contributed by +Jan Jensen

Almost every computed number we report has an uncertainty and ...

... without an assessment of this uncertainty, or a description of how to estimate it, what we have really delivered is a report, not a prediction; “we did X, followed by Y, and got Z”. 

Of course we all know this and we faithfully report RMSD values, Pearson's correlation coefficient ($r$) and other measures of uncertainty.  However, when was the last time you saw an uncertainty attached to these quantities? In others words, how likely is it that a future study would compute the same RMSD value for a different set of experimental values using my method? Or, my $r$ value looks great but do I have enough data points?

This wonderful and very readable paper tells you how to compute the uncertainty in your uncertainties and what they mean. There will be a follow-up paper that will describe how meaningfully compare quantities for which such uncertainties have been computed. I can't wait.


This work is licensed under a Creative Commons Attribution 4.0 International License.

Numerical Errors and Chaotic Behavior in Docking Simulations

Miklos Feher and Christopher I. Williams, Journal of Chemical Information and Modeling, DOI: 10.1021/ci200598m (paywall)

The authors have made a very interesting study of the effect of numerical problems in molecular docking. Specifically, they have studied the two docking software GOLD and GLIDE, both with three different accuracy settings.

As the authors point out in the conclusion:  
"This study clearly demonstrates that seemingly insignificant differences in ligand input, such as small coordinate perturbations or permuting the atom order in an input file, can have a dramatic effect on the final top-scoring docked pose."

The authors have investigated how robust the docking algorithms are with respect to poses and scores for two different input variations. First, very small variations to the input structure (max 0.1 degree of torsional angle changes, total max 0.1 Å RMSD). Second, permutations of atom order in the ligand input files. Both are variations that the normal user would expect to have no effect on the final output of a docking run.

The results are highly interesting and show that the virtual screening settings in the software seem to correlate with low robustness. Hence, using standard settings (or higher) improves robustness for these two methods.
While one could expect that GLIDE which is a deterministic approach (empirical scoring function) perhaps would be less prone to robustness problems than GOLD (genetic algorithm), this does not seem to be the case. The only major difference between the two softwares is that GOLD seem to generate more normal based distributions of scores and RMSDs than GLIDE.

Both small changes in the torsions of the input structures and the permutation of atom order in the input files lead to many cases where the docking protocols simply fail to be robust and can result in vastly different top scoring poses, especially for the low accuracy settings. Thus, applications of docking software with virtual screening settings are more prone to problems with reproducibility, but even with the highest accuracy robustness cannot be guaranteed.

For the future, it would be highly interesting to see if these problems are the same for other docking software, and how they can be alleviated.

[edit: changed the comment on CPU time relative to robustness, since there is no statistical significant difference of the robustness of the two more accurate settings used in the two software]