Provide information on accuracy/precision of an engine

[ltalirz]

Comment provided by reviewer of the article

When it comes to ranking the simulation engines, no factors are more important than the accuracy/precision and pace (comment 1) associated with these engines. I expected to see some discussion regarding the accuracy and pace of simulation engines over the years. In this regard, that would be great if the authors could rank engines based on either accuracy, pace or even both factors.

[ltalirz]

I certainly agree that measures of accuracy and speed would be very useful to include in the collection. I would also point out, however, that defining fair measures for accuracy and speed that can be applied across the codes in the list is a very difficult task; and even if they can be defined, having to measure them periodically for all codes on the list would add a currently unsustainable maintenance burden for the collection.

Perhaps it would be feasible, however, to integrate data from existing efforts in this domain. For example, for DFT codes there is the delta-DFT effort that compares equation- of-states calculations for a range of elemental solids against the Wien2K reference im- plementation. As a first step, I have added a “benchmarks” column to the collection and added the “delta-dft” tag (which links to the delta-dft web site) wherever applicable. This does not yet allow to sort by accuracy directly on atomistic.software but at least the comparison is only one click away. So far, the “benchmarks” column is hidden by default but I would be happy to display it by default if more benchmarks could be added.

Todo:

• [ ] collect further benchmarks
• [ ] decide whether there is enough data on "benchmarks" to keep the column (and potentially even display it by default)