Visibly different structure depending on OMP_NUM_THREADS

[TimoSommer]

Describe the bug xtb structure optimization seems to give visibly different structures depending on the value of OMP_NUM_THREADS.

To Reproduce Steps to reproduce the behaviour:

1. Input file: test.xyz.gz
2. Extract input file with gunzip test.xyz.gz Serial run of xTB
3. run export OMP_NUM_THREADS=1
4. Run xtb --gfn 2 --chrg 0 --uhf 4 --etemp 300 --iterations 250 --tblite --alpb water --spinpol --ohess tight test.xyz
5. Rename the output file xtbopt.xyz to xtbopt_1core.xyz
6. Delete output files g98.out, xtbrestart, vibspectrum, hessian, .xtboptok, xtbopt.log Parallel run of xTB
7. Run export OMP_NUM_THREADS=2
8. Run xtb --gfn 2 --chrg 0 --uhf 4 --etemp 300 --iterations 250 --tblite --alpb water --spinpol --ohess tight test.xyz (exactly as above)
9. Rename the output file xtbopt.xyz to xtbopt_2cores.xyz Results
10. The xyz files and final energies (as in the xyz files comments) between xtbopt_1core.xyz and xtbopt_2cores.xyz are slightly different. When looking at the structures in the viewer VESTA, the atoms visibly shift. xtbopt_1core.xyz.gz xtbopt_2cores.xyz.gz

Expected behaviour Ideally, I would have expected that the output is exactly the same, independent of the number of cores used. I can understand that there might be some tiny numerical differences between the runs, but here it seems like the differences have become quite big. If we set OMP_NUM_THREADS to something higher than 2, the structures are different again.

Additional context I have tried to find a minimal working example with less xtb options, but reducing the options seemed to also reduce the magnitude of the difference. The minimal example in which I could still visibly see atoms shift in VESTA was with the following command: xtb --gfn 2 --chrg 0 --uhf 4 --alpb water --tblite --opt tight test.xyz

Versions xtb version 6.6.1 (8d0f1dd) compiled by '[email protected]' on 2023-08-01

[Albkat]

Hi,

Yes, there were some race conditions in the previous version. Have you tested xtb version 6.7.0?

[TimoSommer]

My laptop is a Mac, it seems like version 6.7.0 is out only for Linux and Windows so far. I will try it as soon as conda is updated to 6.7.0. Btw, I noticed in my tests that sometimes there can also be quite large differences in energy (up to 0.1 eV) for a structural optimization between different runs on different machines (e.g. my Mac laptop and a Linux cluster), even with OMP_NUM_THREADS=1. Would this also be fixed by the update?