Highly Variable and Degraded Performance from PyPi installation

[coltonbh]

Describe the bug

Running calculations using xtb installed using pip leads to highly variable and often severely degraded performance (up to 3 orders of magnitude slower runtime). This performance is stochastic as calculations will sometimes take 3 orders of magnitude less time too!

To Reproduce

Create one conda environment for xtb:

conda create -n xtb xtb-python -c conda-forge 

Create a python environment for a pip install

python -m venv env
source ./env/bin/activate
pip install xtb

Run the following script in each environment to compare timings. Run the script multiple time and you will see the performance of the pip install vary wildly from >.5 seconds to <.003 seconds.

import timeit

import numpy as np
from xtb.interface import Calculator, Param
from xtb.libxtb import VERBOSITY_FULL


numbers = np.array([8, 1, 1])
positions = np.array(
    [
        [0.0, 0.0, 0.0],
        [0.524, 1.687, 0.480],
        [1.146, -0.450, -1.354],
    ]
)


def calculate_xtb():
    calc = Calculator(Param.GFN2xTB, numbers, positions)
    calc.set_verbosity(VERBOSITY_FULL)
    calc.set_output("logs.out")
    calc.singlepoint()


# Number of times you want each function to be executed
number_of_executions = 10

# Time calculate_xtb
xtb_time = timeit.timeit(
    "calculate_xtb()", globals=globals(), number=number_of_executions
)
print(
    f"Average execution time for calculate_xtb: {xtb_time / number_of_executions:.5f} seconds"
)

1. What is your setup, which version are you running (current master is subject to change) Latest conda and pip installs.

Timings in the conda environment will be stable around 0.003 seconds (on my laptop). Timings for the pip environment will vary from ~0.003 to >0.6 seconds.

Expected behaviour

The performance is the same for xtb regardless how it is installed. The performance will not vary over 3 orders of magnitude WITHIN the same install (i.e., the pip install will perform the same as the conda install on some executions and will perform 1000x slower on others. It appears random).

Additional context

The fact that the performance CAN be the same suggests something very funny is going on.

[awvwgk]

For the PyPI distribution xtb-python is linked with OpenBLAS which might not have optimal performance compared to MKL as available on conda-forge. Also, the distribution on PyPI is currently not updated with new xtb releases while I am rebuilding the conda-forge one for each new xtb release to include improvements and bug fixes.

In general xtb-python should be considered in maintenance mode and there probably won't be any future development in this repository. Maybe we can reimplement the API using tblite at some point and xtb-python will just redirect to the new API.

[coltonbh]

Sounds good!

For clarity this performance degradation occurs using the SAME distribution for repeated calculations. The same calculation using the same distribution varies in performance by over 3 order of magnitude! Something funny is happening for sure ;P Hear you on maintenance mode--just wanted to add clarity for future reference!