Optimum is never successfully computed on `scipy>=1.11.0`

[MinusKelvin]

• Chess Tuning Tools version: 0.9.5
• Python version: 3.9.18
• Operating System: Ubuntu 22.04.3 LTS (Linux 5.15.0-84-generic)

Description

I am trying to tune two LMR-related parameters in my engine (https://github.com/MinusKelvin/ice4/tree/log-lmr-spsa). However, CTT always fails to compute the optimum (once the GP kernel becomes non-None after the 15th iteration), displaying Computing current optimum was not successful. This can happen in rare cases and running the tuner again usually works.

command:

PATH=".:$PATH" tune local -c config.json --verbose

config.json:

{
    "engines": [
        {
            "command": "ice4-ob",
            "fixed_parameters": {
                "Hash": 64
            }
        },
        {
            "command": "ice4-ob",
            "fixed_parameters": {
                "Hash": 64
            }
        }
    ],
    "parameter_ranges": {
        "aLMR_CONSTANT": "Integer(0, 150)",
        "bLMR_DIVISOR": "Integer(50, 300)"
    },
    "engine1_tc": "1+0.01",
    "engine2_tc": "1+0.01",
    "rounds": 4,
    "opening_file": "4moves_noob.epd",
    "concurrency": 8,
    "plot_every": 0
}

log.txt

I have tried running the tuner again after deleting model.pkl and data.npz several times but the error continued to occur. I have also tried ignoring the error and letting it run for several hundred iterations but the error persisted. Deleting the model and importing the data points resulted in the same error. I have verified that when CTT runs a match, the parameters are correctly set within the engine. Finally, I have tried changing the parameter ranges to be reals and modified the engine to accept this input, but the error persisted.

Further information

These are the versions of all packages installed in the venv:

Package             Version
------------------- ------------
arviz               0.17.0
atomicwrites        1.4.1
bask                0.10.9
chess-tuning-tools  0.9.5
click               8.1.7
contourpy           1.2.0
cycler              0.12.1
dill                0.3.7
emcee               3.1.4
fonttools           4.47.2
h5netcdf            1.3.0
h5py                3.10.0
importlib-resources 6.1.1
joblib              1.3.2
kiwisolver          1.4.5
matplotlib          3.8.2
numpy               1.23.5
packaging           23.2
pandas              2.2.0
pillow              10.2.0
pip                 23.0.1
pyaml               23.12.0
pyparsing           3.1.1
python-dateutil     2.8.2
pytz                2023.3.post1
PyYAML              6.0.1
scikit-learn        1.4.0
scikit-optimize     0.9.0
scipy               1.12.0
setuptools          69.0.3
six                 1.16.0
threadpoolctl       3.2.0
tqdm                4.66.1
typing_extensions   4.9.0
tzdata              2023.4
xarray              2024.1.0
xarray-einstats     0.7.0
zipp                3.17.0

cutechess-cli is version 1.2.0. More specifically, it is this build: https://github.com/AndyGrant/OpenBench/blob/cf1cabfb92baae475ea8963243a2b83f948e4a7c/CoreFiles/cutechess-linux

[MinusKelvin]

I've narrowed this down to being an issue with the version of scipy. No versions of scipy>=1.11.0 work.

[lithander]

based on above observation I got it to run by installing scipy==1.10.1 manually