fix: pin PyMC version below 5.20.1 to avoid TypeError (closes #1397)
This PR fixes issue #1397 where sampling via PyMC raises a TypeError starting from PyMC v5.20.1.
Changes
- Updated
pyproject.tomlto constrain PyMC version: pymc >=5.0.0, <5.20.1
@Dev-Sudarshan are you able to reproduce the CI error locally when you install things from scratch?
e.g., in a fresh uv environment:
uv venv -p python3.12source .venv/bin/activateuv sync --all-extraspytest -m "not slow and not gpu" tests/
Yes, I was able to reproduce the CI error and the error was caused due to use of np.obj2sctype, which was removed in NumPy2.0. Our repo doesn't use this function directly, but one of the imported dependencies (likely PyTensor or a related backend) still references it. I temporarily resolved the CI failure by pinning NumPy to <2.0, which restores compatibility with the current dependency versions. Would you like me to update this PR to include the numpy<2.0 pin?
I see, thanks for checking @Dev-Sudarshan !
So, we added pymc>=5.0.0,<5.20.1. That upper bound prevents pip from installing the newer PyMC versions that gained NumPy 2 support, and instead it installs an older PyMC + older PyTensor that still use np.obj2sctype.
The two options we have now are either pin numpy<2.0 or drop the pinning of pymc<5.20.1.
I would prefer not restricting the major numpy dependency to <2.0 and instead accept the error we get here when using pymc MCMC.
I gave it a go with ChatGTP Codex and here's the result:
Problem
- Traced the crash to PyMC’s new Rich-based progress bar: the column "{task.fields[step_size]:0.2f}" now receives a NumPy array instead of a scalar. In PyMC ≤ 5.20 this field stayed scalar, but newer builds report step_size (and sometimes step_size_bar) as 1‑D arrays when mass-matrix adaptation is vector-valued, so Rich raises TypeError: unsupported format string passed to numpy.ndarray.format.
- While investigating we confirmed the value is generated inside PyMC’s DualAverageAdaptation.stats() / progress-bar manager, not by sbi.
- There isn’t yet a release-note or issue documenting this regression. What’s Next
Options
(1) disable PyMC’s progress bar from sbi (progressbar=False ∴ no Rich formatting) (2) sanitize those stats before they reach PyMC (convert 1‑element arrays to floats); or (3) pin PyMC to ≤ 5.20 until PyMC releases a fix.
I would suggest option (1) because we don't want (3), and for (2) we would need to write a wrapper around the PyMC API, which is more involved.
Would you be up for looking into this?
I’ve implemented option (1) by disabling the PyMC progress bar, and the update is now merged into the PR. Additionally, pre-commit identified unused sys and pymc imports, so I removed those as well. Please let me know if any further adjustments are required.
Codecov Report
:white_check_mark: All modified and coverable lines are covered by tests.
:warning: Please upload report for BASE (main@f3222f9). Learn more about missing BASE report.
Additional details and impacted files
@@ Coverage Diff @@
## main #1697 +/- ##
=======================================
Coverage ? 84.65%
=======================================
Files ? 137
Lines ? 11487
Branches ? 0
=======================================
Hits ? 9724
Misses ? 1763
Partials ? 0
| Flag | Coverage Δ | |
|---|---|---|
| unittests | 84.65% <ø> (?) |
Flags with carried forward coverage won't be shown. Click here to find out more.
| Files with missing lines | Coverage Δ | |
|---|---|---|
| sbi/inference/posteriors/mcmc_posterior.py | 81.72% <ø> (ø) |
|
| sbi/samplers/mcmc/pymc_wrapper.py | 88.40% <ø> (ø) |