pymc
pymc copied to clipboard
Published
20 hours ago •
pymc-devs
pymc-devs
BUG: Missing `constant_data` in `InferenceData` when defining observed data with same dims.
Describe the issue:
I have defined my data in a pm.Data container along with its dimensionality (dims=) but when i use these points as likelihood and also pass the dimensionality they are no longer in the constant_data of the InferenceData object. A workaround is to avoid defining dimensionality in pm.Normal like this: obs = pm.Normal('obs', mu=mu, sigma=sigma, observed=data).
I think this is a bug from newer versions of pymc because i had no problems with pymc 5.5.
I tried to debug the issue and could be in pm.backend.arviz.find_constants where it checks if an observed variable could be part of the generative graph.
Reproduceable code example:
import pymc as pm
from numpy.random import default_rng
rng = default_rng(42)
data = rng.normal(size=20)
coords = {'data_dim': data}
with pm.Model(coords=coords) as model:
data = pm.Data('data', data, dims='data_dim')
mu = pm.Normal('mu', mu=0, sigma=1)
sigma = pm.HalfNormal('sigma', sigma=4)
obs = pm.Normal('obs', mu=mu, sigma=sigma, observed=data, dims='data_dim')
idata = pm.to_inference_data(prior={})
assert 'data' in idata.constant_data
Error message:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Cell In[8], line 17
14 obs = pm.Normal('obs', mu=mu, sigma=sigma, observed=data, dims='data_dim')
15 idata = pm.to_inference_data(prior={})
---> 17 assert 'data' in idata.constant_data
AttributeError: 'InferenceData' object has no attribute 'constant_data'
PyMC version information:
# Name Version Build Channel
accelerate 1.2.1 pyhd8ed1ab_0 conda-forge
appnope 0.1.4 pyhd8ed1ab_1 conda-forge
archspec 0.2.3 pyhd8ed1ab_0 conda-forge
arviz 0.20.0 pyhd8ed1ab_1 conda-forge
asttokens 3.0.0 pyhd8ed1ab_1 conda-forge
atk-1.0 2.38.0 hd03087b_2 conda-forge
blas 2.125 openblas conda-forge
blas-devel 3.9.0 25_osxarm64_openblas conda-forge
boltons 24.0.0 pyhd8ed1ab_1 conda-forge
brotli 1.1.0 hd74edd7_2 conda-forge
brotli-bin 1.1.0 hd74edd7_2 conda-forge
brotli-python 1.1.0 py312hde4cb15_2 conda-forge
bzip2 1.0.8 h99b78c6_7 conda-forge
c-ares 1.34.4 h5505292_0 conda-forge
ca-certificates 2024.12.14 hf0a4a13_0 conda-forge
cached-property 1.5.2 hd8ed1ab_1 conda-forge
cached_property 1.5.2 pyha770c72_1 conda-forge
cachetools 5.5.0 pyhd8ed1ab_1 conda-forge
cairo 1.18.2 h6a3b0d2_1 conda-forge
cctools_osx-arm64 1010.6 h908b477_2 conda-forge
certifi 2024.8.30 pyhd8ed1ab_0 conda-forge
cffi 1.17.1 py312h0fad829_0 conda-forge
charset-normalizer 3.4.0 pyhd8ed1ab_1 conda-forge
clang 18.1.8 default_h675cc0c_3 conda-forge
clang-18 18.1.8 default_h5c12605_3 conda-forge
clang_impl_osx-arm64 18.1.8 h2ae9ea5_21 conda-forge
clang_osx-arm64 18.1.8 h54d7cd3_21 conda-forge
clangxx 18.1.8 default_h675cc0c_3 conda-forge
clangxx_impl_osx-arm64 18.1.8 h555f467_21 conda-forge
clangxx_osx-arm64 18.1.8 h54d7cd3_21 conda-forge
cloudpickle 3.1.0 pyhd8ed1ab_1 conda-forge
colorama 0.4.6 pyhd8ed1ab_1 conda-forge
comm 0.2.2 pyhd8ed1ab_1 conda-forge
compiler-rt 18.1.8 h856b3c1_1 conda-forge
compiler-rt_osx-arm64 18.1.8 h832e737_1 conda-forge
conda 24.11.0 py312h81bd7bf_0 conda-forge
conda-libmamba-solver 24.11.1 pyhd8ed1ab_0 conda-forge
conda-package-handling 2.4.0 pyha770c72_1 conda-forge
conda-package-streaming 0.11.0 pyhd8ed1ab_0 conda-forge
cons 0.4.6 pyhd8ed1ab_0 conda-forge
contourpy 1.3.1 py312hb23fbb9_0 conda-forge
cpp-expected 1.1.0 hffc8910_0 conda-forge
cycler 0.12.1 pyhd8ed1ab_1 conda-forge
debugpy 1.8.11 py312hd8f9ff3_0 conda-forge
decorator 5.1.1 pyhd8ed1ab_1 conda-forge
distro 1.9.0 pyhd8ed1ab_0 conda-forge
et_xmlfile 2.0.0 pyhd8ed1ab_1 conda-forge
etuples 0.3.9 pyhd8ed1ab_0 conda-forge
exceptiongroup 1.2.2 pyhd8ed1ab_1 conda-forge
executing 2.1.0 pyhd8ed1ab_1 conda-forge
filelock 3.16.1 pyhd8ed1ab_1 conda-forge
fmt 11.0.2 h420ef59_0 conda-forge
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 h77eed37_3 conda-forge
fontconfig 2.15.0 h1383a14_1 conda-forge
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
fonttools 4.55.3 py312h998013c_0 conda-forge
freetype 2.12.1 hadb7bae_2 conda-forge
fribidi 1.0.10 h27ca646_0 conda-forge
frozendict 2.4.6 py312h0bf5046_0 conda-forge
fsspec 2024.10.0 pyhd8ed1ab_1 conda-forge
gdk-pixbuf 2.42.12 h7ddc832_0 conda-forge
gmp 6.3.0 h7bae524_2 conda-forge
gmpy2 2.1.5 py312h524cf62_3 conda-forge
graphite2 1.3.13 hebf3989_1003 conda-forge
graphviz 12.0.0 hbf8cc41_0 conda-forge
gtk2 2.24.33 h91d5085_5 conda-forge
gts 0.7.6 he42f4ea_4 conda-forge
h2 4.1.0 pyhd8ed1ab_1 conda-forge
h5netcdf 1.4.1 pyhd8ed1ab_1 conda-forge
h5py 3.12.1 nompi_py312h3cd0824_102 conda-forge
harfbuzz 9.0.0 h997cde5_1 conda-forge
hdf5 1.14.3 nompi_ha698983_108 conda-forge
hpack 4.0.0 pyhd8ed1ab_1 conda-forge
huggingface_hub 0.26.5 pyhd8ed1ab_1 conda-forge
hyperframe 6.0.1 pyhd8ed1ab_1 conda-forge
icu 75.1 hfee45f7_0 conda-forge
idna 3.10 pyhd8ed1ab_1 conda-forge
importlib-metadata 8.5.0 pyha770c72_1 conda-forge
ipykernel 6.29.5 pyh57ce528_0 conda-forge
ipython 8.30.0 pyh707e725_0 conda-forge
jedi 0.19.2 pyhd8ed1ab_1 conda-forge
jinja2 3.1.4 pyhd8ed1ab_1 conda-forge
jsonpatch 1.33 pyhd8ed1ab_1 conda-forge
jsonpointer 3.0.0 py312h81bd7bf_1 conda-forge
jupyter_client 8.6.3 pyhd8ed1ab_1 conda-forge
jupyter_core 5.7.2 pyh31011fe_1 conda-forge
kiwisolver 1.4.7 py312h6142ec9_0 conda-forge
krb5 1.21.3 h237132a_0 conda-forge
lcms2 2.16 ha0e7c42_0 conda-forge
ld64_osx-arm64 951.9 hfc0fa09_2 conda-forge
lerc 4.0.0 h9a09cb3_0 conda-forge
libabseil 20240722.0 cxx17_hf9b8971_1 conda-forge
libaec 1.1.3 hebf3989_0 conda-forge
libarchive 3.7.7 h3b16cec_3 conda-forge
libblas 3.9.0 25_osxarm64_openblas conda-forge
libbrotlicommon 1.1.0 hd74edd7_2 conda-forge
libbrotlidec 1.1.0 hd74edd7_2 conda-forge
libbrotlienc 1.1.0 hd74edd7_2 conda-forge
libcblas 3.9.0 25_osxarm64_openblas conda-forge
libclang-cpp18.1 18.1.8 default_h5c12605_3 conda-forge
libcurl 8.11.1 h73640d1_0 conda-forge
libcxx 19.1.5 ha82da77_0 conda-forge
libcxx-devel 19.1.5 h6dc3340_0 conda-forge
libdeflate 1.22 hd74edd7_0 conda-forge
libedit 3.1.20191231 hc8eb9b7_2 conda-forge
libev 4.33 h93a5062_2 conda-forge
libexpat 2.6.4 h286801f_0 conda-forge
libffi 3.4.2 h3422bc3_5 conda-forge
libgd 2.3.3 hac1b3a8_10 conda-forge
libgfortran 5.0.0 13_2_0_hd922786_3 conda-forge
libgfortran5 13.2.0 hf226fd6_3 conda-forge
libglib 2.82.2 h07bd6cf_0 conda-forge
libiconv 1.17 h0d3ecfb_2 conda-forge
libintl 0.22.5 h8414b35_3 conda-forge
libjpeg-turbo 3.0.0 hb547adb_1 conda-forge
liblapack 3.9.0 25_osxarm64_openblas conda-forge
liblapacke 3.9.0 25_osxarm64_openblas conda-forge
libllvm18 18.1.8 h5090b49_2 conda-forge
liblzma 5.6.3 h39f12f2_1 conda-forge
liblzma-devel 5.6.3 h39f12f2_1 conda-forge
libmamba 2.0.5 h4621f14_0 conda-forge
libmambapy 2.0.5 py312hd07f1d4_0 conda-forge
libnghttp2 1.64.0 h6d7220d_0 conda-forge
libopenblas 0.3.28 openmp_hf332438_1 conda-forge
libpng 1.6.44 hc14010f_0 conda-forge
libprotobuf 5.28.2 h8f0b736_0 conda-forge
librsvg 2.58.4 h40956f1_0 conda-forge
libsodium 1.0.20 h99b78c6_0 conda-forge
libsolv 0.7.30 h6c9b7f8_0 conda-forge
libsqlite 3.47.2 h3f77e49_0 conda-forge
libssh2 1.11.1 h9cc3647_0 conda-forge
libtiff 4.7.0 ha962b0a_2 conda-forge
libtorch 2.5.1 cpu_generic_h44abd74_6 conda-forge
libuv 1.49.2 h7ab814d_0 conda-forge
libwebp-base 1.4.0 h93a5062_0 conda-forge
libxcb 1.17.0 hdb1d25a_0 conda-forge
libxml2 2.13.5 h178c5d8_1 conda-forge
libzlib 1.3.1 h8359307_2 conda-forge
llvm-openmp 19.1.5 hdb05f8b_0 conda-forge
llvm-tools 18.1.8 h5090b49_2 conda-forge
llvm-tools-18 18.1.8 h5090b49_2 conda-forge
logical-unification 0.4.6 pyhd8ed1ab_0 conda-forge
lz4-c 1.10.0 h286801f_1 conda-forge
lzo 2.10 h93a5062_1001 conda-forge
macosx_deployment_target_osx-arm64 11.0 h6553868_1 conda-forge
markdown-it-py 3.0.0 pyhd8ed1ab_1 conda-forge
markupsafe 3.0.2 py312h998013c_1 conda-forge
matplotlib-base 3.9.4 py312hdbc7e53_0 conda-forge
matplotlib-inline 0.1.7 pyhd8ed1ab_1 conda-forge
mdurl 0.1.2 pyhd8ed1ab_1 conda-forge
menuinst 2.2.0 py312h81bd7bf_0 conda-forge
minikanren 1.0.3 pyhd8ed1ab_0 conda-forge
mpc 1.3.1 h8f1351a_1 conda-forge
mpfr 4.2.1 hb693164_3 conda-forge
mpmath 1.3.0 pyhd8ed1ab_1 conda-forge
multipledispatch 0.6.0 pyhd8ed1ab_1 conda-forge
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
ncurses 6.5 h7bae524_1 conda-forge
nest-asyncio 1.6.0 pyhd8ed1ab_1 conda-forge
networkx 3.4.2 pyh267e887_2 conda-forge
nlohmann_json 3.11.3 h00cdb27_1 conda-forge
nomkl 1.0 h5ca1d4c_0 conda-forge
numpy 1.26.4 py312h8442bc7_0 conda-forge
openblas 0.3.28 openmp_hea878ba_1 conda-forge
openjpeg 2.5.3 h8a3d83b_0 conda-forge
openpyxl 3.1.5 py312hf6e0af7_1 conda-forge
openssl 3.4.0 h39f12f2_0 conda-forge
packaging 24.2 pyhd8ed1ab_2 conda-forge
pandas 2.2.2 py312h8ae5369_1 conda-forge
pango 1.54.0 h3e3e505_3 conda-forge
parso 0.8.4 pyhd8ed1ab_1 conda-forge
patsy 1.0.1 pyhd8ed1ab_1 conda-forge
pcre2 10.44 h297a79d_2 conda-forge
pexpect 4.9.0 pyhd8ed1ab_1 conda-forge
pickleshare 0.7.5 pyhd8ed1ab_1004 conda-forge
pillow 11.0.0 py312haf37ca6_0 conda-forge
pip 24.3.1 pyh8b19718_0 conda-forge
pixman 0.44.2 h2f9eb0b_0 conda-forge
platformdirs 4.3.6 pyhd8ed1ab_1 conda-forge
plotly 5.24.1 pyhd8ed1ab_1 conda-forge
pluggy 1.5.0 pyhd8ed1ab_1 conda-forge
prompt-toolkit 3.0.48 pyha770c72_1 conda-forge
psutil 6.1.0 py312h0bf5046_0 conda-forge
pthread-stubs 0.4 hd74edd7_1002 conda-forge
ptyprocess 0.7.0 pyhd8ed1ab_1 conda-forge
pure_eval 0.2.3 pyhd8ed1ab_1 conda-forge
pybind11-abi 4 hd8ed1ab_3 conda-forge
pycosat 0.6.6 py312hea69d52_2 conda-forge
pycparser 2.22 pyh29332c3_1 conda-forge
pygments 2.18.0 pyhd8ed1ab_1 conda-forge
pymc 5.19.1 hd8ed1ab_0 conda-forge
pymc-base 5.19.1 pyhd8ed1ab_0 conda-forge
pyparsing 3.2.0 pyhd8ed1ab_2 conda-forge
pysocks 1.7.1 pyha55dd90_7 conda-forge
pytensor 2.26.4 py312h2285823_0 conda-forge
pytensor-base 2.26.4 py312h6002e46_0 conda-forge
python 3.12.8 hc22306f_1_cpython conda-forge
python-dateutil 2.9.0.post0 pyhff2d567_1 conda-forge
python-graphviz 0.20.3 pyh91182bf_2 conda-forge
python-tzdata 2024.2 pyhd8ed1ab_1 conda-forge
python_abi 3.12 5_cp312 conda-forge
pytorch 2.5.1 cpu_generic_py312h99d64c8_6 conda-forge
pytz 2024.2 pyhd8ed1ab_1 conda-forge
pyyaml 6.0.2 py312h024a12e_1 conda-forge
pyzmq 26.2.0 py312hf8a1cbd_3 conda-forge
qhull 2020.2 h420ef59_5 conda-forge
readline 8.2 h92ec313_1 conda-forge
reproc 14.2.5.post0 h5505292_0 conda-forge
reproc-cpp 14.2.5.post0 h286801f_0 conda-forge
requests 2.32.3 pyhd8ed1ab_1 conda-forge
rich 13.9.4 pyhd8ed1ab_1 conda-forge
ruamel.yaml 0.18.6 py312h0bf5046_1 conda-forge
ruamel.yaml.clib 0.2.8 py312h0bf5046_1 conda-forge
safetensors 0.4.5 py312he431725_0 conda-forge
scipy 1.14.1 py312h6bb24ec_2 conda-forge
seaborn 0.13.2 hd8ed1ab_3 conda-forge
seaborn-base 0.13.2 pyhd8ed1ab_3 conda-forge
setuptools 75.6.0 pyhff2d567_1 conda-forge
sigtool 0.1.3 h44b9a77_0 conda-forge
simdjson 3.11.3 ha393de7_0 conda-forge
six 1.17.0 pyhd8ed1ab_0 conda-forge
sleef 3.7 h8391f65_2 conda-forge
spdlog 1.14.1 h6d8af72_1 conda-forge
stack_data 0.6.3 pyhd8ed1ab_1 conda-forge
statsmodels 0.14.4 py312h755e627_0 conda-forge
sympy 1.13.3 pypyh2585a3b_103 conda-forge
tapi 1300.6.5 h03f4b80_0 conda-forge
tenacity 9.0.0 pyhd8ed1ab_1 conda-forge
threadpoolctl 3.5.0 pyhc1e730c_0 conda-forge
tk 8.6.13 h5083fa2_1 conda-forge
toolz 1.0.0 pyhd8ed1ab_1 conda-forge
tornado 6.4.2 py312hea69d52_0 conda-forge
tqdm 4.67.1 pyhd8ed1ab_0 conda-forge
traitlets 5.14.3 pyhd8ed1ab_1 conda-forge
truststore 0.10.0 pyhd8ed1ab_0 conda-forge
typing-extensions 4.12.2 hd8ed1ab_1 conda-forge
typing_extensions 4.12.2 pyha770c72_1 conda-forge
tzdata 2024b hc8b5060_0 conda-forge
unicodedata2 15.1.0 py312h0bf5046_1 conda-forge
urllib3 2.2.3 pyhd8ed1ab_1 conda-forge
wcwidth 0.2.13 pyhd8ed1ab_1 conda-forge
wheel 0.45.1 pyhd8ed1ab_1 conda-forge
xarray 2024.11.0 pyhd8ed1ab_0 conda-forge
xarray-einstats 0.8.0 pyhd8ed1ab_0 conda-forge
xorg-libxau 1.0.12 h5505292_0 conda-forge
xorg-libxdmcp 1.1.5 hd74edd7_0 conda-forge
xz 5.6.3 h9a6d368_1 conda-forge
xz-gpl-tools 5.6.3 h9a6d368_1 conda-forge
xz-tools 5.6.3 h39f12f2_1 conda-forge
yaml 0.2.5 h3422bc3_2 conda-forge
yaml-cpp 0.8.0 h13dd4ca_0 conda-forge
zeromq 4.3.5 hc1bb282_7 conda-forge
zipp 3.21.0 pyhd8ed1ab_1 conda-forge
zstandard 0.23.0 py312h15fbf35_1 conda-forge
zstd 1.5.6 hb46c0d2_0 conda-forge
Context for the issue:
No response
]
:tada: Welcome to PyMC! :tada: We're really excited to have your input into the project! :sparkling_heart:
If you haven't done so already, please make sure you check out our Contributing Guidelines and Code of Conduct.
![welcome[bot] avatar](https://avatars.githubusercontent.com/in/4141?v=4 "Published by a pub.dev verified publisher"){kind=small}
I'm also experiencing this issue. Has there been any progress or workarounds been made so far?
@bhroben this is by design. As you are doing observed=data when defining obs you should instead get the values of data as the variable obs in the observed_data group. This allows ArviZ be able to automatically do things like plot_ppc or plot_loo_pit
