cuml
cuml copied to clipboard
Published
20 hours ago •
rapidsai
rapidsai
[BUG]Unexpectedly High Forecast Values in Batch Prediction with cuML's Auto ARIMA
Describe the bug I'm having some issues using cuML's Auto ARIMA model for large-scale time series forecasting. Specifically, when I tried to do a batch forecast on about 50,000 time series data, I got some unusually high values in the forecast results. However, when I select the unusually sequence from these data and predict it alone, I can get normal prediction results. Steps/Code to reproduce bug Hard to discribe it here, the data is private and large. One of the sequence is: [224, 69, 115, 94, 59, 63, 60, 52, 87, 118, 132, 149, 139, 89, 97, 115, 98, 82, 55, 77, 96, 133, 112, 92, 170, 128, 94, 84, 63, 75, 56, 77, 85, 121, 126, 101, 197, 98, 89, 71, 72, 30, 47, 73, 69, 106, 110, 128]
batch prediction gives '7405687891.374923'
Expected behavior The results of individual predictions and batch predictions should be the same. Environment details (please complete the following information):
- Environment location: [Docker]
- GPU Model/Driver: [H800 and 525.125.06]
- CUDA: [12.2]
- Method of cuDF & cuML install: [conda]
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
_sysroot_linux-64_curr_repodata_hack 3 h69a702a_14 conda-forge
aiohttp 3.9.3 pypi_0 pypi
aiosignal 1.3.1 pypi_0 pypi
alsa-lib 1.2.11 hd590300_1 conda-forge
anyio 4.3.0 pypi_0 pypi
argon2-cffi 23.1.0 pypi_0 pypi
argon2-cffi-bindings 21.2.0 pypi_0 pypi
arrow 1.3.0 pypi_0 pypi
asttokens 2.4.1 pyhd8ed1ab_0 conda-forge
async-timeout 4.0.3 pypi_0 pypi
attrs 23.2.0 pypi_0 pypi
beautifulsoup4 4.12.3 pypi_0 pypi
binutils 2.40 hdd6e379_0 conda-forge
binutils_impl_linux-64 2.40 hf600244_0 conda-forge
binutils_linux-64 2.40 hdade7a5_3 conda-forge
bleach 6.1.0 pypi_0 pypi
bokeh 3.3.4 pypi_0 pypi
bzip2 1.0.8 hd590300_5 conda-forge
c-compiler 1.7.0 hd590300_0 conda-forge
ca-certificates 2024.2.2 hbcca054_0 conda-forge
cachetools 5.3.3 pypi_0 pypi
certifi 2024.2.2 pypi_0 pypi
cffi 1.16.0 pypi_0 pypi
charset-normalizer 3.3.2 pypi_0 pypi
click 8.1.7 pypi_0 pypi
click-plugins 1.1.1 pypi_0 pypi
cligj 0.7.2 pypi_0 pypi
cloudpickle 3.0.0 pypi_0 pypi
colorcet 3.1.0 pypi_0 pypi
comm 0.2.2 pyhd8ed1ab_0 conda-forge
contourpy 1.2.0 pypi_0 pypi
cuda-cccl 12.4.99 ha770c72_0 conda-forge
cuda-cccl-impl 2.0.1 ha770c72_1 conda-forge
cuda-cccl_linux-64 12.4.99 ha770c72_0 conda-forge
cuda-command-line-tools 12.4.0 ha770c72_0 conda-forge
cuda-compiler 12.4.0 hbad6d8a_0 conda-forge
cuda-crt-dev_linux-64 12.4.99 ha770c72_0 conda-forge
cuda-crt-tools 12.4.99 ha770c72_0 conda-forge
cuda-cudart 12.4.99 hd3aeb46_0 conda-forge
cuda-cudart-dev 12.4.99 hd3aeb46_0 conda-forge
cuda-cudart-dev_linux-64 12.4.99 h59595ed_0 conda-forge
cuda-cudart-static 12.4.99 hd3aeb46_0 conda-forge
cuda-cudart-static_linux-64 12.4.99 h59595ed_0 conda-forge
cuda-cudart_linux-64 12.4.99 h59595ed_0 conda-forge
cuda-cuobjdump 12.4.99 h59595ed_0 conda-forge
cuda-cupti 12.4.99 h59595ed_0 conda-forge
cuda-cupti-dev 12.4.99 h59595ed_0 conda-forge
cuda-cuxxfilt 12.4.99 h59595ed_0 conda-forge
cuda-documentation 12.2.53 0 nvidia/label/cuda-12.2.0
cuda-driver-dev 12.4.99 hd3aeb46_0 conda-forge
cuda-driver-dev_linux-64 12.4.99 h59595ed_0 conda-forge
cuda-gdb 12.4.99 hd47b8d6_0 conda-forge
cuda-libraries 12.4.0 ha770c72_0 conda-forge
cuda-libraries-dev 12.4.0 ha770c72_0 conda-forge
cuda-libraries-static 12.4.0 ha770c72_0 conda-forge
cuda-nsight 12.4.99 ha770c72_0 conda-forge
cuda-nvcc 12.4.99 hcdd1206_0 conda-forge
cuda-nvcc-dev_linux-64 12.4.99 ha770c72_0 conda-forge
cuda-nvcc-impl 12.4.99 hd3aeb46_0 conda-forge
cuda-nvcc-tools 12.4.99 hd3aeb46_0 conda-forge
cuda-nvcc_linux-64 12.4.99 h8a487aa_0 conda-forge
cuda-nvdisasm 12.4.99 h59595ed_0 conda-forge
cuda-nvml-dev 12.4.99 h59595ed_0 conda-forge
cuda-nvprof 12.4.99 h59595ed_0 conda-forge
cuda-nvprune 12.4.99 h59595ed_0 conda-forge
cuda-nvrtc 12.4.99 hd3aeb46_0 conda-forge
cuda-nvrtc-dev 12.4.99 hd3aeb46_0 conda-forge
cuda-nvrtc-static 12.4.99 hd3aeb46_0 conda-forge
cuda-nvtx 12.4.99 h59595ed_0 conda-forge
cuda-nvvm-dev_linux-64 12.4.99 ha770c72_0 conda-forge
cuda-nvvm-impl 12.4.99 h59595ed_0 conda-forge
cuda-nvvm-tools 12.4.99 h59595ed_0 conda-forge
cuda-nvvp 12.4.99 h59595ed_0 conda-forge
cuda-opencl 12.4.99 h59595ed_0 conda-forge
cuda-opencl-dev 12.4.99 h59595ed_0 conda-forge
cuda-profiler-api 12.4.99 ha770c72_0 conda-forge
cuda-python 12.4.0 pypi_0 pypi
cuda-sanitizer-api 12.4.99 hd3aeb46_0 conda-forge
cuda-toolkit 12.2.0 0 nvidia/label/cuda-12.2.0
cuda-tools 12.4.0 ha770c72_0 conda-forge
cuda-version 12.4 h3060b56_3 conda-forge
cuda-visual-tools 12.4.0 ha770c72_0 conda-forge
cudf-cu12 24.4.0a580 pypi_0 pypi
cuml-cu12 24.4.0a45 pypi_0 pypi
cuproj-cu12 24.4.0a28 pypi_0 pypi
cupy-cuda12x 13.0.0 pypi_0 pypi
cuspatial-cu12 24.4.0a28 pypi_0 pypi
cuxfilter-cu12 24.4.0a15 pypi_0 pypi
cxx-compiler 1.7.0 h00ab1b0_0 conda-forge
dask 2024.1.1 pypi_0 pypi
dask-cuda 24.4.0a12 pypi_0 pypi
dask-cudf-cu12 24.4.0a580 pypi_0 pypi
dask-expr 0.4.0 pypi_0 pypi
datashader 0.16.0 pypi_0 pypi
dbus 1.13.6 h5008d03_3 conda-forge
debugpy 1.8.1 py39h3d6467e_0 conda-forge
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
defusedxml 0.7.1 pypi_0 pypi
distributed 2024.1.1 pypi_0 pypi
exceptiongroup 1.2.0 pyhd8ed1ab_2 conda-forge
executing 2.0.1 pyhd8ed1ab_0 conda-forge
expat 2.6.2 h59595ed_0 conda-forge
fastjsonschema 2.19.1 pypi_0 pypi
fastrlock 0.8.2 pypi_0 pypi
fiona 1.9.6 pypi_0 pypi
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 h77eed37_1 conda-forge
fontconfig 2.14.2 h14ed4e7_0 conda-forge
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
fqdn 1.5.1 pypi_0 pypi
freetype 2.12.1 h267a509_2 conda-forge
frozenlist 1.4.1 pypi_0 pypi
fsspec 2024.3.1 pypi_0 pypi
gcc 12.3.0 h95e488c_3 conda-forge
gcc_impl_linux-64 12.3.0 he2b93b0_5 conda-forge
gcc_linux-64 12.3.0 h6477408_3 conda-forge
gds-tools 1.9.0.20 hd3aeb46_0 conda-forge
geopandas 0.14.3 pypi_0 pypi
gmp 6.3.0 h59595ed_1 conda-forge
gxx 12.3.0 h95e488c_3 conda-forge
gxx_impl_linux-64 12.3.0 he2b93b0_5 conda-forge
gxx_linux-64 12.3.0 h4a1b8e8_3 conda-forge
holoviews 1.18.3 pypi_0 pypi
icu 73.2 h59595ed_0 conda-forge
idna 3.6 pypi_0 pypi
importlib-metadata 7.0.2 pyha770c72_0 conda-forge
importlib_metadata 7.0.2 hd8ed1ab_0 conda-forge
ipykernel 6.29.3 pyhd33586a_0 conda-forge
ipython 8.18.1 pyh707e725_3 conda-forge
isoduration 20.11.0 pypi_0 pypi
jedi 0.19.1 pyhd8ed1ab_0 conda-forge
jinja2 3.1.3 pypi_0 pypi
joblib 1.3.2 pypi_0 pypi
jsonpointer 2.4 pypi_0 pypi
jsonschema 4.21.1 pypi_0 pypi
jsonschema-specifications 2023.12.1 pypi_0 pypi
jupyter-events 0.10.0 pypi_0 pypi
jupyter-server 2.13.0 pypi_0 pypi
jupyter-server-proxy 4.1.2 pypi_0 pypi
jupyter-server-terminals 0.5.3 pypi_0 pypi
jupyter_client 8.6.1 pyhd8ed1ab_0 conda-forge
jupyter_core 5.7.2 py39hf3d152e_0 conda-forge
jupyterlab-pygments 0.3.0 pypi_0 pypi
kernel-headers_linux-64 4.18.0 he073ed8_1 conda-forge
keyutils 1.6.1 h166bdaf_0 conda-forge
krb5 1.21.2 h659d440_0 conda-forge
ld_impl_linux-64 2.40 h41732ed_0 conda-forge
libcublas 12.4.2.65 hd3aeb46_0 conda-forge
libcublas-dev 12.4.2.65 hd3aeb46_0 conda-forge
libcublas-static 12.4.2.65 hd3aeb46_0 conda-forge
libcufft 11.2.0.44 hd3aeb46_0 conda-forge
libcufft-dev 11.2.0.44 hd3aeb46_0 conda-forge
libcufft-static 11.2.0.44 hd3aeb46_0 conda-forge
libcufile 1.9.0.20 hd3aeb46_0 conda-forge
libcufile-dev 1.9.0.20 hd3aeb46_0 conda-forge
libcufile-static 1.9.0.20 hd3aeb46_0 conda-forge
libcurand 10.3.5.119 hd3aeb46_0 conda-forge
libcurand-dev 10.3.5.119 hd3aeb46_0 conda-forge
libcurand-static 10.3.5.119 hd3aeb46_0 conda-forge
libcusolver 11.6.0.99 hd3aeb46_0 conda-forge
libcusolver-dev 11.6.0.99 hd3aeb46_0 conda-forge
libcusolver-static 11.6.0.99 hd3aeb46_0 conda-forge
libcusparse 12.3.0.142 hd3aeb46_0 conda-forge
libcusparse-dev 12.3.0.142 hd3aeb46_0 conda-forge
libcusparse-static 12.3.0.142 hd3aeb46_0 conda-forge
libedit 3.1.20191231 he28a2e2_2 conda-forge
libexpat 2.6.2 h59595ed_0 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libgcc-devel_linux-64 12.3.0 h8bca6fd_105 conda-forge
libgcc-ng 13.2.0 h807b86a_5 conda-forge
libglib 2.80.0 hf2295e7_0 conda-forge
libgomp 13.2.0 h807b86a_5 conda-forge
libiconv 1.17 hd590300_2 conda-forge
libnpp 12.2.5.2 hd3aeb46_0 conda-forge
libnpp-dev 12.2.5.2 hd3aeb46_0 conda-forge
libnpp-static 12.2.5.2 hd3aeb46_0 conda-forge
libnsl 2.0.1 hd590300_0 conda-forge
libnuma 2.0.18 hd590300_0 conda-forge
libnvfatbin 12.4.99 hd3aeb46_0 conda-forge
libnvfatbin-dev 12.4.99 hd3aeb46_0 conda-forge
libnvfatbin-static 12.4.99 hd3aeb46_0 conda-forge
libnvjitlink 12.4.99 hd3aeb46_0 conda-forge
libnvjitlink-dev 12.4.99 hd3aeb46_0 conda-forge
libnvjitlink-static 12.4.99 hd3aeb46_0 conda-forge
libnvjpeg 12.3.1.89 h59595ed_0 conda-forge
libnvjpeg-dev 12.3.1.89 ha770c72_0 conda-forge
libnvjpeg-static 12.3.1.89 ha770c72_0 conda-forge
libpng 1.6.43 h2797004_0 conda-forge
libsanitizer 12.3.0 h0f45ef3_5 conda-forge
libsodium 1.0.18 h36c2ea0_1 conda-forge
libsqlite 3.45.2 h2797004_0 conda-forge
libstdcxx-devel_linux-64 12.3.0 h8bca6fd_105 conda-forge
libstdcxx-ng 13.2.0 h7e041cc_5 conda-forge
libuuid 2.38.1 h0b41bf4_0 conda-forge
libxcb 1.15 h0b41bf4_0 conda-forge
libxcrypt 4.4.36 hd590300_1 conda-forge
libxkbcommon 1.6.0 hd429924_1 conda-forge
libxkbfile 1.1.0 h166bdaf_1 conda-forge
libxml2 2.12.6 h232c23b_0 conda-forge
libzlib 1.2.13 hd590300_5 conda-forge
linkify-it-py 2.0.3 pypi_0 pypi
llvmlite 0.42.0 pypi_0 pypi
locket 1.0.0 pypi_0 pypi
markdown 3.6 pypi_0 pypi
markdown-it-py 3.0.0 pypi_0 pypi
markupsafe 2.1.5 pypi_0 pypi
matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge
mdit-py-plugins 0.4.0 pypi_0 pypi
mdurl 0.1.2 pypi_0 pypi
mistune 3.0.2 pypi_0 pypi
msgpack 1.0.8 pypi_0 pypi
multidict 6.0.5 pypi_0 pypi
multipledispatch 1.0.0 pypi_0 pypi
nbclient 0.10.0 pypi_0 pypi
nbconvert 7.16.2 pypi_0 pypi
nbformat 5.10.3 pypi_0 pypi
ncurses 6.4.20240210 h59595ed_0 conda-forge
nest-asyncio 1.6.0 pyhd8ed1ab_0 conda-forge
nsight-compute 2024.1.0.13 hf972014_1 conda-forge
nspr 4.35 h27087fc_0 conda-forge
nss 3.98 h1d7d5a4_0 conda-forge
numba 0.59.1 pypi_0 pypi
numpy 1.26.4 pypi_0 pypi
nvtx 0.2.10 pypi_0 pypi
ocl-icd 2.3.2 hd590300_0 conda-forge
openssl 3.2.1 hd590300_1 conda-forge
overrides 7.7.0 pypi_0 pypi
packaging 24.0 pyhd8ed1ab_0 conda-forge
pandas 2.2.1 pypi_0 pypi
pandocfilters 1.5.1 pypi_0 pypi
panel 1.3.8 pypi_0 pypi
param 2.0.2 pypi_0 pypi
parso 0.8.3 pyhd8ed1ab_0 conda-forge
partd 1.4.1 pypi_0 pypi
patsy 0.5.6 pypi_0 pypi
pcre2 10.43 hcad00b1_0 conda-forge
pexpect 4.9.0 pyhd8ed1ab_0 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pillow 10.2.0 pypi_0 pypi
pip 24.0 pyhd8ed1ab_0 conda-forge
platformdirs 4.2.0 pyhd8ed1ab_0 conda-forge
prometheus-client 0.20.0 pypi_0 pypi
prompt-toolkit 3.0.42 pyha770c72_0 conda-forge
protobuf 4.25.3 pypi_0 pypi
psutil 5.9.8 py39hd1e30aa_0 conda-forge
pthread-stubs 0.4 h36c2ea0_1001 conda-forge
ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge
pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge
pyarrow 14.0.2 pypi_0 pypi
pycparser 2.21 pypi_0 pypi
pyct 0.5.0 pypi_0 pypi
pygments 2.17.2 pyhd8ed1ab_0 conda-forge
pylibraft-cu12 24.4.0a88 pypi_0 pypi
pynvjitlink-cu12 0.1.14 pypi_0 pypi
pynvml 11.4.1 pypi_0 pypi
pyproj 3.6.1 pypi_0 pypi
python 3.9.18 h0755675_1_cpython conda-forge
python-dateutil 2.9.0 pyhd8ed1ab_0 conda-forge
python-json-logger 2.0.7 pypi_0 pypi
python_abi 3.9 4_cp39 conda-forge
pytz 2024.1 pypi_0 pypi
pyviz-comms 3.0.1 pypi_0 pypi
pyyaml 6.0.1 pypi_0 pypi
pyzmq 25.1.2 py39h8c080ef_0 conda-forge
raft-dask-cu12 24.4.0a88 pypi_0 pypi
rapids-dask-dependency 24.4.0a7 pypi_0 pypi
readline 8.2 h8228510_1 conda-forge
referencing 0.34.0 pypi_0 pypi
requests 2.31.0 pypi_0 pypi
rfc3339-validator 0.1.4 pypi_0 pypi
rfc3986-validator 0.1.1 pypi_0 pypi
rich 13.7.1 pypi_0 pypi
rmm-cu12 24.4.0a38 pypi_0 pypi
rpds-py 0.18.0 pypi_0 pypi
scipy 1.12.0 pypi_0 pypi
send2trash 1.8.2 pypi_0 pypi
setuptools 69.2.0 pyhd8ed1ab_0 conda-forge
shapely 2.0.3 pypi_0 pypi
simpervisor 1.0.0 pypi_0 pypi
six 1.16.0 pyh6c4a22f_0 conda-forge
sniffio 1.3.1 pypi_0 pypi
sortedcontainers 2.4.0 pypi_0 pypi
soupsieve 2.5 pypi_0 pypi
stack_data 0.6.2 pyhd8ed1ab_0 conda-forge
statsmodels 0.14.1 pypi_0 pypi
sysroot_linux-64 2.28 he073ed8_1 conda-forge
tblib 3.0.0 pypi_0 pypi
terminado 0.18.1 pypi_0 pypi
tinycss2 1.2.1 pypi_0 pypi
tk 8.6.13 noxft_h4845f30_101 conda-forge
toolz 0.12.1 pypi_0 pypi
tornado 6.4 py39hd1e30aa_0 conda-forge
tqdm 4.66.2 pypi_0 pypi
traitlets 5.14.2 pyhd8ed1ab_0 conda-forge
treelite 4.0.0 pypi_0 pypi
types-python-dateutil 2.9.0.20240316 pypi_0 pypi
typing_extensions 4.10.0 pyha770c72_0 conda-forge
tzdata 2024.1 pypi_0 pypi
uc-micro-py 1.0.3 pypi_0 pypi
ucx-py-cu12 0.37.0a15 pypi_0 pypi
uri-template 1.3.0 pypi_0 pypi
urllib3 2.2.1 pypi_0 pypi
wayland 1.22.0 h8c25dac_1 conda-forge
wcwidth 0.2.13 pyhd8ed1ab_0 conda-forge
webcolors 1.13 pypi_0 pypi
webencodings 0.5.1 pypi_0 pypi
websocket-client 1.7.0 pypi_0 pypi
wheel 0.42.0 pyhd8ed1ab_0 conda-forge
xarray 2024.2.0 pypi_0 pypi
xcb-util 0.4.0 hd590300_1 conda-forge
xcb-util-cursor 0.1.4 hd590300_1 conda-forge
xcb-util-image 0.4.0 h8ee46fc_1 conda-forge
xcb-util-keysyms 0.4.0 h8ee46fc_1 conda-forge
xcb-util-renderutil 0.3.9 hd590300_1 conda-forge
xcb-util-wm 0.4.1 h8ee46fc_1 conda-forge
xkeyboard-config 2.41 hd590300_0 conda-forge
xorg-compositeproto 0.4.2 h7f98852_1001 conda-forge
xorg-damageproto 1.2.1 h7f98852_1002 conda-forge
xorg-fixesproto 5.0 h7f98852_1002 conda-forge
xorg-inputproto 2.3.2 h7f98852_1002 conda-forge
xorg-kbproto 1.0.7 h7f98852_1002 conda-forge
xorg-libice 1.1.1 hd590300_0 conda-forge
xorg-libsm 1.2.4 h7391055_0 conda-forge
xorg-libx11 1.8.7 h8ee46fc_0 conda-forge
xorg-libxau 1.0.11 hd590300_0 conda-forge
xorg-libxcomposite 0.4.6 h0b41bf4_1 conda-forge
xorg-libxdamage 1.1.5 h7f98852_1 conda-forge
xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
xorg-libxext 1.3.4 h0b41bf4_2 conda-forge
xorg-libxfixes 5.0.3 h7f98852_1004 conda-forge
xorg-libxi 1.7.10 h7f98852_0 conda-forge
xorg-libxrandr 1.5.2 h7f98852_1 conda-forge
xorg-libxrender 0.9.11 hd590300_0 conda-forge
xorg-libxtst 1.2.3 h7f98852_1002 conda-forge
xorg-randrproto 1.5.0 h7f98852_1001 conda-forge
xorg-recordproto 1.14.2 h7f98852_1002 conda-forge
xorg-renderproto 0.11.1 h7f98852_1002 conda-forge
xorg-util-macros 1.19.3 h7f98852_0 conda-forge
xorg-xextproto 7.3.0 h0b41bf4_1003 conda-forge
xorg-xproto 7.0.31 h7f98852_1007 conda-forge
xyzservices 2023.10.1 pypi_0 pypi
xz 5.2.6 h166bdaf_0 conda-forge
yarl 1.9.4 pypi_0 pypi
zeromq 4.3.5 h59595ed_1 conda-forge
zict 3.0.0 pypi_0 pypi
zipp 3.17.0 pyhd8ed1ab_0 conda-forge
After several attempts, I determined that the problem was that there were some problems with auto-arima's search. Even for the same sequence, different best model search results may appear under different batch combinations. Here is a minimal example.
ts = [
[128, 88, 228, 357, 376, 448, 558, 521, 395, 211, 294, 253, 314, 272, 314, 295, 432, 565, 583, 473, 343, 195, 200, 255, 371, 189, 269, 272, 352, 374, 433, 296, 182, 187, 136, 82, 113, 105, 217, 203, 416, 504, 488, 519, 424, 263, 263, 220, 237, 160],
[214, 1351, 965, 848, 362, 273, 176, 187, 163, 201, 207, 173, 239, 137, 233, 231, 248, 160, 200, 222, 160,219, 173, 170, 225, 212, 216, 279, 197, 174, 163, 184, 149, 169, 178, 181, 212, 177, 172, 224, 234, 225, 204, 165, 178, 163, 140, 138, 125, 175],
]
ys = cupy.array(ts)
ys = ys.T
model = AutoARIMA(ys)
model.search(s = 0, d=(0, 1, 2), p=range(7), q=range(4), method="auto")
model.fit()
model.forecast(2)
ts = [
[214, 1351, 965, 848, 362, 273, 176, 187, 163, 201, 207, 173, 239, 137, 233, 231, 248, 160, 200, 222, 160,219, 173, 170, 225, 212, 216, 279, 197, 174, 163, 184, 149, 169, 178, 181, 212, 177, 172, 224, 234, 225, 204, 165, 178, 163, 140, 138, 125, 175],
]
ys = cupy.array(ts)
ys = ys.T
model = AutoARIMA(ys)
model.search(s = 0, d=(0, 1, 2), p=range(7), q=range(4), method="auto")
model.fit()
model.forecast(2)
@Nyrio I noticed that you are the main contributor of this code, could you provide some help and mark this as a feature or I can also try to fix this bug.
This problem is further confirmed to be that when ARIMA(method='ml'), the fitting results of batch input and individual input are different, but method='css' does not have this problem.
@Apolsus thanks for the issue and reproducer! Would using the css method suffice for you for now? We will look into the bug in ml, but not sure of an ETA for a fix
no, css method will also cause this in a much larger batchsize (10000). I checked the source code. In theory, the parameter optimization process is performed independently for each sequence, but some sequences do not converge in batch prediction, but converge in individual prediction. I think one possible place is the parameter initialization process, I haven't had time to check yet.
After looking into it (and with help from @Nyrio ) this seems to stem from numerical stability issues particularly around different code paths for different batch sizes. It might take some time to create workarounds or fixes in general, but we will try to look into it as soon as we can.