mkl_fft
mkl_fft copied to clipboard
Limitation of: Data size of one of transform dimensions exceeds 2^31 - 1 bytes
Based on the error message, it seems like this is a known limitation, but I'm resampling data using ffts and get an error related to the size of the data. The data shape is: (17863680, 128) and the fft is being taken along axis=0.
I'm guessing this might be related to an index being int32, and if it is possible to get rid of this limitation, that would be helpful.
File "/home/jlivezey/process_nwb/process_nwb/fft.py", line 22, in rfft
return mklrfft(*args, **kwargs)
File "/clusterfs/bebb/users/jlivezey/anaconda3/envs/nsds_nwb/lib/python3.7/site-packages/mkl_fft/_numpy_fft.py", line 414, in rfft
(x,), {'n': n, 'axis': axis})
File "/clusterfs/bebb/users/jlivezey/anaconda3/envs/nsds_nwb/lib/python3.7/site-packages/mkl_fft/_numpy_fft.py", line 103, in trycall
raise ve
File "/clusterfs/bebb/users/jlivezey/anaconda3/envs/nsds_nwb/lib/python3.7/site-packages/mkl_fft/_numpy_fft.py", line 98, in trycall
res = func(*args, **kwrds)
File "mkl_fft/_pydfti.pyx", line 792, in mkl_fft._pydfti.rfft_numpy
File "mkl_fft/_pydfti.pyx", line 701, in mkl_fft._pydfti._rc_fft1d_impl
ValueError: Internal error occurred: b'Intel MKL DFTI ERROR: Data size of one of transform dimensions exceeds 2^31 - 1 bytes'