bcolz icon indicating copy to clipboard operation
bcolz copied to clipboard

Runtime and Memory errors when moving from python 2.7 to python 3.6

Open iNLyze opened this issue 7 years ago • 0 comments

Recently, I migrated my python environment to 3.6 and updated bcolz along with it. Now I have trouble opening bcolz containers I created with the python 2.7 environment. I get Exception ignored in: 'bcolz.carray_ext.chunk._getitem' RuntimeError: fatal error during Blosc decompression: -1 and the data I get is all zeros. I noticed that a lot of changes have been made to the compression algorithms which you call with the "cname" argument. I used to use with bcolz.defaults_ctx(vm='python', cparams=bcolz.cparams(clevel=5, cname='zstd')): These settings are apparently not offered any more accourding to the manual on page 11. Would it be possible to include the old algorithms for backward compatibility?

Moreover, I noticed that the same data which used to fit into my RAM just about now raises a memory error. This could be related to changing the compression algorithms, too. However, I am a bit at a loss on when to use which algorithm (cname) with which compression level (clevel). It would be very helpful if you could publish comprehensive benchmarks or guidelines on when to use which setting.

For example, which settings would be the closest match to the above ones?

iNLyze avatar Mar 28 '17 10:03 iNLyze