Francesc Alted

icon indicating a website link https://ironarray.io

icon company @FrancescAlted icon location Castelló de la Plana, Spain icon location CEO of https://ironarray.io Project leader of the Blosc project. Compression and efficient data I/O to efficient storage and computation.

Results 320 comments of Francesc Alted

Improvements to C code that can improve performance

Hi. Yeah, bcolz vendors pretty old versions of ztsd (and other codecs) essentially because it is unmaintained. For more info: https://github.com/Blosc/bcolz#unmaintained-package-notice

introduction of an abstraction layer for the "results array"

I need to think a bit more on this, but at least initially this seems like a good idea to me.

introduction of an abstraction layer for the "results array"

I think that this would definitely be a nice addition. We still need tests and update the documentation so that user "may bring their own implementation". @ARF1 could you please...

ctable.todataframe() does not save dataframe index values

Yes, the index is a special column that is not represented in ctable containers, so roundtrips are currently not possible. Perhaps adding some special attribute to designate the index column...

pandas out_flavor for ctable

Would you mind to add some benchmarks in the 'bench/' directory showing the advantage of this approach? My idea is to setup a speed regression check based on different benchmarks...

index_as_columns option in ctable.fromdataframe()

Looks good to me; I have added a couple of comments though. Also, a couple of test units would be needed. Thanks!

question: is there a way to do in-place arithmetic operations?

Well, I have added the capability (see 24c4d5c) to preserve the cparams during `my_ctable['f0'] = x + 1`, so you don't strictly need to use `.eval()` (if you have an...

question: is there a way to do in-place arithmetic operations?

Here you are supposing that compression/decompression is a very expensive operation, and it is not so, at least on modern CPUs with several cores. Also, bcolz is highly optimized for...

Consider adding a Categorical datatype to bcolz

Hi all, Thanks for the detailed discussions. I am not able to dig into this a lot now, but I am planning to put more time on this in the...

Document using nosetests

I think that is a great idea. Maybe you could take work in PR #227 and finish it.