bcolz icon indicating copy to clipboard operation
bcolz copied to clipboard

introduction of an abstraction layer for the "results array"

Open ARF1 opened this issue 10 years ago • 3 comments

Implicitly closes #66 and #176 by allowing users to "bring their own" implementations. Would benefit from #189.

This introduces an abstraction layer for the generation of the "results array" in ctable.__getitem__() and for data access.

It has minor impact on core bcolz code and minimizes overhead using lookup optimizations:

Without abstraction layer:

In [3]: a = bcolz.open(rootdir='testdata.bcolz')

In [4]: %timeit a[1]
10000 loops, best of 3: 38.5 µs per loop

With abstraction layer:

In [3]: a = bcolz.open(rootdir='testdata.bcolz')

In [4]: %timeit a[1]
10000 loops, best of 3: 42.3 µs per loop

ARF1 avatar May 05 '15 14:05 ARF1

I need to think a bit more on this, but at least initially this seems like a good idea to me.

FrancescAlted avatar May 05 '15 17:05 FrancescAlted

Tests?

esc avatar May 23 '15 04:05 esc

I think that this would definitely be a nice addition. We still need tests and update the documentation so that user "may bring their own implementation". @ARF1 could you please do that?

FrancescAlted avatar Feb 01 '17 08:02 FrancescAlted