Oliver Schulz

Results 665 comments of Oliver Schulz

I should probably also add support for Functors.jl to ArraysOfArrays.jl, to enable `fmap` and `functor`.

> I don't see advantage of for table in partition(ttree) for analysis because [...] It'll still be nice to have it, since it gives users more power to optimize the...

> When I tried to do Arrow.write() with the partitions, I was eating up all the RAM on my machine Hm, when the writes are done within the ```for tbl...

If we implement `Tables.partitions` (with a mechanism that robustly chooses a "good" partition size), we could actually use that internally in the loop-implementation over whole trees. Could make things nicely...

> isn't "processing by chunk" a workaround to the slow-for-loop problem in Python? No, we'd still loop over the content of the chunks Julia-fashion, etc. This is not about Python-style...

> The iterate(tree) loop doesn't need to partition across all branches thus doesn't have this problem. each branch is individually accessed with whatever event number we're at. Ah, right. Ok,...

It kinda depends on what's done per entry, I would say - if the operation per entry is somewhat expensive, iterating over the tree row-by-row will be great. For use...

> relatively cheap, we do want to offer user-accessible columnar memory-layout. Let's say (just an example from my own practice) that I want to do signal processing, signals and other...

So I do want to have that double loop (and the inner loop may or may not be offloaded to a different computing hardware, even).

> Btw there's no "unsynced" ever, in looping Ah - with "synced" I meant chunks that cover the same events for each branch, in comparison to the internal chunks that...