Jesse Wright

@rubensworks I'd suspect you're correct in that the intermediary buffers are the main overhead. I was thinking that one way around this is to have a 'chained transform' which applies...

@jacoscaz - no I have not done profiling, it was just an observation I made when digging through the code when working on a PR I made earlier

> Would this not require knowing what those transformations look like ahead of time? I was thinking that methods like `map`, `filter` and `transform` would add to the internal set...

Because of [this](https://github.com/RubenVerborgh/AsyncIterator/blob/7a089e36e8c33c35b27b34cd670518f2fa51acce/asynciterator.ts#L446-L534) the use of `.map`, `.filter`, `.transform` etc. *all* create a new `SimpleTransformIterator` and hence there are many places in Comunica that create new `SimpleTransformIterator`s for instance Mapping:...

To summarize the results I have in the following comments 1. `ArrayIterator` causes about a 5x speed degradation compared to `range` iterator on 20000 elements (because as I raised [here](https://bugs.chromium.org/p/v8/issues/detail?id=12730)...

With the PR (https://github.com/RubenVerborgh/AsyncIterator/pull/45) I have made I get; for the `ArrayIterator` method ``` ArrayIterator#map(): 5532 MappingIterator: 4770 ``` and for the `range` method I get ``` ArrayIterator#map(): 944 MappingIterator:...

I've also implemented a `FilterIterator` against my PR (below), for 50 iterations on `range` the results are ```ts ArrayIterator#filter(): 72 FilterIterator: 13 ``` ```ts class FilterIterator extends AsyncIterator { constructor(source:...

Sorry to keep spamming this thread - I've also implemented a `CompositeIterator` and it also shows non-trivial performance gains relative to just repeatedly applying the `MapIterator` and `FliterIterator` (~10-15% improvement)...

You can see a further 50-100% performance improvement by 'pre-compiling' the `CompositeTransform` - see https://gist.github.com/jeswr/f49a6975f1bfc088a9c32565e69761bf

> `RangeIterator` that appears to run 500x faster than `ArrayIterator` Impressive - and I can't see any problems with it; though I wonder why it also seems to be a...