Flowise
Flowise copied to clipboard
Refactor for loops
Refactor some of the for loops into maps, including other code simplifications
Results: Document Loaders:
- Cheerio: Pass
going to put this on hold first until we have less PRs
@burn2delete I was just having a glance at the PR code - because so many (20) files were altered - and I wonder if there are some maps happening now in your PR where, in fact, you want to keep the sequential dependence the slower for loop may provide.
In particular, and I may be getting this wrong because I was just glancing at the code, but you seem to be using map when assembling previous chat messages at one point into a context to be provided again. It could be argued that the sequence may not matter, but I think the stronger argument may be that the sequence COULD matter whereas the speedup in those instances is marginal? What do you think? I'm all for unrolling and parallelising, but if one simply does a search for all 'for' loops this doesn't filter if some of them are implicitly handling dependent sequences which shouldn't be parallelised and I don't know if you applied such checking given you did so many updates?
I realise this PR is almost a year old, so maybe things have overtaken this, but in case it matters for your own fork or if indeed this PR may yet get incorporated, I thought I should raise the issue before any merging.
I think would be better close this PR because it is very old and doesn't make sense.