FsToolkit.ErrorHandling icon indicating copy to clipboard operation
FsToolkit.ErrorHandling copied to clipboard

List.traverseResultA and friends have x @ y on the error branch, this could potentially blow up

Open abelbraaksma opened this issue 3 years ago • 1 comments

Describe the bug The List.traverseResultA function and similar ones have this critical path:

            match state, fR with
            | Ok ys, Ok y -> traverseResultA' (Ok(y :: ys)) f xs
            | Error errs, Error e -> traverseResultA' (Error(errs @ e)) f xs
            | Ok _, Error e
            | Error e, Ok _ -> traverseResultA' (Error e) f xs

Namely: Error(errs @ e).

If you have many errors (in my situation is was importing an Excel file and all cells in a column were bad), the performance is exponential.

To solve this, one way would be a simple fix to just e :: errs the errors and do a List.rev at the end.

abelbraaksma avatar May 05 '22 11:05 abelbraaksma

Ouch, yeah that's bad. I mentioned the error path needs to be considered as well in https://github.com/demystifyfp/FsToolkit.ErrorHandling/issues/167 when doing a performance pass at it.

TheAngryByrd avatar May 05 '22 14:05 TheAngryByrd

I looked into this briefly and errs @ e are two lists so your recommended fix wouldn't work exactly. So I think this would be better changed to be some type of while loop semantics to get performance.

TheAngryByrd avatar Oct 18 '22 20:10 TheAngryByrd

Oh, yeah, you're right. Though, arguably, the new lists will be shorter than the tally, so a quick rec with e::errs would be nice. But that would, perhaps, invert the order of errors? Hmm, tricky. Yet, such loops are quite common, and with a List.rev at the end, should work.

abelbraaksma avatar Oct 18 '22 20:10 abelbraaksma

I did benchmark this a bit ago: https://github.com/demystifyfp/FsToolkit.ErrorHandling/issues/167, various tradeoffs

Yeah idk why it's currently implemented with the @ after looking at it now.

TheAngryByrd avatar Oct 18 '22 21:10 TheAngryByrd

Change looks good, thanks!

abelbraaksma avatar Oct 20 '22 20:10 abelbraaksma