Sorting output for tests

[michaelmior]

I have a program that produces a number of lines of output that do not need to be in any particular order. I suppose I could add an option to my program to sort output and then use that in my tests, but in my case, that would require quite a bit of restructuring. It would be nice if there was a way to have trycmd sort lines of output before comparing.

[epage]

Yes, we've been considering something along these lines.

One of our benchmarks for trycmd/snapbox is if it is powerful enough to replace cargo's home grown testing tools. Cargo has non-deterministic output and they have matchers to help with that. We haven't fully settled on how we want to handle it.

One overall challenge with this is not limiting ourselves to output being single-line only. I'd like a more general solution but we'll fall back to that assumption if needed

[epage]

With #291, we can have callers control when sorting should be done or not

[epage]

insta has the concept of sorted redactions. The concept works well for some forms of structured data, it has a downside when it comes to unordered messages (lines of text, jsonlines). Switching cargo to this would be a meaningful reduction in functionality. It could also be complicated when dealing with ... lines as we'd want to make sure that they get swallowed up, regardless of order.

I also find the name odd. It does not redact data but filter data. It also declares the operation to be performed by Cargo describes the property of the data (unordered).

What we're really doing both with our redaction globs and this is patterning the actual off of the expected.