HotShot icon indicating copy to clipboard operation
HotShot copied to clipboard

[CX_HARDENING] - Develop A Better Framework for Input Permutation

Open jparr721 opened this issue 9 months ago • 0 comments

Closes #3161

This PR:

Inputs are mostly written for the happy path. The existing tests are written with a single stream of inputs, and this has bitten us in the past. However, as we see in the proposal_ordering tests, the permutation with index order is not sufficient and requires the implementor to arduously call a significant number of cases with manually prescribed indices. We can do better. We need a scalable and reliable way to permute inputs (and outputs) in a way that ensures that all possible orderings can be handled by the test.

This PR does just that, it adds a randomization layer over the inputs and auto-injects a nested modification of the all predicate to make the outputs ignore the ordering.

This PR also removes the uses of permute_input_with_index_order and the helpers file entirely.

This PR does not:

Key places to review:

jparr721 avatar May 22 '24 17:05 jparr721