HotShot
HotShot copied to clipboard
[CX_HARDENING] - Develop A Better Framework for Input Permutation
Closes #3161
This PR:
Inputs are mostly written for the happy path. The existing tests are written with a single stream of inputs, and this has bitten us in the past. However, as we see in the proposal_ordering tests, the permutation with index order is not sufficient and requires the implementor to arduously call a significant number of cases with manually prescribed indices. We can do better. We need a scalable and reliable way to permute inputs (and outputs) in a way that ensures that all possible orderings can be handled by the test.
This PR does just that, it adds a randomization layer over the inputs and auto-injects a nested modification of the all
predicate to make the outputs ignore the ordering.
This PR also removes the uses of permute_input_with_index_order
and the helpers file entirely.