orion
orion copied to clipboard
Add tests for handling of colliding trials in algo test suite
With a low precision for instance, algos are likely to try to suggest trials that are rounded to the same values. We should have tests in the test-suite to verify that algos handle this properly. For instance, if a wrapper gets two trials rounding to the same values, the underlying algo may wait for results for both although only one rounded version will be executed. We need to make sure algos handle this properly by returned the result to all corresponding similar trials.
This would be easier to test when #802 is done. For instance if we have an internal storage of trials mapping original IDs to transformed IDs, then we could test that duplicates are added to the list that is assigned to the original ID, and we could test that observing the original trial with a result assigns the result to all corresponding transformed trial.