orion icon indicating copy to clipboard operation
orion copied to clipboard

Add tests for handling of colliding trials in algo test suite

Open bouthilx opened this issue 2 years ago • 1 comments

With a low precision for instance, algos are likely to try to suggest trials that are rounded to the same values. We should have tests in the test-suite to verify that algos handle this properly. For instance, if a wrapper gets two trials rounding to the same values, the underlying algo may wait for results for both although only one rounded version will be executed. We need to make sure algos handle this properly by returned the result to all corresponding similar trials.

bouthilx avatar Feb 18 '22 17:02 bouthilx

This would be easier to test when #802 is done. For instance if we have an internal storage of trials mapping original IDs to transformed IDs, then we could test that duplicates are added to the list that is assigned to the original ID, and we could test that observing the original trial with a result assigns the result to all corresponding transformed trial.

bouthilx avatar Feb 24 '22 21:02 bouthilx