[Bug] sample_polytope step size is correlated with direction when number of samples is small

[sbernasek-albert]

🐛 Bug

In the botorch.utils.sampling.sample_polytope() function the same seed is used to generate both the random step sizes and the random step directions. For very small numbers of steps (e.g. n=1), this causes the the step size and step direction to be strongly correlated, yielding strong sampling bias.

This is irrelevant for those using the optimize_acqf procedure that leverages large batches of hit-and-run samples generated in parallel. However, for custom optimization routines that require sequential hit-and-run samples, it may be necessary to call sample_polytope(..., n=1).

Looping through many such sequential calls yields this correlation between step size (rands) and step direction (Rs):

A super simple fix would be to add 1 to the seed used to generate a step direction, e.g. sample_hypersphere(..., seed=seed+1) . This eliminates any correlation between step-size and step-direction.

[saitcakmak]

Hi @sbernasek-albert. Apologies for the late response. The proposed fix seems quite reasonable. We'd be happy to accept a PR implementing the fix.

[Balandat]

Fixed by #2290