FLAMEGPU2 icon indicating copy to clipboard operation
FLAMEGPU2 copied to clipboard

Improved Testing for RNG

Open ptheywood opened this issue 4 years ago • 2 comments

@Robadob has potentially found (and fixed?) an issue with the range of random values produced by AgentRandom::uniform.

The test case for this is a little weak (i.e. it doesn't test for uniformity, or the bounds.

Tests need strengthenign to assert that the generated distribution of numbers is aproximately uniform within the expected range, and never generates out of range values.

  1. Seed with a sane seed (or do this for multiple seeds)
  2. Generate lots and lots of random values
  3. Ensure all values are in range (max/min reductions could be used)
    • Ideally also check min/max expected value is produce, although this will potentially require an incredibly large sample size for floating point values, or large-range integers. Limiting the range of values to be relatively small should (i.e. will be much more likely) to produce values of min and max.
  4. Generata a histogram, and assert that the values are (approximately) uniform.
    • This should only be important for the any methods we have implemented scaling of the value, and otherwise trust the underlying RNG methods which we have little control over.
  5. Ensure that all values

Edit - todo list

  • [ ] Test(s) for inclusivity of floating point rng (inclusive one end, preferably 0 <= x < 1 to match stdlib, if the extra 1.f- is not too expensive ).
  • [ ] Test(s) for inclusivty of integer ranges (inlcusive both ends)
  • [ ] Test distribution of integer range uniformity, as this is not just provided by curand (although relies on curand providing uniform distribution)
  • No need to test distribution of floats, as we just have to trust curand.

ptheywood avatar Jun 30 '20 13:06 ptheywood

Branch: int_rng_fix

I won't address this further until SubModels is merged, as SubModels removes some deprecated stuff from the associated test file.

Robadob avatar Jun 30 '20 13:06 Robadob

I wrote a standalone test for the uniform float implementation now present in master (after #411). Might want some sanity checking though.

The code can be found here: https://gist.github.com/Robadob/5e3a068267f8a351991bef16d138dd7c

To my surprise it executes instantly for me locally, producing the below output.

Found 256 zeros
Found 0 ones
Min Bin Size 4194304
Max Bin Size 4194304

Hence, when mapping every generated float value to one of 1024 bins, the bins are uniformly filled.

Similar could be carried out for the uniform double implementation.

Robadob avatar Feb 10 '23 18:02 Robadob