scenery icon indicating copy to clipboard operation
scenery copied to clipboard

Make examples more deterministic for automated testing

Open skalarproduktraum opened this issue 3 years ago • 3 comments

We are currently using the examples as integration tests as well. Some of them use randomised values, e.g. for lighting. As the screenshots created from these examples are automatically compared via Argos CI (see e.g. https://www.argos-ci.com/scenerygraphics/scenery/builds/38), this throws up quite a few false positives.

Ideas to fix this:

  • use a fixed random seed in the example runs on CI to produce deterministic values
  • don't use random values at all

skalarproduktraum avatar Dec 16 '21 09:12 skalarproduktraum

Could you provide some link where those values are exactly set?

elect86 avatar Dec 16 '21 09:12 elect86

An example would be https://github.com/scenerygraphics/scenery/blob/108ba6affef799633400ffb3e6080f685fdf8b75/src/test/kotlin/graphics/scenery/tests/examples/advanced/RibbonExampleSecondaryStructures.kt#L29

There's plenty of places in the examples where Random.random3DVectorFromRange() is used e.g. for light colors.

skalarproduktraum avatar Dec 16 '21 13:12 skalarproduktraum

We might generate a static random palette offline and fetch during runtime from that

Or, we generate the palette at the start. If on CI, the palette will be always the same

elect86 avatar Dec 16 '21 13:12 elect86