qsharp icon indicating copy to clipboard operation
qsharp copied to clipboard

Reduce kata validation test time

Open billti opened this issue 2 years ago • 2 comments
trafficstars

Some of these take multiple seconds, which is way to long for unit tests. We need to figure out how to get these to sub-second times.

For example, from latest CI on main, these 2 tests alone take close to 10 seconds.

# Subtest: random_numbers kata is valid
ok 14 - random_numbers kata is valid
  ---
  duration_ms: 6979.4028
  ...
# Subtest: oracles kata is valid
ok 15 - oracles kata is valid
  ---
  duration_ms: 2711.1274

billti avatar Oct 04 '23 19:10 billti

@billti @cesarzc I wonder if we can just add these to the integration test suite?

minestarks avatar Oct 12 '23 15:10 minestarks

I'm not bothered by the time it takes to validate the katas in CI builds (this will only grow as we move from 10 katas to 40), but when I work on the katas locally (and build and run tests for them using "python build.py --npm --play --no-check"), I end up removing most of the existing katas from the index.json so as to test only the one or two that I work on right now. It would be good to be able to have a flag to specify "build, test, and include in the playground only kata X".

tcNickolas avatar Apr 19 '24 19:04 tcNickolas