quilc icon indicating copy to clipboard operation
quilc copied to clipboard

Study non-deterministic tests that exceed thresholds often

Open jmbr opened this issue 4 years ago • 0 comments

@ecpeterson points out that: "test-compiler-hook and test-compiler-hook-random-4q are the two typical culprits, usually in their second halves where state preparation methods are turned on."

It would be desirable to inspect the aforementioned tests and isolate the conditions that trigger abnormal results. Once isolated, we could have two subsets of test inputs:

  1. regular ones, for which the tolerance is low and
  2. harder ones, for which we must increase the tolerance.

In the best case scenario, pinpointing the hard cases would also give us some idea of how to address them with high precision.

jmbr avatar Jan 16 '20 21:01 jmbr