Stim icon indicating copy to clipboard operation
Stim copied to clipboard

Doube check that depolarize2 error rate statistical test failure on macos was likely a false positive

Open Strilanc opened this issue 3 years ago • 5 comments

E AssertionError: Sample rate 0.0108 is over 5 standard deviations away from 0.006666665896773338.

This is presumably a one in a million failure. But the test has been run a lot of times and there are a lot of similar tests. Just compute the probability of seeing this sort of thing by chance and eyeball if it's reasonable.

Note that this occurred very recently after a change to how seeding is done.

work/Stim/Stim/src/stim/circuit/circuit_gate_target_pybind_test.py ..... [  1%]
  ....                                                                     [  1%]
  work/Stim/Stim/src/stim/circuit/circuit_instruction_pybind_test.py ..... [  2%]
                                                                           [  2%]
  work/Stim/Stim/src/stim/circuit/circuit_pybind_test.py ................. [  6%]
  ...............                                                          [  9%]
  work/Stim/Stim/src/stim/circuit/circuit_repeat_block_test.py ...         [ 10%]
  work/Stim/Stim/src/stim/dem/detector_error_model_instruction_pybind_test.py . [ 10%]
  .......                                                                  [ 11%]
  work/Stim/Stim/src/stim/dem/detector_error_model_pybind_test.py ........ [ 13%]
  .....                                                                    [ 14%]
  work/Stim/Stim/src/stim/dem/detector_error_model_repeat_block_pybind_test.py . [ 14%]
  ..                                                                       [ 14%]
  work/Stim/Stim/src/stim/dem/detector_error_model_target_pybind_test.py . [ 15%]
  .....                                                                    [ 16%]
  work/Stim/Stim/src/stim/py/compiled_detector_sampler_pybind_test.py .    [ 16%]
  work/Stim/Stim/src/stim/py/compiled_measurement_sampler_pybind_test.py . [ 16%]
  ....                                                                     [ 17%]
  work/Stim/Stim/src/stim/py/stim_pybind_test.py ....                      [ 18%]
  work/Stim/Stim/src/stim/simulators/matched_error_pybind_test.py .......  [ 19%]
  work/Stim/Stim/src/stim/simulators/measurements_to_detection_events_test.py . [ 19%]
  .                                                                        [ 20%]
  work/Stim/Stim/src/stim/simulators/tableau_simulator_pybind_test.py .... [ 20%]
  ....................................                                     [ 28%]
  work/Stim/Stim/src/stim/stabilizers/pauli_string_pybind_test.py ........ [ 29%]
  ...............                                                          [ 33%]
  work/Stim/Stim/src/stim/stabilizers/tableau_pybind_test.py ............. [ 35%]
  .........                                                                [ 37%]
  work/Stim/Stim/glue/cirq/stimcirq/_cirq_to_stim_test.py ................ [ 40%]
  ........................................................................ [ 55%]
  .........................................................F.............. [ 70%]
  ............                                                             [ 72%]
  work/Stim/Stim/glue/cirq/stimcirq/_det_annotation_test.py ......         [ 74%]
  work/Stim/Stim/glue/cirq/stimcirq/_measure_and_or_reset_gate_test.py ..  [ 74%]
  work/Stim/Stim/glue/cirq/stimcirq/_obs_annotation_test.py ......         [ 75%]
  work/Stim/Stim/glue/cirq/stimcirq/_shift_coords_annotation_test.py ..... [ 76%]
  .                                                                        [ 77%]
  work/Stim/Stim/glue/cirq/stimcirq/_stim_sampler_test.py .....            [ 78%]
  work/Stim/Stim/glue/cirq/stimcirq/_stim_to_cirq_test.py ................ [ 81%]
  ........................................................................ [ 96%]
  ...........                                                              [ 98%]
  work/Stim/Stim/glue/cirq/stimcirq/_sweep_pauli_test.py ....              [ 99%]
  work/Stim/Stim/glue/cirq/stimcirq/_two_qubit_asymmetric_depolarize_test.py . [ 99%]
  ...                                                                      [100%]
  
  =================================== FAILURES ===================================
  __ test_tableau_simulator_sampling_noisy_gates_agrees_with_cirq_data[gate14] ___
  
  gate = cirq.depolarize(p=0.1,n_qubits=2)
  
      @pytest.mark.parametrize("gate", ROUND_TRIP_NOISY_GATES)
      def test_tableau_simulator_sampling_noisy_gates_agrees_with_cirq_data(gate: cirq.Gate):
          # Technically this be a test of the `stim` package itself, but it's so convenient to compare to cirq.
      
          # Create test circuit that uses superdense coding to quantify arbitrary Pauli error mixtures.
          n = cirq.num_qubits(gate)
          qs = cirq.LineQubit.range(n)
          circuit = cirq.Circuit(
              cirq.H.on_each(qs),
              [cirq.CNOT(q, q + n) for q in qs],
              gate(*qs),
              [cirq.CNOT(q, q + n) for q in qs],
              cirq.H.on_each(qs),
          )
          expected_rates = cirq.final_density_matrix(circuit).diagonal().real
      
          # Convert test circuit to Stim and sample from it.
          stim_circuit, _ = cirq_circuit_to_stim_data(
              circuit + cirq.measure(*sorted(circuit.all_qubits())[::-1])
          )
          sample_count = 10000
          samples = []
          for _ in range(sample_count):
              sim = stim.TableauSimulator()
              sim.do(stim_circuit)
              s = 0
              for k, v in enumerate(sim.current_measurement_record()):
                  s |= v << k
              samples.append(s)
      
          unique, counts = np.unique(samples, return_counts=True)
      
          # Compare sample rates to expected rates.
          for value, count in zip(unique, counts):
              expected_rate = expected_rates[value]
              actual_rate = count / sample_count
              allowed_variation = 5 * (expected_rate * (1 - expected_rate) / sample_count) ** 0.5
              if not 0 <= expected_rate - allowed_variation <= 1:
                  raise ValueError("Not enough samples to bound results away from extremes.")
  >           assert abs(expected_rate - actual_rate) < allowed_variation, (
                  f"Sample rate {actual_rate} is over 5 standard deviations away from {expected_rate}.\n"
                  f"Gate: {gate}\n"
                  f"Test circuit:\n{circuit}\n"
                  f"Converted circuit:\n{stim_circuit}\n"
              )
  E           AssertionError: Sample rate 0.0108 is over 5 standard deviations away from 0.006666665896773338.
  E             Gate: depolarize(p=0.1,n_qubits=2)
  E             Test circuit:
  E                       ┌──┐            ┌──┐
  E             0: ───H────@─────D(0.1)────@─────H───
  E                        │     │         │
  E             1: ───H────┼@────#2────────┼@────H───
  E                        ││              ││
  E             2: ────────X┼──────────────X┼────────
  E                         │               │
  E             3: ─────────X───────────────X────────
  E                       └──┘            └──┘
  E             Converted circuit:
  E             H 0 1
  E             TICK
  E             CX 0 2 1 3
  E             TICK
  E             DEPOLARIZE2(0.1) 0 1
  E             TICK
  E             CX 0 2 1 3
  E             TICK
  E             H 0 1
  E             TICK
  E             M 3 2 1 0
  E             TICK
  E             
  E           assert 0.004133334103226662 < 0.004068851638544387
  E            +  where 0.004133334103226662 = abs((0.006666666 - 0.0108))
  
  work/Stim/Stim/glue/cirq/stimcirq/_cirq_to_stim_test.py:235: AssertionError
  =========================== short test summary info ============================
  FAILED work/Stim/Stim/glue/cirq/stimcirq/_cirq_to_stim_test.py::test_tableau_simulator_sampling_noisy_gates_agrees_with_cirq_data[gate14]
  =================== 1 failed, 486 passed in 72.11s (0:01:12) ===================
Error: Command  pytest /Users/runner/work/Stim/Stim/src /Users/runner/work/Stim/Stim/glue/cirq failed with code 1. None

                                                            ✕ 122.06s
Error: Process completed with exit code 1.

Strilanc avatar Mar 24 '22 14:03 Strilanc

Saw this again https://github.com/quantumlib/Stim/runs/6332576101?check_suite_focus=true

============================= test session starts ==============================
  platform darwin -- Python 3.9.12, pytest-7.1.2, pluggy-1.0.0
  rootdir: /Users/runner/work/Stim/Stim
  collected 491 items
  
  work/Stim/Stim/src/stim/circuit/circuit_gate_target_pybind_test.py ..... [  1%]
  ....                                                                     [  1%]
  work/Stim/Stim/src/stim/circuit/circuit_instruction_pybind_test.py ..... [  2%]
                                                                           [  2%]
  work/Stim/Stim/src/stim/circuit/circuit_pybind_test.py ................. [  6%]
  ..................                                                       [  9%]
  work/Stim/Stim/src/stim/circuit/circuit_repeat_block_test.py ...         [ 10%]
  work/Stim/Stim/src/stim/dem/detector_error_model_instruction_pybind_test.py . [ 10%]
  .......                                                                  [ 12%]
  work/Stim/Stim/src/stim/dem/detector_error_model_pybind_test.py ........ [ 13%]
  .....                                                                    [ 14%]
  work/Stim/Stim/src/stim/dem/detector_error_model_repeat_block_pybind_test.py . [ 15%]
  ..                                                                       [ 15%]
  work/Stim/Stim/src/stim/dem/detector_error_model_target_pybind_test.py . [ 15%]
  .....                                                                    [ 16%]
  work/Stim/Stim/src/stim/py/compiled_detector_sampler_pybind_test.py .    [ 16%]
  work/Stim/Stim/src/stim/py/compiled_measurement_sampler_pybind_test.py . [ 17%]
  ....                                                                     [ 17%]
  work/Stim/Stim/src/stim/py/stim_pybind_test.py ....                      [ 18%]
  work/Stim/Stim/src/stim/simulators/matched_error_pybind_test.py .......  [ 20%]
  work/Stim/Stim/src/stim/simulators/measurements_to_detection_events_test.py . [ 20%]
  .                                                                        [ 20%]
  work/Stim/Stim/src/stim/simulators/tableau_simulator_pybind_test.py .... [ 21%]
  ....................................                                     [ 28%]
  work/Stim/Stim/src/stim/stabilizers/pauli_string_pybind_test.py ........ [ 30%]
  ...............                                                          [ 33%]
  work/Stim/Stim/src/stim/stabilizers/tableau_pybind_test.py ............. [ 36%]
  .........                                                                [ 37%]
  work/Stim/Stim/glue/cirq/stimcirq/_cirq_to_stim_test.py ................ [ 41%]
  ........................................................................ [ 55%]
  ........................................................................ [ 70%]
  .......F.....                                                            [ 73%]
  work/Stim/Stim/glue/cirq/stimcirq/_det_annotation_test.py ......         [ 74%]
  work/Stim/Stim/glue/cirq/stimcirq/_measure_and_or_reset_gate_test.py ..  [ 74%]
  work/Stim/Stim/glue/cirq/stimcirq/_obs_annotation_test.py ......         [ 75%]
  work/Stim/Stim/glue/cirq/stimcirq/_shift_coords_annotation_test.py ..... [ 76%]
  .                                                                        [ 77%]
  work/Stim/Stim/glue/cirq/stimcirq/_stim_sampler_test.py .....            [ 78%]
  work/Stim/Stim/glue/cirq/stimcirq/_stim_to_cirq_test.py ................ [ 81%]
  ........................................................................ [ 96%]
  ...........                                                              [ 98%]
  work/Stim/Stim/glue/cirq/stimcirq/_sweep_pauli_test.py ....              [ 99%]
  work/Stim/Stim/glue/cirq/stimcirq/_two_qubit_asymmetric_depolarize_test.py . [ 99%]
  ...                                                                      [100%]
  
  =================================== FAILURES ===================================
  __ test_tableau_simulator_sampling_noisy_gates_agrees_with_cirq_data[gate36] ___
  
  gate = stimcirq.TwoQubitAsymmetricDepolarizingChannel((0.0033333333333333335, 0.006666666666666667, 0.01, 0.01333333333333333...66666666666667, 0.03, 0.03333333333333333, 0.03666666666666667, 0.04, 0.043333333333333335, 0.04666666666666667, 0.05))
  
      @pytest.mark.parametrize("gate", ROUND_TRIP_NOISY_GATES)
      def test_tableau_simulator_sampling_noisy_gates_agrees_with_cirq_data(gate: cirq.Gate):
          # Technically this be a test of the `stim` package itself, but it's so convenient to compare to cirq.
      
          # Create test circuit that uses superdense coding to quantify arbitrary Pauli error mixtures.
          n = cirq.num_qubits(gate)
          qs = cirq.LineQubit.range(n)
          circuit = cirq.Circuit(
              cirq.H.on_each(qs),
              [cirq.CNOT(q, q + n) for q in qs],
              gate(*qs),
              [cirq.CNOT(q, q + n) for q in qs],
              cirq.H.on_each(qs),
          )
          expected_rates = cirq.final_density_matrix(circuit).diagonal().real
      
          # Convert test circuit to Stim and sample from it.
          stim_circuit, _ = cirq_circuit_to_stim_data(
              circuit + cirq.measure(*sorted(circuit.all_qubits())[::-1])
          )
          sample_count = 10000
          samples = []
          for _ in range(sample_count):
              sim = stim.TableauSimulator()
              sim.do(stim_circuit)
              s = 0
              for k, v in enumerate(sim.current_measurement_record()):
                  s |= v << k
              samples.append(s)
      
          unique, counts = np.unique(samples, return_counts=True)
      
          # Compare sample rates to expected rates.
          for value, count in zip(unique, counts):
              expected_rate = expected_rates[value]
              actual_rate = count / sample_count
              allowed_variation = 5 * (expected_rate * (1 - expected_rate) / sample_count) ** 0.5
              if not 0 <= expected_rate - allowed_variation <= 1:
                  raise ValueError("Not enough samples to bound results away from extremes.")
  >           assert abs(expected_rate - actual_rate) < allowed_variation, (
                  f"Sample rate {actual_rate} is over 5 standard deviations away from {expected_rate}.\n"
                  f"Gate: {gate}\n"
                  f"Test circuit:\n{circuit}\n"
                  f"Converted circuit:\n{stim_circuit}\n"
              )
  E           AssertionError: Sample rate 0.0386 is over 5 standard deviations away from 0.0299999937415123.
  E             Gate: stimcirq.TwoQubitAsymmetricDepolarizingChannel((0.0033333333333333335, 0.006666666666666667, 0.01, 0.013333333333333334, 0.016666666666666666, 0.02, 0.023333333333333334, 0.02666666666666667, 0.03, 0.03333333333333333, 0.03666666666666667, 0.04, 0.043333333333333335, 0.04666666666666667, 0.05))
  E             Test circuit:
  E                       ┌──┐                                                                                                                                                                    ┌──┐
  E             0: ───H────@─────PauliMix(II:0.6,IX:0.00333,IY:0.00667,IZ:0.01,XI:0.0133,XX:0.0167,XY:0.02,XZ:0.0233,YI:0.0267,YX:0.03,YY:0.0333,YZ:0.0367,ZI:0.04,ZX:0.0433,ZY:0.0467,ZZ:0.05)────@─────H───
  E                        │     │                                                                                                                                                                 │
  E             1: ───H────┼@────#2────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼@────H───
  E                        ││                                                                                                                                                                      ││
  E             2: ────────X┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────X┼────────
  E                         │                                                                                                                                                                       │
  E             3: ─────────X───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────X────────
  E                       └──┘                                                                                                                                                                    └──┘
  E             Converted circuit:
  E             H 0 1
  E             TICK
  E             CX 0 2 1 3
  E             TICK
  E             PAULI_CHANNEL_2(0.00333333, 0.00666667, 0.01, 0.0133333, 0.0166667, 0.02, 0.0233333, 0.0266667, 0.03, 0.0333333, 0.0366667, 0.04, 0.0433333, 0.0466667, 0.05) 0 1
  E             TICK
  E             CX 0 2 1 3
  E             TICK
  E             H 0 1
  E             TICK
  E             M 3 2 1 0
  E             TICK
  E             
  E           assert 0.008600006258487704 < 0.00852936019[245](https://github.com/quantumlib/Stim/runs/6332576101?check_suite_focus=true#step:10:245)018
  E            +  where 0.008600006[258](https://github.com/quantumlib/Stim/runs/6332576101?check_suite_focus=true#step:10:258)487704 = abs((0.0[299](https://github.com/quantumlib/Stim/runs/6332576101?check_suite_focus=true#step:10:299)99994 - 0.0386))
  
  work/Stim/Stim/glue/cirq/stimcirq/_cirq_to_stim_test.py:235: AssertionError
  =============================== warnings summary ===============================

Strilanc avatar May 07 '22 06:05 Strilanc

I think, both times, this failed on osx. This most recent one was OSX. If it happens again make sure to document.

image

Strilanc avatar May 07 '22 06:05 Strilanc

Saw one on windows this time:

================================== FAILURES ===================================
  __ test_tableau_simulator_sampling_noisy_gates_agrees_with_cirq_data[gate36] __
  
  gate = stimcirq.TwoQubitAsymmetricDepolarizingChannel((0.0033333333333333335, 0.006666666666666667, 0.01, 0.01333333333333333...66666666666667, 0.03, 0.03333333333333333, 0.03666666666666667, 0.04, 0.043333333333333335, 0.04666666666666667, 0.05))
  
      @pytest.mark.parametrize("gate", ROUND_TRIP_NOISY_GATES)
      def test_tableau_simulator_sampling_noisy_gates_agrees_with_cirq_data(gate: cirq.Gate):
          # Technically this be a test of the `stim` package itself, but it's so convenient to compare to cirq.
      
          # Create test circuit that uses superdense coding to quantify arbitrary Pauli error mixtures.
          n = cirq.num_qubits(gate)
          qs = cirq.LineQubit.range(n)
          circuit = cirq.Circuit(
              cirq.H.on_each(qs),
              [cirq.CNOT(q, q + n) for q in qs],
              gate(*qs),
              [cirq.CNOT(q, q + n) for q in qs],
              cirq.H.on_each(qs),
          )
          expected_rates = cirq.final_density_matrix(circuit).diagonal().real
      
          # Convert test circuit to Stim and sample from it.
          stim_circuit, _ = cirq_circuit_to_stim_data(
              circuit + cirq.measure(*sorted(circuit.all_qubits())[::-1])
          )
          sample_count = 10000
          samples = []
          for _ in range(sample_count):
              sim = stim.TableauSimulator()
              sim.do(stim_circuit)
              s = 0
              for k, v in enumerate(sim.current_measurement_record()):
                  s |= v << k
              samples.append(s)
      
          unique, counts = np.unique(samples, return_counts=True)
      
          # Compare sample rates to expected rates.
          for value, count in zip(unique, counts):
              expected_rate = expected_rates[value]
              actual_rate = count / sample_count
              allowed_variation = 5 * (expected_rate * (1 - expected_rate) / sample_count) ** 0.5
              if not 0 <= expected_rate - allowed_variation <= 1:
                  raise ValueError("Not enough samples to bound results away from extremes.")
  >           assert abs(expected_rate - actual_rate) < allowed_variation, (
                  f"Sample rate {actual_rate} is over 5 standard deviations away from {expected_rate}.\n"
                  f"Gate: {gate}\n"
                  f"Test circuit:\n{circuit}\n"
                  f"Converted circuit:\n{stim_circuit}\n"
              )
  E           AssertionError: Sample rate 0.0266 is over 5 standard deviations away from 0.03666665777564049.
  E             Gate: stimcirq.TwoQubitAsymmetricDepolarizingChannel((0.00[333](https://github.com/quantumlib/Stim/runs/6641808548?check_suite_focus=true#step:10:338)33333333333335, 0.006666666666666667, 0.01, 0.013333333333333[334](https://github.com/quantumlib/Stim/runs/6641808548?check_suite_focus=true#step:10:339), 0.016666666666666666, 0.02, 0.023333333333333334, 0.02666666666666667, 0.03, 0.03333333333333333, 0.03666666666666667, 0.04, 0.043333333333333[335](https://github.com/quantumlib/Stim/runs/6641808548?check_suite_focus=true#step:10:340), 0.04666666666666667, 0.05))
  E             Test circuit:
  E                       \u250c\u2500\u2500\u2510                                                                                                                                                                    \u250c\u2500\u2500\u2510
  E             0: \u2500\u2500\u2500H\u2500\u2500\u2500\u2500@\u2500\u2500\u2500\u2500\u2500PauliMix(II:0.6,IX:0.00333,IY:0.00667,IZ:0.01,XI:0.0133,XX:0.0167,XY:0.02,XZ:0.0233,YI:0.0267,YX:0.03,YY:0.0333,YZ:0.0367,ZI:0.04,ZX:0.0433,ZY:0.0467,ZZ:0.05)\u2500\u2500\u2500\u2500@\u2500\u2500\u2500\u2500\u2500H\u2500\u2500\u2500
  E                        \u2502     \u2502                                                                                                                                                                 \u2502
  E             1: \u2500\u2500\u2500H\u2500\u2500\u2500\u2500\u253c@\u2500\u2500\u2500\u2500#2\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c@\u2500\u2500\u2500\u2500H\u2500\u2500\u2500
  E                        \u2502\u2502                                                                                                                                                                      \u2502\u2502
  E             2: \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500X\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500X\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500
  E                         \u2502                                                                                                                                                                       \u2502
  E             3: \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500X\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500X\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500
  E                       \u2514\u2500\u2500\u2518                                                                                                                                                                    \u2514\u2500\u2500\u2518
  E             Converted circuit:
  E             H 0 1
  E             TICK
  E             CX 0 2 1 3
  E             TICK
  E             PAULI_CHANNEL_2(0.00333333, 0.00666667, 0.01, 0.0133333, 0.0166667, 0.02, 0.0233333, 0.0266667, 0.03, 0.0333333, 0.0[366](https://github.com/quantumlib/Stim/runs/6641808548?check_suite_focus=true#step:10:371)667, 0.04, 0.0433333, 0.0466667, 0.05) 0 1
  E             TICK
  E             CX 0 2 1 3
  E             TICK
  E             H 0 1
  E             TICK
  E             M 3 2 1 0
  E             TICK
  E             
  E           assert 0.010066657775640489 < 0.009397102476721821
  E            +  where 0.010066657775640489 = abs((0.036666658 - 0.0266))
  
  D:\a\Stim\Stim\glue\cirq\stimcirq\_cirq_to_stim_test.py:235: AssertionError
  ============================== warnings summary ===============================

Strilanc avatar May 29 '22 10:05 Strilanc

Was the source of this ever pinpointed? I am really glad that this is documented regardless of the source

ChrisPattison avatar Aug 16 '22 19:08 ChrisPattison

@ChrisPattison No.

I suspect it's just that the tests have tolerances that are too tight, but I'm not closing the issue until I actually do the math and work out the expected false positive rate.

Strilanc avatar Aug 16 '22 20:08 Strilanc