Mar 30 '24 23:03 Fe-r-oz

improved SRM(1,r)

Outputs:

julia> parity_checks(SteaneReedMuller(1,3))
+ XXXXXXXX
+ ZZZZZZZZ
+ __ZZXXYY
+ _ZXY_ZXY
+ _XZYZY_X

julia> parity_checks(SteaneReedMuller(1,4))
+ XXXXXXXXXXXXXXXX
+ ZZZZZZZZZZZZZZZZ
+ ____ZZZZXXXXYYYY
+ __ZZXXYY__ZZXXYY
+ _ZXY_ZXY_ZXY_ZXY
+ _X_XZYZYZYZY_X_X

julia> parity_checks(SteaneReedMuller(1,5))
+ XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
+ ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ
+ ________ZZZZZZZZXXXXXXXXYYYYYYYY
+ ____ZZZZXXXXYYYY____ZZZZXXXXYYYY
+ __ZZXXYY__ZZXXYY__ZZXXYY__ZZXXYY
+ _ZXY_ZXY_ZXY_ZXY_ZXY_ZXY_ZXY_ZXY
+ _X_X_X_XZYZYZYZYZYZYZYZY_X_X_X_X
canonicalize:

julia> canonicalize!(parity_checks(SteaneReedMuller(1,3)))
+ XZ_YZXY_
+ _XZYZY_X
+ _ZXY_ZXY
+ __ZZXXYY
+ ZZZZZZZZ

julia> canonicalize!(parity_checks(Gottesman(3)))
+ XZ_Y_YXZ
+ _XZY_XZY
- _ZXYZ_YX
- __ZZYYXX
+ ZZZZZZZZ

Had to close #249 because of the merge conflicts introduced by #250.

Gottesman subset is being tests with official Gottesman test.

Mar 30 '24 23:03 Fe-r-oz

It was told that the Stabilizer can accept two bool arguments, no need to pass ;UINt8 argument, but the following happens when testing for large stabilizer circuits;

Stabilizer(extended_Hx, extended_Hz) TypeError: in typeassert, expected Matrix{UInt64}, got a value of type Vector{Any}

SteaneReedMuller(1,3) to (1,7) don't have this error.

Mar 30 '24 23:03 Fe-r-oz

It was told that the Stabilizer can accept two bool arguments, no need to pass ;UINt8 argument, but the following happens when testing for large stabilizer circuits;

Stabilizer(extended_Hx, extended_Hz) TypeError: in typeassert, expected Matrix{UInt64}, got a value of type Vector{Any}

SteaneReedMuller(1,3) to (1,7) don't have this error.

An easy way to check what constructors are available is:

julia> methods(Stabilizer)
# 10 methods for type constructor:
  [1] Stabilizer(s::Stabilizer)
  [2] Stabilizer(xzs::AbstractMatrix{Bool})
  [3] Stabilizer(phases::Tzv, nqubits::Int64, xzs::Tm) where {Tzv<:AbstractVector{UInt8}, Tm<:(AbstractMatrix{<:Unsigned})}
  [4] Stabilizer(paulis::AbstractArray{PauliOperator{Tz, Tv}, 1}) where {Tz, Tv}
  [5] Stabilizer(phases::AbstractVector{UInt8}, xs::AbstractMatrix{Bool}, zs::AbstractMatrix{Bool})
  [6] Stabilizer(phases::AbstractVector{UInt8}, xzs::AbstractMatrix{Bool})
  [7] Stabilizer(xs::AbstractMatrix{Bool}, zs::AbstractMatrix{Bool})
  [8] Stabilizer(tab::T) where T<:QuantumClifford.Tableau
  [9] Stabilizer(g::Graphs.SimpleGraphs.SimpleGraph)
 [10] Stabilizer(c::QuantumClifford.ECC.AbstractECC)

I think you have been using method 5, but method 7 should be more convenient. Given your error message, it seems the problem is that somewhere a Vector{Any} is being fed instead of a Matrix{Bool}. More information is necessary to pinpoint the exact issue.

Also, it is worrisome that this happens only for some sizes. It implies there might be some size-dependent type-instability, which is usually a bad sign.

Mar 31 '24 00:03 Krastanov

The Hx and Hz are both Boolean Matrices before they are passed down to the Stabilizer.

I will check out how the vector{any} comes into play. Also, I will introspect on the tip: size-dependent type-instability,

Mar 31 '24 00:03 Fe-r-oz

On the master branch tests pass. You can disregard nightly/performance/downgrade/buildkite, but the rest should pass here. Let's keep this in draft form until the tests pass.

Mar 31 '24 01:03 Krastanov

To run the tests locally, you can do ] test QuantumClifford. Avoid pushing here until things work locally.

Mar 31 '24 01:03 Krastanov

julia> naive_encoding_circuit(parity_checks(SteaneReedMuller(3,6)))
690-element Vector{QuantumClifford.AbstractOperation}:
 QuantumClifford.sHadamard(1)
 QuantumClifford.sZCX(1,43)
 QuantumClifford.sZCX(1,44)
 QuantumClifford.sZCX(1,45)
 QuantumClifford.sZCX(1,46)
 QuantumClifford.sZCX(1,47)
 QuantumClifford.sZCX(1,49)
 QuantumClifford.sZCX(1,50)
 QuantumClifford.sZCX(1,51)
 QuantumClifford.sZCX(1,52)
 QuantumClifford.sZCX(1,54)
 QuantumClifford.sZCX(1,55)
 QuantumClifford.sZCX(1,56)
 QuantumClifford.sZCX(1,58)
 QuantumClifford.sZCX(1,59)
 QuantumClifford.sZCX(1,61)
 QuantumClifford.sHadamard(2)
 ⋮
 QuantumClifford.sSWAP(33,27)
 QuantumClifford.sSWAP(32,28)
 QuantumClifford.sSWAP(31,29)
 QuantumClifford.sSWAP(30,28)
 QuantumClifford.sSWAP(29,26)
 QuantumClifford.sSWAP(28,26)
 QuantumClifford.sSWAP(27,25)
 QuantumClifford.sSWAP(26,24)
 QuantumClifford.sSWAP(25,23)
 QuantumClifford.sSWAP(24,23)
 QuantumClifford.sSWAP(23,22)
 QuantumClifford.sSWAP(22,21)
 QuantumClifford.sSWAP(21,20)
 QuantumClifford.sSWAP(20,19)
 QuantumClifford.sSWAP(19,18)
 QuantumClifford.sSWAP(18,17)
 QuantumClifford.sSWAP(17,16)

julia> naive_encoding_circuit(parity_checks(SteaneReedMuller(3,4)))
127-element Vector{QuantumClifford.AbstractOperation}:
 QuantumClifford.sHadamard(1)
 QuantumClifford.sPhase(1)
 QuantumClifford.sZCX(1,16)
 QuantumClifford.sHadamard(2)
 QuantumClifford.sZCX(2,16)
 QuantumClifford.sHadamard(3)
 QuantumClifford.sZCZ(3,2)
 QuantumClifford.sZCX(3,16)
 QuantumClifford.sHadamard(4)
 QuantumClifford.sZCX(4,16)
 QuantumClifford.sHadamard(5)
 QuantumClifford.sPhase(5)
 QuantumClifford.sZCZ(5,3)
 QuantumClifford.sZCX(5,16)
 QuantumClifford.sHadamard(6)
 QuantumClifford.sPhase(6)
 QuantumClifford.sZCZ(6,4)
 ⋮
 QuantumClifford.sZCZ(15,8)
 QuantumClifford.sZCZ(15,9)
 QuantumClifford.sZCZ(15,10)
 QuantumClifford.sZCZ(15,11)
 QuantumClifford.sZCZ(15,12)
 QuantumClifford.sZCZ(15,13)
 QuantumClifford.sZCX(15,16)
 QuantumClifford.sZ(1)
 QuantumClifford.sZ(2)
 QuantumClifford.sZ(3)
 QuantumClifford.sZ(4)
 QuantumClifford.sZ(5)
 QuantumClifford.sZ(6)
 QuantumClifford.sZ(7)
 QuantumClifford.sZ(9)
 QuantumClifford.sZ(11)
 QuantumClifford.sZ(13)


julia> naive_encoding_circuit(parity_checks(SteaneReedMuller(5,6)))
1732-element Vector{QuantumClifford.AbstractOperation}:
 QuantumClifford.sHadamard(1)
 QuantumClifford.sPhase(1)
 QuantumClifford.sZCX(1,64)
 QuantumClifford.sHadamard(2)
 QuantumClifford.sZCX(2,64)
 QuantumClifford.sHadamard(3)
 QuantumClifford.sZCZ(3,2)
 QuantumClifford.sZCX(3,64)
 QuantumClifford.sHadamard(4)
 QuantumClifford.sZCX(4,64)
 QuantumClifford.sHadamard(5)
 QuantumClifford.sZCZ(5,3)
 QuantumClifford.sZCX(5,64)
 QuantumClifford.sHadamard(6)
 QuantumClifford.sZCZ(6,4)
 QuantumClifford.sZCX(6,64)
 QuantumClifford.sHadamard(7)
 ⋮
 QuantumClifford.sZ(36)
 QuantumClifford.sZ(37)
 QuantumClifford.sZ(38)
 QuantumClifford.sZ(39)
 QuantumClifford.sZ(41)
 QuantumClifford.sZ(42)
 QuantumClifford.sZ(44)
 QuantumClifford.sZ(46)
 QuantumClifford.sZ(47)
 QuantumClifford.sZ(49)
 QuantumClifford.sZ(50)
 QuantumClifford.sZ(51)
 QuantumClifford.sZ(54)
 QuantumClifford.sZ(55)
 QuantumClifford.sZ(57)
 QuantumClifford.sZ(59)
 QuantumClifford.sZ(61)

Mar 31 '24 12:03 Fe-r-oz

code_k(c::....) = ....will give 0 at (4, 2) based on Eq 2. Steane calls it ''code size falls to 0.'' Not sure how the base test perceives k =0 for frame or other calculations that require k.

I've come to appreciate that even my mistakes uncover loopholes in the tests. The base test is more comprehensive now as a result.

code_k(c::....) = ....will give 0 at (4, 2) based on Eq 2. Steane calls it ''code size falls to 0.'' Not sure how the base test perceives k =0 for frame or other calculations that require k.

As per Steane's paper, I have made sure that when the k falls to 0, it should be a throw as Steane point that the method does not work in these cases. Kindly please confirm this as well from the paper.

n + k

elseif _nplusk_qrm(t, r) == 0
             throw(ArgumentError("Invalid parameters when attempting to construct a Steane-Reed-Muller code. The method to construct Steane-Reed-Muller code fails when k is 0."))

This helps to identify the following cases:

julia> _nplusk_qrm(3, 2)
0

julia> _nplusk_qrm(4, 2)
0

julia> _nplusk_qrm(4, 1)
0

julia> _nplusk_qrm(4, 3)
0


julia> _nplusk_qrm(5, 1)
0

julia> _nplusk_qrm(5, 2)
0

julia> _nplusk_qrm(5, 3)
0

julia> _nplusk_qrm(5, 4)
0

julia> _nplusk_qrm(6, 1)
0

julia> _nplusk_qrm(6, 2)
0

julia> _nplusk_qrm(6, 3)
0

julia> _nplusk_qrm(6, 4)
0

julia> _nplusk_qrm(6, 5)
0

Edit: Actually, I wanted to quote this as a new comment by >, but ended up editing the same comment instead. In the edit, I have added the cases where k falls to zero and that should be thrown when the SteaneReedMuller runs.

Apr 01 '24 04:04 Fe-r-oz

I don't know precisely the definition of code_s(c::...) so I didn't included it. Maybe it's total number of syndromes bits?

I have included code_k, code_n. Distance is 2^c.t + 2^(c.t-1) as the improved base requires them for calculations of frames

If there are improvements that can be made, if further attention is required in tests, etc. please let me know.

Apr 01 '24 10:04 Fe-r-oz

Maybe it's better to include a sentence in docstring why Generator matrics are actually stabilizers meaning same as parity check tableaus for some values, this could really prevent confusion if people see parity check Tableau and ask "Why is formed in the structure of Generator Matrixs which is G = H as follows

 [ G1| 0 ]  
 [ 0| G2 ]
 [ Dx|Dz ]

G = H (in some cases) After reading the paper, the reason is that

"The method works for k > 0, and the generators for k < 0 are stabilisers of k > 0 codes."

It would be helpful if permission is granted to add this line in the docstring.

Apr 01 '24 13:04 Fe-r-oz

I don't know precisely the definition of code_s(c::...) so I didn't included it. Maybe it's total number of syndromes bits?

As per def. of code_s in CSS: code_s(c::CSS) = size(c.Hx, 1) + size(c.Hz, 1)

I guess code_s is total rows of stabilizer since in CSS, we have different rows for Hz and Hx. This can be seen in different examples of CSS.

In Steane's case, I have something like

code_s(c::...) = _nplusk(c.t, c.r)

since that's tells us total number of rows in this case.

Edit:

I don't think code_s(c::...) = _nplusk(c.t, c.r) is correct guess.

Apr 01 '24 14:04 Fe-r-oz

More information is necessary to pinpoint the exact issue. [5] Stabilizer(phases::AbstractVector{UInt8}, xs::AbstractMatrix{Bool}, zs::AbstractMatrix{Bool}) [7] Stabilizer(xs::AbstractMatrix{Bool}, zs::AbstractMatrix{Bool})

Very sorry for not providing complete details. My bad.

Problem was in the whennaive_encoding_circuit(which starts from line 18 to line 53), Maybe because in [5], we provide a parameter phases, (what is phased btw?) and in [7], we don't, so maybe it expects a unit8 phase parameter?

Complete appropriate trace: I have added (please see here in the trace). This is from where we see the source innaive_encoding_circuit

 BoundsError: attempt to access 16-element view(::Vector{UInt8}, 17:32) with eltype UInt8 at index [17]
  Stacktrace:
    [1] throw_boundserror(A::SubArray{UInt8, 1, Vector{UInt8}, Tuple{UnitRange{Int64}}, true}, I::Tuple{Int64})
      @ Base ./abstractarray.jl:737
    [2] checkbounds
      @ ./abstractarray.jl:702 [inlined]
    [3] getindex
      @ ./subarray.jl:322 [inlined]
    [4] naive_encoding_circuit(code::QuantumClifford.ECC.SteaneReedMuller; undoperm::Bool)
      @ QuantumClifford.ECC ~/Desktop/New/b10/QuantumClifford.jl/src/ecc/circuits.jl:53 (please see here)
    [5] naive_encoding_circuit
      @ ~/Desktop/New/b10/QuantumClifford.jl/src/ecc/circuits.jl:18 [inlined] (please see here)
    [6] test_with_pframes(code::QuantumClifford.ECC.SteaneReedMuller)
      @ Main.var"##ecc#256" ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:47

Apr 01 '24 14:04 Fe-r-oz

It might help to run the other ECC tests as well, to see what fails. The one about encoding&syndrome circuits is probably too involved, with too many pieces that work together, so it is difficult to debug.

What other tests fail? In particular, what are the test_ecc_codeproperties results.

It is indeed quite probably that the issue is the k=0 codes. If it happens only for them, we should: (1) understand what specifically about the assumptions in the tests is wrong and (2) filter out these codes only for these specific tests.

Apr 01 '24 20:04 Krastanov

"The method works for k > 0, and the generators for k < 0 are stabilisers of k > 0 codes."

It would be helpful if permission is granted to add this line in the docstring.

I would need to understand better what that means, but yeah, go ahead and add it. I will ask more questions when I get to reviewing this.

Apr 01 '24 20:04 Krastanov

What other tests fail? In particular, what are the test_ecc_codeproperties results.

Every other test seem to work well. It's seems to me that it's the k = 0, and the way the code_base is not to be setup does provides error when k = 0.

Please have a look at CI tests. They provide the same error that is locally.


 Testing Running tests...
Starting tests with 1 threads out of `Sys.CPU_THREADS = 4`...
skipping gpu tests (set GPU_TESTS=true to test gpu)
Test Summary: | Pass  Total  Time
throws        |  101    101  4.3s
Test Summary: | Pass  Total  Time
paulis        |  451    451  4.9s
Test Summary: | Pass  Total  Time
stabs         |   95     95  8.9s
Test Summary: | Pass  Total  Time
stabcanon     |  226    226  3.7s
Test Summary: | Pass  Total  Time
mul_leftright | 1260   1260  3.0s
Test Summary: | Pass  Total  Time
inner         |    9      9  0.6s
Test Summary: | Pass  Total  Time
embed         |    5      5  0.0s
Test Summary: | Pass  Total  Time
gf2           |   90     90  2.2s
Test Summary: | Pass  Total   Time
projections   |  379    379  10.6s
Test Summary: | Pass  Total  Time
expect        |    7      7  0.1s
Test Summary: | Pass  Total  Time
trace         |  286    286  3.7s
Test Summary: | Pass  Total  Time
cliff         |  197    197  6.6s
Test Summary: | Pass  Total  Time
symcliff      |  329    329  4.3s
Test Summary: | Pass  Total   Time
symcontrolled |  203    203  12.4s
Test Summary: | Pass  Total  Time
classicalreg  |    8      8  1.3s
Test Summary: | Pass  Total   Time
random        |  192    192  16.6s
Test Summary: | Pass  Total  Time
noisycircuits |   91     91  5.6s
Test Summary: | Pass  Total  Time
syndromemeas  |  100    100  0.2s
Test Summary: | Pass  Total  Time
bitpack       |   96     96  7.5s
Test Summary: | Pass  Total  Time
memorylayout  |   40     40  7.4s
Test Summary: | Pass  Total  Time
graphs        |   22     22  0.6s
Test Summary: | Pass  Total  Time
hash          |    4      4  0.2s
Test Summary: | Pass  Total  Time
entanglement  |   57     57  3.6s
Test Summary: | Pass  Total  Time
enumerate     |    5      5  2.1s
Test Summary: | Pass  Total   Time
quantumoptics |  346    346  15.9s
naive syndrome circuits - zero syndrome for logical states: Error During Test at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:37
  Got exception outside of a @test
  ArgumentError: dimension size must be ≥ 0, got -10 for dimension 1
  Stacktrace:
    [1] BitArray
      @ ./bitarray.jl:32 [inlined]
    [2] BitArray
      @ ./bitarray.jl:71 [inlined]
    [3] falses(dims::Tuple{Int64})
      @ Base ./bitarray.jl:405
    [4] falses
      @ ./bitarray.jl:403 [inlined]
    [5] quantum_mallows(rng::Random._GLOBAL_RNG, n::Int64)
      @ QuantumClifford ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:189
    [6] random_destabilizer(rng::Random._GLOBAL_RNG, n::Int64; phases::Bool)
      @ QuantumClifford ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:73
    [7] random_destabilizer
      @ ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:72 [inlined]
    [8] random_stabilizer(rng::Random._GLOBAL_RNG, n::Int64; phases::Bool)
      @ QuantumClifford ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:166
    [9] random_stabilizer
      @ ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:166 [inlined]
   [10] random_stabilizer
      @ ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:167 [inlined]
   [11] test_naive_syndrome(c::QuantumClifford.ECC.SteaneReedMuller, e::Bool)
      @ Main.var"##ecc#256" ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:12
   [12] macro expansion
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:39 [inlined]
   [13] macro expansion
      @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
   [14] top-level scope
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:38
   [15] include(mod::Module, _path::String)
      @ Base ./Base.jl:495
   [16] include(x::String)
      @ Main.var"##ecc#256" ~/.julia/packages/SafeTestsets/raUNr/src/SafeTestsets.jl:28
   [17] macro expansion
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/runtests.jl:20 [inlined]
   [18] macro expansion
      @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
   [19] macro expansion
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/runtests.jl:20 [inlined]
   [20] top-level scope
      @ ~/.julia/packages/SafeTestsets/raUNr/src/SafeTestsets.jl:30
   [21] eval(m::Module, e::Any)
      @ Core ./boot.jl:385
   [22] macro expansion
      @ ~/.julia/packages/SafeTestsets/raUNr/src/SafeTestsets.jl:28 [inlined]
   [23] top-level scope
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/runtests.jl:19
   [24] include(fname::String)
      @ Base.MainInclude ./client.jl:489
   [25] top-level scope
      @ none:6
   [26] eval
      @ ./boot.jl:385 [inlined]
   [27] exec_options(opts::Base.JLOptions)
      @ Base ./client.jl:291
   [28] _start()
      @ Base ./client.jl:552
naive syndrome circuits - zero syndrome for logical states: Error During Test at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:59
  Got exception outside of a @test
  BoundsError: attempt to access 16-element view(::Vector{UInt8}, 17:32) with eltype UInt8 at index [17]
  Stacktrace:
    [1] throw_boundserror(A::SubArray{UInt8, 1, Vector{UInt8}, Tuple{UnitRange{Int64}}, true}, I::Tuple{Int64})
      @ Base ./abstractarray.jl:737
    [2] checkbounds
      @ ./abstractarray.jl:702 [inlined]
    [3] getindex
      @ ./subarray.jl:322 [inlined]
    [4] naive_encoding_circuit(code::QuantumClifford.ECC.SteaneReedMuller; undoperm::Bool)
      @ QuantumClifford.ECC ~/Desktop/New/b10/QuantumClifford.jl/src/ecc/circuits.jl:53
    [5] naive_encoding_circuit
      @ ~/Desktop/New/b10/QuantumClifford.jl/src/ecc/circuits.jl:18 [inlined]
    [6] test_with_pframes(code::QuantumClifford.ECC.SteaneReedMuller)
      @ Main.var"##ecc#256" ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:47
    [7] macro expansion
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:61 [inlined]
    [8] macro expansion
      @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
    [9] top-level scope
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:60
   [10] include(mod::Module, _path::String)
      @ Base ./Base.jl:495
   [11] include(x::String)
      @ Main.var"##ecc#256" ~/.julia/packages/SafeTestsets/raUNr/src/SafeTestsets.jl:28
   [12] macro expansion
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/runtests.jl:20 [inlined]
   [13] macro expansion
      @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
   [14] macro expansion
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/runtests.jl:20 [inlined]
   [15] top-level scope
      @ ~/.julia/packages/SafeTestsets/raUNr/src/SafeTestsets.jl:30
   [16] eval(m::Module, e::Any)
      @ Core ./boot.jl:385
   [17] macro expansion
      @ ~/.julia/packages/SafeTestsets/raUNr/src/SafeTestsets.jl:28 [inlined]
   [18] top-level scope
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/runtests.jl:19
   [19] include(fname::String)
      @ Base.MainInclude ./client.jl:489
   [20] top-level scope
      @ none:6
   [21] eval
      @ ./boot.jl:385 [inlined]
   [22] exec_options(opts::Base.JLOptions)
      @ Base ./client.jl:291
   [23] _start()
      @ Base ./client.jl:552
Test Summary:                                                | Pass  Error  Total  Time
ecc                                                          |  340      2    342  9.4s
  naive syndrome circuits - zero syndrome for logical states |  320      1    321  4.7s
  naive syndrome circuits - zero syndrome for logical states |   16      1     17  2.2s
  is degenerate function - test on popular codes             |    4             4  0.3s
ERROR: LoadError: Some tests did not pass: 340 passed, 0 failed, 2 errored, 0 broken.
in expression starting at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/runtests.jl:61
ERROR: Package QuantumClifford errored during testing

Apr 02 '24 07:04 Fe-r-oz

Breakdown after running each encoding and decoding file separately

Firstly, test_ecc_decoder_all_setups pass for the given values of SteaneReedMuller

julia> include("test_ecc_decoder_all_setups.jl")
WARNING: redefinition of constant Main.code_instance_args. This may fail, cause incorrect answers, or produce other errors.
Test Summary:                       |Time
table decoder, good for small codes | None  8.1s
Test Summary:                               |Time
belief prop decoders, good for sparse codes | None  0.1s
Test Summary:                                                          |Time
matching decoder, good as long as column weight of the code is limited | None  9.2s
Test.DefaultTestSet("matching decoder, good as long as column weight of the code is limited", Any[], 0, false, false, true, 1.712044735925158e9, 1.712044745110257e9, false, "/home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_decoder_all_setups.jl")

K<0 error in test_ecc.

julia> include("test_ecc.jl")
WARNING: redefinition of constant Main.code_instance_args. This may fail, cause incorrect answers, or produce other errors.
naive syndrome circuits - zero syndrome for logical states: Error During Test at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:37
  Got exception outside of a @test
  ArgumentError: dimension size must be ≥ 0, got -10 for dimension 1
  Stacktrace:
    [1] BitArray
      @ ./bitarray.jl:32 [inlined]
    [2] BitArray
      @ ./bitarray.jl:71 [inlined]
    [3] falses(dims::Tuple{Int64})
      @ Base ./bitarray.jl:405
    [4] falses
      @ ./bitarray.jl:403 [inlined]
    [5] quantum_mallows(rng::Random._GLOBAL_RNG, n::Int64)
      @ QuantumClifford ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:189
    [6] random_destabilizer(rng::Random._GLOBAL_RNG, n::Int64; phases::Bool)
      @ QuantumClifford ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:73
    [7] random_destabilizer
      @ ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:72 [inlined]
    [8] random_stabilizer(rng::Random._GLOBAL_RNG, n::Int64; phases::Bool)
      @ QuantumClifford ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:166
    [9] random_stabilizer
      @ ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:166 [inlined]
   [10] random_stabilizer
      @ ~/Desktop/New/b10/QuantumClifford.jl/src/randoms.jl:167 [inlined]
   [11] test_naive_syndrome(c::SteaneReedMuller, e::Bool)
      @ Main ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:12
   [12] macro expansion
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:39 [inlined]
   [13] macro expansion
      @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
   [14] top-level scope
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:38
   [15] include(fname::String)
      @ Base.MainInclude ./client.jl:489
   [16] top-level scope
      @ REPL[7]:1
   [17] eval
      @ ./boot.jl:385 [inlined]
   [18] eval_user_input(ast::Any, backend::REPL.REPLBackend, mod::Module)
      @ REPL ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/REPL/src/REPL.jl:150
   [19] repl_backend_loop(backend::REPL.REPLBackend, get_module::Function)
      @ REPL ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/REPL/src/REPL.jl:246
   [20] start_repl_backend(backend::REPL.REPLBackend, consumer::Any; get_module::Function)
      @ REPL ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/REPL/src/REPL.jl:231
   [21] run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool, backend::Any)
      @ REPL ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/REPL/src/REPL.jl:389
   [22] run_repl(repl::REPL.AbstractREPL, consumer::Any)
      @ REPL ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/REPL/src/REPL.jl:375
   [23] (::Base.var"#1013#1015"{Bool, Bool, Bool})(REPL::Module)
      @ Base ./client.jl:432
   [24] #invokelatest#2
      @ ./essentials.jl:892 [inlined]
   [25] invokelatest
      @ ./essentials.jl:889 [inlined]
   [26] run_main_repl(interactive::Bool, quiet::Bool, banner::Bool, history_file::Bool, color_set::Bool)
      @ Base ./client.jl:416
   [27] exec_options(opts::Base.JLOptions)
      @ Base ./client.jl:333
   [28] _start()
      @ Base ./client.jl:552
Test Summary:                                              | Pass  Error  Total  Time
naive syndrome circuits - zero syndrome for logical states |  320      1    321  2.5s
ERROR: LoadError: Some tests did not pass: 320 passed, 0 failed, 1 errored, 0 broken.
in expression starting at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc.jl:37

test_ecc_codeproperties: I have not provided the code_s in the file as I don't know it's definition.


julia> include("test_ecc_codeproperties.jl")
WARNING: redefinition of constant Main.code_instance_args. This may fail, cause incorrect answers, or produce other errors.
Test Summary: | Pass  Total  Time
is CSS        |   21     21  0.4s
code tableau consistency: Test Failed at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:34
  Expression: code_s(code) + code_k(code) == code_n(code)
   Evaluated: 6 == 16

Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:672 [inlined]
 [2] macro expansion
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:34 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
 [4] top-level scope
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:30
code tableau consistency: Test Failed at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:35
  Expression: size(H, 1) < size(H, 2)
   Evaluated: 16 < 16

Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:672 [inlined]
 [2] macro expansion
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:35 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
 [4] top-level scope
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:30
code tableau consistency: Test Failed at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:38
  Expression: QuantumClifford.stab_looks_good(copy(H))

Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:672 [inlined]
 [2] macro expansion
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:38 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
 [4] top-level scope
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:30
code tableau consistency: Test Failed at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:35
  Expression: size(H, 1) < size(H, 2)
   Evaluated: 64 < 64

Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:672 [inlined]
 [2] macro expansion
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:35 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
 [4] top-level scope
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:30
code tableau consistency: Test Failed at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:38
  Expression: QuantumClifford.stab_looks_good(copy(H))

Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:672 [inlined]
 [2] macro expansion
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:38 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
 [4] top-level scope
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:30
code tableau consistency: Test Failed at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:34
  Expression: code_s(code) + code_k(code) == code_n(code)
   Evaluated: 8 == 64

Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:672 [inlined]
 [2] macro expansion
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:34 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
 [4] top-level scope
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:30
code tableau consistency: Test Failed at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:35
  Expression: size(H, 1) < size(H, 2)
   Evaluated: 64 < 64

Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:672 [inlined]
 [2] macro expansion
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:35 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
 [4] top-level scope
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:30
code tableau consistency: Test Failed at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:38
  Expression: QuantumClifford.stab_looks_good(copy(H))

Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:672 [inlined]
 [2] macro expansion
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:38 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
 [4] top-level scope
   @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:30
Test Summary:            | Pass  Fail  Total  Time
code tableau consistency |  118     8    126  0.7s
ERROR: LoadError: Some tests did not pass: 118 passed, 8 failed, 0 errored, 0 broken.
in expression starting at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_codeproperties.jl:29

syndromes

include("test_ecc_syndromes.jl")
WARNING: redefinition of constant Main.code_instance_args. This may fail, cause incorrect answers, or produce other errors.
naive and shor measurement circuits: Error During Test at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_syndromes.jl:45
  Got exception outside of a @test
  BoundsError: attempt to access 16-element view(::Vector{UInt8}, 17:32) with eltype UInt8 at index [17]
  Stacktrace:
    [1] throw_boundserror(A::SubArray{UInt8, 1, Vector{UInt8}, Tuple{UnitRange{Int64}}, true}, I::Tuple{Int64})
      @ Base ./abstractarray.jl:737
    [2] checkbounds
      @ ./abstractarray.jl:702 [inlined]
    [3] getindex
      @ ./subarray.jl:322 [inlined]
    [4] naive_encoding_circuit(code::SteaneReedMuller; undoperm::Bool)
      @ QuantumClifford.ECC ~/Desktop/New/b10/QuantumClifford.jl/src/ecc/circuits.jl:53
    [5] naive_encoding_circuit
      @ ~/Desktop/New/b10/QuantumClifford.jl/src/ecc/circuits.jl:18 [inlined]
    [6] pframe_naive_vs_shor_syndrome(code::SteaneReedMuller)
      @ Main ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_syndromes.jl:10
    [7] macro expansion
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_syndromes.jl:47 [inlined]
    [8] macro expansion
      @ ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]
    [9] top-level scope
      @ ~/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_syndromes.jl:46
   [10] include(fname::String)
      @ Base.MainInclude ./client.jl:489
   [11] top-level scope
      @ REPL[5]:1
   [12] eval
      @ ./boot.jl:385 [inlined]
   [13] eval_user_input(ast::Any, backend::REPL.REPLBackend, mod::Module)
      @ REPL ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/REPL/src/REPL.jl:150
   [14] repl_backend_loop(backend::REPL.REPLBackend, get_module::Function)
      @ REPL ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/REPL/src/REPL.jl:246
   [15] start_repl_backend(backend::REPL.REPLBackend, consumer::Any; get_module::Function)
      @ REPL ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/REPL/src/REPL.jl:231
   [16] run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool, backend::Any)
      @ REPL ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/REPL/src/REPL.jl:389
   [17] run_repl(repl::REPL.AbstractREPL, consumer::Any)
      @ REPL ~/.julia/juliaup/julia-1.10.2+0.x64.linux.gnu/share/julia/stdlib/v1.10/REPL/src/REPL.jl:375
   [18] (::Base.var"#1013#1015"{Bool, Bool, Bool})(REPL::Module)
      @ Base ./client.jl:432
   [19] #invokelatest#2
      @ ./essentials.jl:892 [inlined]
   [20] invokelatest
      @ ./essentials.jl:889 [inlined]
   [21] run_main_repl(interactive::Bool, quiet::Bool, banner::Bool, history_file::Bool, color_set::Bool)
      @ Base ./client.jl:416
   [22] exec_options(opts::Base.JLOptions)
      @ Base ./client.jl:333
   [23] _start()
      @ Base ./client.jl:552
Test Summary:                       | Pass  Error  Total  Time
naive and shor measurement circuits |   88      1     89  0.4s
ERROR: LoadError: Some tests did not pass: 88 passed, 0 failed, 1 errored, 0 broken.
in expression starting at /home/Transcendence/Desktop/New/b10/QuantumClifford.jl/test/test_ecc_syndromes.jl:45

Apr 02 '24 08:04 Fe-r-oz

What's does stab_looks_good actually checks?

Apr 02 '24 11:04 Fe-r-oz

What's does stab_looks_good actually checks?

This list of properties https://github.com/QuantumSavory/QuantumClifford.jl/blob/11ae38c7342c76a75ad9201b609b352d38117256/src/QuantumClifford.jl#L1082

Apr 02 '24 13:04 Krastanov

K<0

I think I understand what k=0 is. I do not think it makes sense to have situations where k<0.

Apr 02 '24 13:04 Krastanov

Every other test seem to work well. It's seems to me that it's the k = 0, and the way the code_base is not to be setup does provides error when k = 0.

Please have a look at CI tests. They provide the same error that is locally.

It is not obvious to me that the other tests work. If I am reading this right, all these tests are not executed because the test before them failed. https://github.com/QuantumSavory/QuantumClifford.jl/blob/master/test/runtests.jl#L62-L74

Apr 02 '24 13:04 Krastanov

Every other test seem to work well.

I phrased it wrong as a general statement. Testing individually, test_ecc_decoder_all_setups.jl passes.

Apr 02 '24 14:04 Fe-r-oz

I think I understand what k=0 is. I do not think it makes sense to have situations where k<0.

I have been trying to ask whether we should include k = 0 for sometime ;/ Even prepared an equation image for it!

I will add an condition that does not include k =0 cases to avoid cases which have the following error ArgumentError: dimension size must be ≥ 0, got -10 for dimension 1

I will make sure that the verify with Steane Paper, so that the algorithm is correct.

Thanks for your comments! I will try to improve the work.

Apr 02 '24 14:04 Fe-r-oz

I think I understand what k=0 is. I do not think it makes sense to have situations where k<0.

I have been trying to ask whether we should include k = 0 for sometime ;/ Even prepared an equation image for it!

I will add an condition that does not include k =0 cases to avoid cases which have the following error ArgumentError: dimension size must be ≥ 0, got -10 for dimension 1

I will make sure that the verify with Steane Paper, so that the algorithm is correct.

Thanks for your comments! I will try to improve the work.

To clarify, k=0 can make perfect sense. I do not understand what k<0 is.

Apr 02 '24 14:04 Krastanov

I do not understand what k<0 is.

This k<0 has been dangerous for me as well ;/ I don't understand how 'generators for k < 0 are stabilizers of k > 0 codes.'

Given my limited knowledge on the subject as I am just starting out, I have not read how generator matrices are considered as stabilizers/parity matrices when k goes negative.

The paper does not elaborate on this either.

Apr 02 '24 15:04 Fe-r-oz

Codecov Report

Attention: Patch coverage is 0% with 64 lines in your changes are missing coverage. Please review.

Project coverage is 0.00%. Comparing base (cae94d8) to head (2197365). Report is 2 commits behind head on master.

:exclamation: Current head 2197365 differs from pull request most recent head 378b7c9. Consider uploading reports for the commit 378b7c9 to get more accurate results

Files	Patch %	Lines
src/ecc/codes/steanereedmuller.jl	0.00%	64 Missing :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #251       +/-   ##
==========================================
- Coverage   82.63%   0.00%   -82.64%     
==========================================
  Files          58      44       -14     
  Lines        3794    3392      -402     
==========================================
- Hits         3135       0     -3135     
- Misses        659    3392     +2733

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Apr 02 '24 18:04 codecov[bot]

Docstring added from paper which tries to explain the k< 0. I wanted to add unicode for subscript z, but I am not sure how to add that. These codes have peculiar properties, and thus, maybe we need to less the strictness some code properties.

Edit Added the condition to not consider the codes when k<0. They are not in stabilizer matrices!

Apr 02 '24 18:04 Fe-r-oz

Seeking out review so that I can make it better.

I have not read how generator matrices are considered as stabilizers/parity matrices when k goes negative. The paper does not elaborate on this either.

Paper has 1 small paragraph about it, so I had carefully added it in docstring.

Edit Tagged my comment from earlier using >

Apr 02 '24 18:04 Fe-r-oz

Kindly please review this as well. Looking forward to your comments!

Apr 04 '24 20:04 Fe-r-oz

I will look into it sometime late next week. Ping me if I have not by then.

Apr 04 '24 21:04 Krastanov

Benchmark Result

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Time of benchmarks:

Target: 14 Apr 2024 - 11:40
Baseline: 14 Apr 2024 - 11:48

Package commits:

Target: e00308
Baseline: 64f695

Julia commits:

Target: b9aeaf
Baseline: b9aeaf

Julia command flags:

Target: None
Baseline: None

Environment variables:

Target: None
Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with :x:), while a ratio less than 1.0 denotes a possible improvement (marked with :white_check_mark:). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds).

ID	time ratio	memory ratio
`["circuitsim", "mctrajectories_sumtype", "q1001_r1"]`	0.94 (5%) :white_check_mark:	1.00 (1%)
`["circuitsim", "mctrajectories_union", "q1001_r1"]`	0.93 (5%) :white_check_mark:	1.00 (1%)
`["circuitsim", "pftrajectories", "q1001_r1"]`	1.07 (5%) :x:	1.00 (1%)
`["circuitsim", "pftrajectories", "q101_r1"]`	1.09 (5%) :x:	1.00 (1%)
`["clifford", "dense", "cnot250_on_diag500_stab"]`	1.14 (5%) :x:	1.00 (1%)
`["clifford", "dense", "cnot_on_dense500_stab"]`	0.95 (5%) :white_check_mark:	1.00 (1%)
`["clifford", "dense", "dense500_on_diag500_stab"]`	1.14 (5%) :x:	1.00 (1%)
`["ecc", "evaluate_decoder", "shor_bp_shorsyn"]`	0.95 (5%) :white_check_mark:	0.99 (1%) :white_check_mark:
`["ecc", "evaluate_decoder", "shor_pybp_comm"]`	0.82 (5%) :white_check_mark:	1.00 (1%)
`["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]`	0.89 (5%) :white_check_mark:	1.00 (1%)
`["stabilizer", "canon", "diag_rref500"]`	1.06 (5%) :x:	1.00 (1%)
`["stabilizer", "project", "stabilizer"]`	1.05 (5%) :x:	1.00 (1%)
`["stabilizer", "tensor", "diag_pow5_20"]`	0.90 (5%) :white_check_mark:	1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["circuitsim", "mctrajectories"]
["circuitsim", "mctrajectories_sumtype"]
["circuitsim", "mctrajectories_union"]
["circuitsim", "pftrajectories"]
["circuitsim", "pftrajectories_sumtype"]
["circuitsim", "pftrajectories_union"]
["clifford", "dense"]
["clifford", "symbolic"]
["ecc", "evaluate_decoder"]
["pauli", "mul"]
["stabilizer", "canon"]
["stabilizer", "project"]
["stabilizer", "tensor"]
["stabilizer", "trace"]

Julia versioninfo

Target

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2263 s 0 s 298 s 6834 s 0 s
 #2 0 MHz 1850 s 0 s 265 s 7274 s 0 s
 #3 0 MHz 2832 s 0 s 300 s 6257 s 0 s
 #4 0 MHz 2533 s 0 s 302 s 6546 s 0 s
 Memory: 15.606494903564453 GB (13352.0390625 MB free)
 Uptime: 944.78 sec
 Load Avg: 1.04 1.11 0.8
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 3385 s 0 s 386 s 9880 s 0 s
 #2 0 MHz 3281 s 0 s 351 s 10017 s 0 s
 #3 0 MHz 3869 s 0 s 397 s 9380 s 0 s
 #4 0 MHz 3349 s 0 s 408 s 9880 s 0 s
 Memory: 15.606494903564453 GB (13222.359375 MB free)
 Uptime: 1371.9 sec
 Load Avg: 1.07 1.08 0.92
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Time of benchmark: 14 Apr 2024 - 11:40
Package commit: e00308
Julia commit: b9aeaf
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks. The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero.

ID	time	GC time	memory	allocations
`["circuitsim", "mctrajectories", "q1001_r1"]`	15.156 ms (5%)		500.77 KiB (1%)	18019
`["circuitsim", "mctrajectories", "q101_r1"]`	171.779 μs (5%)		50.53 KiB (1%)	1818
`["circuitsim", "mctrajectories_sumtype", "q1001_r1"]`	13.436 ms (5%)		496 bytes (1%)	9
`["circuitsim", "mctrajectories_sumtype", "q101_r1"]`	118.991 μs (5%)		256 bytes (1%)	8
`["circuitsim", "mctrajectories_union", "q1001_r1"]`	12.931 ms (5%)		496 bytes (1%)	9
`["circuitsim", "mctrajectories_union", "q101_r1"]`	117.960 μs (5%)		256 bytes (1%)	8
`["circuitsim", "pftrajectories", "q1001_r1"]`	63.549 μs (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q1001_r100"]`	174.865 μs (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q1001_r10000"]`	1.107 ms (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q101_r1"]`	6.492 μs (5%)		9.42 KiB (1%)	201
`["circuitsim", "pftrajectories_sumtype", "q1001_r1"]`	44.293 μs (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r100"]`	154.527 μs (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]`	1.083 ms (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]`	5.859 ms (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q101_r1"]`	4.478 μs (5%)		9.47 KiB (1%)	202
`["circuitsim", "pftrajectories_union", "q1001_r1"]`	22.602 μs (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q1001_r100"]`	132.907 μs (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q1001_r10000"]`	1.054 ms (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q101_r1"]`	2.334 μs (5%)		96 bytes (1%)	2
`["clifford", "dense", "cnot250_on_dense500_destab"]`	11.636 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_dense500_stab"]`	5.733 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_diag500_destab"]`	1.122 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_diag500_stab"]`	560.663 μs (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot_on_dense500_destab"]`	51.095 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_dense500_stab"]`	23.043 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_diag500_destab"]`	26.490 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_diag500_stab"]`	13.325 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "dense500_on_dense500_destab"]`	11.619 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_dense500_stab"]`	5.716 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_diag500_destab"]`	979.803 μs (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_diag500_stab"]`	560.663 μs (5%)		512 bytes (1%)	8
`["clifford", "symbolic", "cnot250_on_dense500_destab"]`	1.446 ms (5%)
`["clifford", "symbolic", "cnot250_on_dense500_stab"]`	686.788 μs (5%)
`["clifford", "symbolic", "cnot250_on_diag500_destab"]`	1.158 ms (5%)
`["clifford", "symbolic", "cnot250_on_diag500_stab"]`	569.710 μs (5%)
`["clifford", "symbolic", "cnot_on_dense500_destab"]`	4.729 μs (5%)
`["clifford", "symbolic", "cnot_on_dense500_stab"]`	2.304 μs (5%)
`["clifford", "symbolic", "cnot_on_diag500_destab"]`	4.649 μs (5%)
`["clifford", "symbolic", "cnot_on_diag500_stab"]`	2.294 μs (5%)
`["ecc", "evaluate_decoder", "shor_bp_comm"]`	1.864 ms (5%)		1.77 MiB (1%)	42497
`["ecc", "evaluate_decoder", "shor_bp_naivesyn"]`	4.154 ms (5%)		3.60 MiB (1%)	81172
`["ecc", "evaluate_decoder", "shor_bp_shorsyn"]`	4.585 ms (5%)		3.71 MiB (1%)	82804
`["ecc", "evaluate_decoder", "shor_pybp_comm"]`	55.796 ms (5%)		4.76 MiB (1%)	144207
`["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]`	126.504 ms (5%)		9.57 MiB (1%)	284018
`["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]`	138.460 ms (5%)		9.66 MiB (1%)	285104
`["ecc", "evaluate_decoder", "shor_table_comm"]`	256.788 μs (5%)		241.42 KiB (1%)	4593
`["ecc", "evaluate_decoder", "shor_table_naivesyn"]`	873.325 μs (5%)		334.20 KiB (1%)	4790
`["ecc", "evaluate_decoder", "shor_table_shorsyn"]`	1.274 ms (5%)		427.73 KiB (1%)	5876
`["ecc", "evaluate_decoder", "toric8_bp_comm"]`	687.924 ms (5%)	14.117 ms	173.01 MiB (1%)	1037289
`["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]`	1.380 s (5%)	28.500 ms	348.16 MiB (1%)	2109359
`["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]`	1.431 s (5%)	28.882 ms	359.72 MiB (1%)	2195029
`["ecc", "evaluate_decoder", "toric8_pybp_comm"]`	458.415 ms (5%)		8.11 MiB (1%)	163696
`["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]`	947.037 ms (5%)		18.04 MiB (1%)	360388
`["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]`	951.169 ms (5%)		20.56 MiB (1%)	392802
`["ecc", "evaluate_decoder", "toric8_pymatch_comm"]`	4.253 ms (5%)		1.95 MiB (1%)	23176
`["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]`	12.541 ms (5%)		5.58 MiB (1%)	79355
`["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]`	21.109 ms (5%)		8.09 MiB (1%)	111769
`["ecc", "evaluate_decoder", "toric8_table_comm"]`	2.647 ms (5%)		1.64 MiB (1%)	23082
`["ecc", "evaluate_decoder", "toric8_table_naivesyn"]`	10.236 ms (5%)		4.93 MiB (1%)	79160
`["ecc", "evaluate_decoder", "toric8_table_shorsyn"]`	18.772 ms (5%)		7.44 MiB (1%)	111574
`["pauli", "mul", "100"]`	14.838 ns (5%)
`["pauli", "mul", "1000"]`	17.930 ns (5%)
`["pauli", "mul", "100000"]`	723.346 ns (5%)
`["pauli", "mul", "20000000"]`	180.556 μs (5%)
`["stabilizer", "canon", "cano500"]`	3.109 ms (5%)
`["stabilizer", "canon", "diag_cano500"]`	676.749 μs (5%)
`["stabilizer", "canon", "diag_gott500"]`	4.184 ms (5%)		5.19 MiB (1%)	34521
`["stabilizer", "canon", "diag_rref500"]`	675.567 μs (5%)
`["stabilizer", "canon", "gott500"]`	6.551 ms (5%)		5.19 MiB (1%)	34531
`["stabilizer", "canon", "md_cano500"]`	1.592 ms (5%)
`["stabilizer", "canon", "md_rref500"]`	1.629 ms (5%)
`["stabilizer", "canon", "rref500"]`	3.196 ms (5%)
`["stabilizer", "project", "destabilizer"]`	18.223 μs (5%)		288 bytes (1%)	5
`["stabilizer", "project", "stabilizer"]`	9.428 μs (5%)		80 bytes (1%)	2
`["stabilizer", "tensor", "diag_pow5_20"]`	1.139 ms (5%)		23.97 MiB (1%)	34
`["stabilizer", "tensor", "pow5_20"]`	2.869 μs (5%)		6.44 KiB (1%)	31
`["stabilizer", "trace", "destabilizer"]`	34.114 μs (5%)		80 bytes (1%)	2
`["stabilizer", "trace", "stabilizer"]`	24.495 μs (5%)		112 bytes (1%)	3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["circuitsim", "mctrajectories"]
["circuitsim", "mctrajectories_sumtype"]
["circuitsim", "mctrajectories_union"]
["circuitsim", "pftrajectories"]
["circuitsim", "pftrajectories_sumtype"]
["circuitsim", "pftrajectories_union"]
["clifford", "dense"]
["clifford", "symbolic"]
["ecc", "evaluate_decoder"]
["pauli", "mul"]
["stabilizer", "canon"]
["stabilizer", "project"]
["stabilizer", "tensor"]
["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2263 s 0 s 298 s 6834 s 0 s
 #2 0 MHz 1850 s 0 s 265 s 7274 s 0 s
 #3 0 MHz 2832 s 0 s 300 s 6257 s 0 s
 #4 0 MHz 2533 s 0 s 302 s 6546 s 0 s
 Memory: 15.606494903564453 GB (13352.0390625 MB free)
 Uptime: 944.78 sec
 Load Avg: 1.04 1.11 0.8
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Time of benchmark: 14 Apr 2024 - 11:48
Package commit: 64f695
Julia commit: b9aeaf
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks. The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero.

ID	time	GC time	memory	allocations
`["circuitsim", "mctrajectories", "q1001_r1"]`	15.134 ms (5%)		500.77 KiB (1%)	18019
`["circuitsim", "mctrajectories", "q101_r1"]`	171.709 μs (5%)		50.53 KiB (1%)	1818
`["circuitsim", "mctrajectories_sumtype", "q1001_r1"]`	14.271 ms (5%)		496 bytes (1%)	9
`["circuitsim", "mctrajectories_sumtype", "q101_r1"]`	118.701 μs (5%)		256 bytes (1%)	8
`["circuitsim", "mctrajectories_union", "q1001_r1"]`	13.836 ms (5%)		496 bytes (1%)	9
`["circuitsim", "mctrajectories_union", "q101_r1"]`	116.867 μs (5%)		256 bytes (1%)	8
`["circuitsim", "pftrajectories", "q1001_r1"]`	59.240 μs (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q1001_r100"]`	173.462 μs (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q1001_r10000"]`	1.100 ms (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q101_r1"]`	5.971 μs (5%)		9.42 KiB (1%)	201
`["circuitsim", "pftrajectories_sumtype", "q1001_r1"]`	44.884 μs (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r100"]`	157.844 μs (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]`	1.087 ms (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]`	5.849 ms (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q101_r1"]`	4.488 μs (5%)		9.47 KiB (1%)	202
`["circuitsim", "pftrajectories_union", "q1001_r1"]`	22.612 μs (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q1001_r100"]`	134.370 μs (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q1001_r10000"]`	1.056 ms (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q101_r1"]`	2.344 μs (5%)		96 bytes (1%)	2
`["clifford", "dense", "cnot250_on_dense500_destab"]`	11.423 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_dense500_stab"]`	5.735 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_diag500_destab"]`	1.123 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_diag500_stab"]`	490.432 μs (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot_on_dense500_destab"]`	50.644 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_dense500_stab"]`	24.376 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_diag500_destab"]`	26.249 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_diag500_stab"]`	13.715 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "dense500_on_dense500_destab"]`	11.427 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_dense500_stab"]`	5.730 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_diag500_destab"]`	980.164 μs (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_diag500_stab"]`	490.212 μs (5%)		512 bytes (1%)	8
`["clifford", "symbolic", "cnot250_on_dense500_destab"]`	1.433 ms (5%)
`["clifford", "symbolic", "cnot250_on_dense500_stab"]`	684.583 μs (5%)
`["clifford", "symbolic", "cnot250_on_diag500_destab"]`	1.161 ms (5%)
`["clifford", "symbolic", "cnot250_on_diag500_stab"]`	568.315 μs (5%)
`["clifford", "symbolic", "cnot_on_dense500_destab"]`	4.749 μs (5%)
`["clifford", "symbolic", "cnot_on_dense500_stab"]`	2.304 μs (5%)
`["clifford", "symbolic", "cnot_on_diag500_destab"]`	4.649 μs (5%)
`["clifford", "symbolic", "cnot_on_diag500_stab"]`	2.284 μs (5%)
`["ecc", "evaluate_decoder", "shor_bp_comm"]`	1.822 ms (5%)		1.77 MiB (1%)	42665
`["ecc", "evaluate_decoder", "shor_bp_naivesyn"]`	4.042 ms (5%)		3.61 MiB (1%)	81606
`["ecc", "evaluate_decoder", "shor_bp_shorsyn"]`	4.828 ms (5%)		3.76 MiB (1%)	84120
`["ecc", "evaluate_decoder", "shor_pybp_comm"]`	68.329 ms (5%)		4.76 MiB (1%)	144207
`["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]`	141.506 ms (5%)		9.57 MiB (1%)	284018
`["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]`	142.219 ms (5%)		9.66 MiB (1%)	285104
`["ecc", "evaluate_decoder", "shor_table_comm"]`	252.741 μs (5%)		241.42 KiB (1%)	4593
`["ecc", "evaluate_decoder", "shor_table_naivesyn"]`	869.669 μs (5%)		334.20 KiB (1%)	4790
`["ecc", "evaluate_decoder", "shor_table_shorsyn"]`	1.266 ms (5%)		427.73 KiB (1%)	5876
`["ecc", "evaluate_decoder", "toric8_bp_comm"]`	682.276 ms (5%)	11.528 ms	171.80 MiB (1%)	1030191
`["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]`	1.388 s (5%)	24.881 ms	350.12 MiB (1%)	2120895
`["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]`	1.419 s (5%)	26.709 ms	356.86 MiB (1%)	2178201
`["ecc", "evaluate_decoder", "toric8_pybp_comm"]`	465.957 ms (5%)		8.11 MiB (1%)	163696
`["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]`	952.794 ms (5%)		18.04 MiB (1%)	360388
`["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]`	950.216 ms (5%)		20.56 MiB (1%)	392802
`["ecc", "evaluate_decoder", "toric8_pymatch_comm"]`	4.224 ms (5%)		1.95 MiB (1%)	23176
`["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]`	12.500 ms (5%)		5.58 MiB (1%)	79355
`["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]`	20.975 ms (5%)		8.09 MiB (1%)	111769
`["ecc", "evaluate_decoder", "toric8_table_comm"]`	2.645 ms (5%)		1.64 MiB (1%)	23082
`["ecc", "evaluate_decoder", "toric8_table_naivesyn"]`	10.174 ms (5%)		4.93 MiB (1%)	79160
`["ecc", "evaluate_decoder", "toric8_table_shorsyn"]`	18.636 ms (5%)		7.44 MiB (1%)	111574
`["pauli", "mul", "100"]`	14.626 ns (5%)
`["pauli", "mul", "1000"]`	18.873 ns (5%)
`["pauli", "mul", "100000"]`	729.046 ns (5%)
`["pauli", "mul", "20000000"]`	173.172 μs (5%)
`["stabilizer", "canon", "cano500"]`	3.081 ms (5%)
`["stabilizer", "canon", "diag_cano500"]`	708.669 μs (5%)
`["stabilizer", "canon", "diag_gott500"]`	4.091 ms (5%)		5.19 MiB (1%)	34521
`["stabilizer", "canon", "diag_rref500"]`	637.937 μs (5%)
`["stabilizer", "canon", "gott500"]`	6.507 ms (5%)		5.19 MiB (1%)	34531
`["stabilizer", "canon", "md_cano500"]`	1.591 ms (5%)
`["stabilizer", "canon", "md_rref500"]`	1.611 ms (5%)
`["stabilizer", "canon", "rref500"]`	3.138 ms (5%)
`["stabilizer", "project", "destabilizer"]`	18.484 μs (5%)		288 bytes (1%)	5
`["stabilizer", "project", "stabilizer"]`	8.967 μs (5%)		80 bytes (1%)	2
`["stabilizer", "tensor", "diag_pow5_20"]`	1.264 ms (5%)		23.97 MiB (1%)	34
`["stabilizer", "tensor", "pow5_20"]`	2.851 μs (5%)		6.44 KiB (1%)	31
`["stabilizer", "trace", "destabilizer"]`	32.781 μs (5%)		80 bytes (1%)	2
`["stabilizer", "trace", "stabilizer"]`	24.866 μs (5%)		112 bytes (1%)	3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["circuitsim", "mctrajectories"]
["circuitsim", "mctrajectories_sumtype"]
["circuitsim", "mctrajectories_union"]
["circuitsim", "pftrajectories"]
["circuitsim", "pftrajectories_sumtype"]
["circuitsim", "pftrajectories_union"]
["clifford", "dense"]
["clifford", "symbolic"]
["ecc", "evaluate_decoder"]
["pauli", "mul"]
["stabilizer", "canon"]
["stabilizer", "project"]
["stabilizer", "tensor"]
["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 3385 s 0 s 386 s 9880 s 0 s
 #2 0 MHz 3281 s 0 s 351 s 10017 s 0 s
 #3 0 MHz 3869 s 0 s 397 s 9380 s 0 s
 #4 0 MHz 3349 s 0 s 408 s 9880 s 0 s
 Memory: 15.606494903564453 GB (13222.359375 MB free)
 Uptime: 1371.9 sec
 Load Avg: 1.07 1.08 0.92
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Runtime information

Runtime Info
BLAS #threads	2
`BLAS.vendor()`	`lbt`
`Sys.CPU_THREADS`	4

lscpu output:

Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 48 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 4 On-line CPU(s) list: 0-3 Vendor ID: AuthenticAMD Model name: AMD EPYC 7763 64-Core Processor CPU family: 25 Model: 1 Thread(s) per core: 2 Core(s) per socket: 2 Socket(s): 1 Stepping: 1 BogoMIPS: 4890.85 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm Virtualization: AMD-V Hypervisor vendor: Microsoft Virtualization type: full L1d cache: 64 KiB (2 instances) L1i cache: 64 KiB (2 instances) L2 cache: 1 MiB (2 instances) L3 cache: 32 MiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0-3 Vulnerability Gather data sampling: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode Vulnerability Spec store bypass: Vulnerable Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected

Cpu Property	Value
Brand	AMD EPYC 7763 64-Core Processor
Vendor	:AMD
Architecture	:Unknown
Model	Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00
Cores	16 physical cores, 16 logical cores (on executing CPU)
	No Hyperthreading hardware capability detected
Clock Frequencies	Not supported by CPU
Data Cache	Level 1:3 : (32, 512, 32768) kbytes
	64 byte cache line size
Address Size	48 bits virtual, 48 bits physical
SIMD	256 bit = 32 byte max. SIMD vector size
Time Stamp Counter	TSC is accessible via `rdtsc`
	TSC runs at constant rate (invariant from clock frequency)
Perf. Monitoring	Performance Monitoring Counters (PMC) are not supported
Hypervisor	Yes, Microsoft

Benchmark Result

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Time of benchmarks:

Target: 14 Apr 2024 - 12:08
Baseline: 14 Apr 2024 - 12:15

Package commits:

Target: 390841
Baseline: 64f695

Julia commits:

Target: b9aeaf
Baseline: b9aeaf

Julia command flags:

Target: None
Baseline: None

Environment variables:

Target: None
Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with :x:), while a ratio less than 1.0 denotes a possible improvement (marked with :white_check_mark:). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds).

ID	time ratio	memory ratio
`["circuitsim", "mctrajectories_union", "q1001_r1"]`	0.92 (5%) :white_check_mark:	1.00 (1%)
`["circuitsim", "pftrajectories", "q1001_r1"]`	1.12 (5%) :x:	1.00 (1%)
`["circuitsim", "pftrajectories", "q1001_r100"]`	1.07 (5%) :x:	1.00 (1%)
`["circuitsim", "pftrajectories", "q1001_r10000"]`	1.05 (5%) :x:	1.00 (1%)
`["circuitsim", "pftrajectories", "q101_r1"]`	1.12 (5%) :x:	1.00 (1%)
`["clifford", "dense", "cnot_on_dense500_stab"]`	0.95 (5%) :white_check_mark:	1.00 (1%)
`["ecc", "evaluate_decoder", "shor_bp_shorsyn"]`	0.94 (5%) :white_check_mark:	0.97 (1%) :white_check_mark:
`["ecc", "evaluate_decoder", "shor_pybp_comm"]`	1.08 (5%) :x:	1.00 (1%)
`["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]`	0.88 (5%) :white_check_mark:	1.00 (1%)
`["ecc", "evaluate_decoder", "toric8_bp_comm"]`	0.97 (5%)	0.99 (1%) :white_check_mark:
`["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]`	0.99 (5%)	0.98 (1%) :white_check_mark:
`["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]`	1.01 (5%)	1.02 (1%) :x:
`["pauli", "mul", "20000000"]`	1.05 (5%) :x:	1.00 (1%)
`["stabilizer", "tensor", "diag_pow5_20"]`	1.14 (5%) :x:	1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["circuitsim", "mctrajectories"]
["circuitsim", "mctrajectories_sumtype"]
["circuitsim", "mctrajectories_union"]
["circuitsim", "pftrajectories"]
["circuitsim", "pftrajectories_sumtype"]
["circuitsim", "pftrajectories_union"]
["clifford", "dense"]
["clifford", "symbolic"]
["ecc", "evaluate_decoder"]
["pauli", "mul"]
["stabilizer", "canon"]
["stabilizer", "project"]
["stabilizer", "tensor"]
["stabilizer", "trace"]

Julia versioninfo

Target

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2360 s 0 s 290 s 7985 s 0 s
 #2 0 MHz 2461 s 0 s 290 s 7886 s 0 s
 #3 0 MHz 2536 s 0 s 311 s 7785 s 0 s
 #4 0 MHz 2244 s 0 s 305 s 8086 s 0 s
 Memory: 15.606494903564453 GB (13355.05859375 MB free)
 Uptime: 1069.87 sec
 Load Avg: 1.01 1.08 0.78
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 3222 s 0 s 378 s 11328 s 0 s
 #2 0 MHz 2942 s 0 s 389 s 11600 s 0 s
 #3 0 MHz 4166 s 0 s 400 s 10358 s 0 s
 #4 0 MHz 3718 s 0 s 405 s 10804 s 0 s
 Memory: 15.606494903564453 GB (13236.34375 MB free)
 Uptime: 1500.63 sec
 Load Avg: 1.0 1.08 0.92
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Time of benchmark: 14 Apr 2024 - 12:8
Package commit: 390841
Julia commit: b9aeaf
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks. The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero.

ID	time	GC time	memory	allocations
`["circuitsim", "mctrajectories", "q1001_r1"]`	15.294 ms (5%)		500.77 KiB (1%)	18019
`["circuitsim", "mctrajectories", "q101_r1"]`	171.821 μs (5%)		50.53 KiB (1%)	1818
`["circuitsim", "mctrajectories_sumtype", "q1001_r1"]`	13.320 ms (5%)		496 bytes (1%)	9
`["circuitsim", "mctrajectories_sumtype", "q101_r1"]`	118.832 μs (5%)		256 bytes (1%)	8
`["circuitsim", "mctrajectories_union", "q1001_r1"]`	13.144 ms (5%)		496 bytes (1%)	9
`["circuitsim", "mctrajectories_union", "q101_r1"]`	116.928 μs (5%)		256 bytes (1%)	8
`["circuitsim", "pftrajectories", "q1001_r1"]`	66.755 μs (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q1001_r100"]`	181.770 μs (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q1001_r10000"]`	1.152 ms (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q101_r1"]`	6.692 μs (5%)		9.42 KiB (1%)	201
`["circuitsim", "pftrajectories_sumtype", "q1001_r1"]`	43.873 μs (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r100"]`	158.295 μs (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]`	1.129 ms (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]`	5.857 ms (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q101_r1"]`	4.488 μs (5%)		9.47 KiB (1%)	202
`["circuitsim", "pftrajectories_union", "q1001_r1"]`	22.311 μs (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q1001_r100"]`	136.175 μs (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q1001_r10000"]`	1.101 ms (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q101_r1"]`	2.304 μs (5%)		96 bytes (1%)	2
`["clifford", "dense", "cnot250_on_dense500_destab"]`	11.740 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_dense500_stab"]`	5.792 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_diag500_destab"]`	1.122 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_diag500_stab"]`	561.700 μs (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot_on_dense500_destab"]`	50.554 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_dense500_stab"]`	23.243 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_diag500_destab"]`	26.449 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_diag500_stab"]`	13.395 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "dense500_on_dense500_destab"]`	11.746 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_dense500_stab"]`	5.794 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_diag500_destab"]`	980.844 μs (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_diag500_stab"]`	561.700 μs (5%)		512 bytes (1%)	8
`["clifford", "symbolic", "cnot250_on_dense500_destab"]`	1.435 ms (5%)
`["clifford", "symbolic", "cnot250_on_dense500_stab"]`	687.866 μs (5%)
`["clifford", "symbolic", "cnot250_on_diag500_destab"]`	1.161 ms (5%)
`["clifford", "symbolic", "cnot250_on_diag500_stab"]`	570.026 μs (5%)
`["clifford", "symbolic", "cnot_on_dense500_destab"]`	4.729 μs (5%)
`["clifford", "symbolic", "cnot_on_dense500_stab"]`	2.304 μs (5%)
`["clifford", "symbolic", "cnot_on_diag500_destab"]`	4.659 μs (5%)
`["clifford", "symbolic", "cnot_on_diag500_stab"]`	2.304 μs (5%)
`["ecc", "evaluate_decoder", "shor_bp_comm"]`	1.841 ms (5%)		1.77 MiB (1%)	42651
`["ecc", "evaluate_decoder", "shor_bp_naivesyn"]`	4.173 ms (5%)		3.61 MiB (1%)	81578
`["ecc", "evaluate_decoder", "shor_bp_shorsyn"]`	4.573 ms (5%)		3.70 MiB (1%)	82622
`["ecc", "evaluate_decoder", "shor_pybp_comm"]`	63.400 ms (5%)		4.76 MiB (1%)	144207
`["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]`	119.778 ms (5%)		9.57 MiB (1%)	284018
`["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]`	140.635 ms (5%)		9.66 MiB (1%)	285104
`["ecc", "evaluate_decoder", "shor_table_comm"]`	258.123 μs (5%)		241.42 KiB (1%)	4593
`["ecc", "evaluate_decoder", "shor_table_naivesyn"]`	870.818 μs (5%)		334.20 KiB (1%)	4790
`["ecc", "evaluate_decoder", "shor_table_shorsyn"]`	1.263 ms (5%)		427.73 KiB (1%)	5876
`["ecc", "evaluate_decoder", "toric8_bp_comm"]`	678.360 ms (5%)	15.917 ms	172.26 MiB (1%)	1032858
`["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]`	1.398 s (5%)	26.988 ms	347.24 MiB (1%)	2103955
`["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]`	1.402 s (5%)	37.628 ms	353.22 MiB (1%)	2156767
`["ecc", "evaluate_decoder", "toric8_pybp_comm"]`	454.812 ms (5%)		8.11 MiB (1%)	163696
`["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]`	957.254 ms (5%)		18.04 MiB (1%)	360388
`["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]`	989.646 ms (5%)		20.56 MiB (1%)	392802
`["ecc", "evaluate_decoder", "toric8_pymatch_comm"]`	4.277 ms (5%)		1.95 MiB (1%)	23176
`["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]`	12.431 ms (5%)		5.58 MiB (1%)	79355
`["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]`	20.979 ms (5%)		8.09 MiB (1%)	111769
`["ecc", "evaluate_decoder", "toric8_table_comm"]`	2.685 ms (5%)		1.64 MiB (1%)	23082
`["ecc", "evaluate_decoder", "toric8_table_naivesyn"]`	10.147 ms (5%)		4.93 MiB (1%)	79160
`["ecc", "evaluate_decoder", "toric8_table_shorsyn"]`	18.603 ms (5%)		7.44 MiB (1%)	111574
`["pauli", "mul", "100"]`	14.205 ns (5%)
`["pauli", "mul", "1000"]`	17.968 ns (5%)
`["pauli", "mul", "100000"]`	717.629 ns (5%)
`["pauli", "mul", "20000000"]`	192.479 μs (5%)
`["stabilizer", "canon", "cano500"]`	3.049 ms (5%)
`["stabilizer", "canon", "diag_cano500"]`	709.055 μs (5%)
`["stabilizer", "canon", "diag_gott500"]`	4.051 ms (5%)		5.19 MiB (1%)	34521
`["stabilizer", "canon", "diag_rref500"]`	627.062 μs (5%)
`["stabilizer", "canon", "gott500"]`	6.400 ms (5%)		5.19 MiB (1%)	34531
`["stabilizer", "canon", "md_cano500"]`	1.548 ms (5%)
`["stabilizer", "canon", "md_rref500"]`	1.562 ms (5%)
`["stabilizer", "canon", "rref500"]`	3.118 ms (5%)
`["stabilizer", "project", "destabilizer"]`	18.335 μs (5%)		288 bytes (1%)	5
`["stabilizer", "project", "stabilizer"]`	9.598 μs (5%)		80 bytes (1%)	2
`["stabilizer", "tensor", "diag_pow5_20"]`	1.432 ms (5%)		23.97 MiB (1%)	34
`["stabilizer", "tensor", "pow5_20"]`	2.792 μs (5%)		6.44 KiB (1%)	31
`["stabilizer", "trace", "destabilizer"]`	31.298 μs (5%)		80 bytes (1%)	2
`["stabilizer", "trace", "stabilizer"]`	24.115 μs (5%)		112 bytes (1%)	3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["circuitsim", "mctrajectories"]
["circuitsim", "mctrajectories_sumtype"]
["circuitsim", "mctrajectories_union"]
["circuitsim", "pftrajectories"]
["circuitsim", "pftrajectories_sumtype"]
["circuitsim", "pftrajectories_union"]
["clifford", "dense"]
["clifford", "symbolic"]
["ecc", "evaluate_decoder"]
["pauli", "mul"]
["stabilizer", "canon"]
["stabilizer", "project"]
["stabilizer", "tensor"]
["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2360 s 0 s 290 s 7985 s 0 s
 #2 0 MHz 2461 s 0 s 290 s 7886 s 0 s
 #3 0 MHz 2536 s 0 s 311 s 7785 s 0 s
 #4 0 MHz 2244 s 0 s 305 s 8086 s 0 s
 Memory: 15.606494903564453 GB (13355.05859375 MB free)
 Uptime: 1069.87 sec
 Load Avg: 1.01 1.08 0.78
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Time of benchmark: 14 Apr 2024 - 12:15
Package commit: 64f695
Julia commit: b9aeaf
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks. The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero.

ID	time	GC time	memory	allocations
`["circuitsim", "mctrajectories", "q1001_r1"]`	15.503 ms (5%)		500.77 KiB (1%)	18019
`["circuitsim", "mctrajectories", "q101_r1"]`	169.847 μs (5%)		50.53 KiB (1%)	1818
`["circuitsim", "mctrajectories_sumtype", "q1001_r1"]`	13.717 ms (5%)		496 bytes (1%)	9
`["circuitsim", "mctrajectories_sumtype", "q101_r1"]`	118.171 μs (5%)		256 bytes (1%)	8
`["circuitsim", "mctrajectories_union", "q1001_r1"]`	14.247 ms (5%)		496 bytes (1%)	9
`["circuitsim", "mctrajectories_union", "q101_r1"]`	117.240 μs (5%)		256 bytes (1%)	8
`["circuitsim", "pftrajectories", "q1001_r1"]`	59.481 μs (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q1001_r100"]`	169.186 μs (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q1001_r10000"]`	1.097 ms (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q101_r1"]`	6.001 μs (5%)		9.42 KiB (1%)	201
`["circuitsim", "pftrajectories_sumtype", "q1001_r1"]`	44.593 μs (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r100"]`	154.789 μs (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]`	1.084 ms (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]`	5.831 ms (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q101_r1"]`	4.458 μs (5%)		9.47 KiB (1%)	202
`["circuitsim", "pftrajectories_union", "q1001_r1"]`	22.603 μs (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q1001_r100"]`	132.027 μs (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q1001_r10000"]`	1.052 ms (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q101_r1"]`	2.344 μs (5%)		96 bytes (1%)	2
`["clifford", "dense", "cnot250_on_dense500_destab"]`	11.725 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_dense500_stab"]`	5.756 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_diag500_destab"]`	1.122 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_diag500_stab"]`	561.591 μs (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot_on_dense500_destab"]`	50.644 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_dense500_stab"]`	24.485 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_diag500_destab"]`	26.129 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_diag500_stab"]`	13.725 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "dense500_on_dense500_destab"]`	11.725 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_dense500_stab"]`	5.760 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_diag500_destab"]`	980.134 μs (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_diag500_stab"]`	561.591 μs (5%)		512 bytes (1%)	8
`["clifford", "symbolic", "cnot250_on_dense500_destab"]`	1.440 ms (5%)
`["clifford", "symbolic", "cnot250_on_dense500_stab"]`	687.125 μs (5%)
`["clifford", "symbolic", "cnot250_on_diag500_destab"]`	1.161 ms (5%)
`["clifford", "symbolic", "cnot250_on_diag500_stab"]`	569.425 μs (5%)
`["clifford", "symbolic", "cnot_on_dense500_destab"]`	4.769 μs (5%)
`["clifford", "symbolic", "cnot_on_dense500_stab"]`	2.294 μs (5%)
`["clifford", "symbolic", "cnot_on_diag500_destab"]`	4.658 μs (5%)
`["clifford", "symbolic", "cnot_on_diag500_stab"]`	2.294 μs (5%)
`["ecc", "evaluate_decoder", "shor_bp_comm"]`	1.850 ms (5%)		1.78 MiB (1%)	42833
`["ecc", "evaluate_decoder", "shor_bp_naivesyn"]`	4.157 ms (5%)		3.62 MiB (1%)	81830
`["ecc", "evaluate_decoder", "shor_bp_shorsyn"]`	4.881 ms (5%)		3.81 MiB (1%)	85170
`["ecc", "evaluate_decoder", "shor_pybp_comm"]`	58.758 ms (5%)		4.76 MiB (1%)	144207
`["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]`	136.706 ms (5%)		9.57 MiB (1%)	284018
`["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]`	140.608 ms (5%)		9.66 MiB (1%)	285104
`["ecc", "evaluate_decoder", "shor_table_comm"]`	253.654 μs (5%)		241.42 KiB (1%)	4593
`["ecc", "evaluate_decoder", "shor_table_naivesyn"]`	866.530 μs (5%)		334.20 KiB (1%)	4790
`["ecc", "evaluate_decoder", "shor_table_shorsyn"]`	1.265 ms (5%)		427.73 KiB (1%)	5876
`["ecc", "evaluate_decoder", "toric8_bp_comm"]`	696.963 ms (5%)	12.891 ms	174.39 MiB (1%)	1045374
`["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]`	1.409 s (5%)	29.752 ms	354.52 MiB (1%)	2146767
`["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]`	1.383 s (5%)	24.310 ms	344.77 MiB (1%)	2107074
`["ecc", "evaluate_decoder", "toric8_pybp_comm"]`	456.334 ms (5%)		8.11 MiB (1%)	163696
`["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]`	965.308 ms (5%)		18.04 MiB (1%)	360388
`["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]`	983.870 ms (5%)		20.56 MiB (1%)	392802
`["ecc", "evaluate_decoder", "toric8_pymatch_comm"]`	4.262 ms (5%)		1.95 MiB (1%)	23176
`["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]`	12.409 ms (5%)		5.58 MiB (1%)	79355
`["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]`	20.895 ms (5%)		8.09 MiB (1%)	111769
`["ecc", "evaluate_decoder", "toric8_table_comm"]`	2.626 ms (5%)		1.64 MiB (1%)	23082
`["ecc", "evaluate_decoder", "toric8_table_naivesyn"]`	10.054 ms (5%)		4.93 MiB (1%)	79160
`["ecc", "evaluate_decoder", "toric8_table_shorsyn"]`	18.498 ms (5%)		7.44 MiB (1%)	111574
`["pauli", "mul", "100"]`	14.526 ns (5%)
`["pauli", "mul", "1000"]`	18.820 ns (5%)
`["pauli", "mul", "100000"]`	728.864 ns (5%)
`["pauli", "mul", "20000000"]`	183.152 μs (5%)
`["stabilizer", "canon", "cano500"]`	3.078 ms (5%)
`["stabilizer", "canon", "diag_cano500"]`	709.035 μs (5%)
`["stabilizer", "canon", "diag_gott500"]`	4.037 ms (5%)		5.19 MiB (1%)	34521
`["stabilizer", "canon", "diag_rref500"]`	627.144 μs (5%)
`["stabilizer", "canon", "gott500"]`	6.392 ms (5%)		5.19 MiB (1%)	34531
`["stabilizer", "canon", "md_cano500"]`	1.537 ms (5%)
`["stabilizer", "canon", "md_rref500"]`	1.555 ms (5%)
`["stabilizer", "canon", "rref500"]`	3.119 ms (5%)
`["stabilizer", "project", "destabilizer"]`	17.944 μs (5%)		288 bytes (1%)	5
`["stabilizer", "project", "stabilizer"]`	9.808 μs (5%)		80 bytes (1%)	2
`["stabilizer", "tensor", "diag_pow5_20"]`	1.254 ms (5%)		23.97 MiB (1%)	34
`["stabilizer", "tensor", "pow5_20"]`	2.849 μs (5%)		6.44 KiB (1%)	31
`["stabilizer", "trace", "destabilizer"]`	31.048 μs (5%)		80 bytes (1%)	2
`["stabilizer", "trace", "stabilizer"]`	24.206 μs (5%)		112 bytes (1%)	3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["circuitsim", "mctrajectories"]
["circuitsim", "mctrajectories_sumtype"]
["circuitsim", "mctrajectories_union"]
["circuitsim", "pftrajectories"]
["circuitsim", "pftrajectories_sumtype"]
["circuitsim", "pftrajectories_union"]
["clifford", "dense"]
["clifford", "symbolic"]
["ecc", "evaluate_decoder"]
["pauli", "mul"]
["stabilizer", "canon"]
["stabilizer", "project"]
["stabilizer", "tensor"]
["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 3222 s 0 s 378 s 11328 s 0 s
 #2 0 MHz 2942 s 0 s 389 s 11600 s 0 s
 #3 0 MHz 4166 s 0 s 400 s 10358 s 0 s
 #4 0 MHz 3718 s 0 s 405 s 10804 s 0 s
 Memory: 15.606494903564453 GB (13236.34375 MB free)
 Uptime: 1500.63 sec
 Load Avg: 1.0 1.08 0.92
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Runtime information

Runtime Info
BLAS #threads	2
`BLAS.vendor()`	`lbt`
`Sys.CPU_THREADS`	4

lscpu output:

Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 48 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 4 On-line CPU(s) list: 0-3 Vendor ID: AuthenticAMD Model name: AMD EPYC 7763 64-Core Processor CPU family: 25 Model: 1 Thread(s) per core: 2 Core(s) per socket: 2 Socket(s): 1 Stepping: 1 BogoMIPS: 4890.84 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm Virtualization: AMD-V Hypervisor vendor: Microsoft Virtualization type: full L1d cache: 64 KiB (2 instances) L1i cache: 64 KiB (2 instances) L2 cache: 1 MiB (2 instances) L3 cache: 32 MiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0-3 Vulnerability Gather data sampling: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode Vulnerability Spec store bypass: Vulnerable Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected

Cpu Property	Value
Brand	AMD EPYC 7763 64-Core Processor
Vendor	:AMD
Architecture	:Unknown
Model	Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00
Cores	16 physical cores, 16 logical cores (on executing CPU)
	No Hyperthreading hardware capability detected
Clock Frequencies	Not supported by CPU
Data Cache	Level 1:3 : (32, 512, 32768) kbytes
	64 byte cache line size
Address Size	48 bits virtual, 48 bits physical
SIMD	256 bit = 32 byte max. SIMD vector size
Time Stamp Counter	TSC is accessible via `rdtsc`
	TSC runs at constant rate (invariant from clock frequency)
Perf. Monitoring	Performance Monitoring Counters (PMC) are not supported
Hypervisor	Yes, Microsoft

Benchmark Result

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Time of benchmarks:

Target: 14 Apr 2024 - 13:08
Baseline: 14 Apr 2024 - 13:15

Package commits:

Target: 960242
Baseline: 64f695

Julia commits:

Target: b9aeaf
Baseline: b9aeaf

Julia command flags:

Target: None
Baseline: None

Environment variables:

Target: None
Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with :x:), while a ratio less than 1.0 denotes a possible improvement (marked with :white_check_mark:). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds).

ID	time ratio	memory ratio
`["circuitsim", "mctrajectories_sumtype", "q1001_r1"]`	0.92 (5%) :white_check_mark:	1.00 (1%)
`["clifford", "dense", "cnot_on_dense500_stab"]`	1.06 (5%) :x:	1.00 (1%)
`["clifford", "dense", "dense500_on_diag500_destab"]`	1.15 (5%) :x:	1.00 (1%)
`["ecc", "evaluate_decoder", "shor_bp_shorsyn"]`	0.89 (5%) :white_check_mark:	0.96 (1%) :white_check_mark:
`["ecc", "evaluate_decoder", "shor_pybp_comm"]`	0.93 (5%) :white_check_mark:	1.00 (1%)
`["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]`	1.27 (5%) :x:	1.00 (1%)
`["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]`	1.36 (5%) :x:	1.00 (1%)
`["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]`	1.01 (5%)	0.99 (1%) :white_check_mark:
`["stabilizer", "canon", "diag_cano500"]`	0.94 (5%) :white_check_mark:	1.00 (1%)
`["stabilizer", "canon", "diag_rref500"]`	0.87 (5%) :white_check_mark:	1.00 (1%)
`["stabilizer", "tensor", "diag_pow5_20"]`	0.81 (5%) :white_check_mark:	1.00 (1%)
`["stabilizer", "trace", "destabilizer"]`	0.93 (5%) :white_check_mark:	1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["circuitsim", "mctrajectories"]
["circuitsim", "mctrajectories_sumtype"]
["circuitsim", "mctrajectories_union"]
["circuitsim", "pftrajectories"]
["circuitsim", "pftrajectories_sumtype"]
["circuitsim", "pftrajectories_union"]
["clifford", "dense"]
["clifford", "symbolic"]
["ecc", "evaluate_decoder"]
["pauli", "mul"]
["stabilizer", "canon"]
["stabilizer", "project"]
["stabilizer", "tensor"]
["stabilizer", "trace"]

Julia versioninfo

Target

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2282 s 0 s 294 s 13346 s 0 s
 #2 0 MHz 2785 s 0 s 306 s 12841 s 0 s
 #3 0 MHz 2109 s 0 s 283 s 13529 s 0 s
 #4 0 MHz 2420 s 0 s 296 s 13217 s 0 s
 Memory: 15.606498718261719 GB (13357.84375 MB free)
 Uptime: 1599.32 sec
 Load Avg: 1.06 1.08 0.77
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 3283 s 0 s 381 s 16590 s 0 s
 #2 0 MHz 3967 s 0 s 414 s 15888 s 0 s
 #3 0 MHz 2882 s 0 s 376 s 16992 s 0 s
 #4 0 MHz 3941 s 0 s 392 s 15930 s 0 s
 Memory: 15.606498718261719 GB (13173.26953125 MB free)
 Uptime: 2034.01 sec
 Load Avg: 1.0 1.06 0.91
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Time of benchmark: 14 Apr 2024 - 13:8
Package commit: 960242
Julia commit: b9aeaf
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks. The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero.

ID	time	GC time	memory	allocations
`["circuitsim", "mctrajectories", "q1001_r1"]`	15.031 ms (5%)		500.77 KiB (1%)	18019
`["circuitsim", "mctrajectories", "q101_r1"]`	167.663 μs (5%)		50.53 KiB (1%)	1818
`["circuitsim", "mctrajectories_sumtype", "q1001_r1"]`	13.410 ms (5%)		496 bytes (1%)	9
`["circuitsim", "mctrajectories_sumtype", "q101_r1"]`	118.051 μs (5%)		256 bytes (1%)	8
`["circuitsim", "mctrajectories_union", "q1001_r1"]`	12.953 ms (5%)		496 bytes (1%)	9
`["circuitsim", "mctrajectories_union", "q101_r1"]`	117.139 μs (5%)		256 bytes (1%)	8
`["circuitsim", "pftrajectories", "q1001_r1"]`	59.792 μs (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q1001_r100"]`	173.083 μs (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q1001_r10000"]`	1.098 ms (5%)		93.80 KiB (1%)	2001
`["circuitsim", "pftrajectories", "q101_r1"]`	5.992 μs (5%)		9.42 KiB (1%)	201
`["circuitsim", "pftrajectories_sumtype", "q1001_r1"]`	44.303 μs (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r100"]`	154.218 μs (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]`	1.082 ms (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]`	5.863 ms (5%)		93.84 KiB (1%)	2002
`["circuitsim", "pftrajectories_sumtype", "q101_r1"]`	4.458 μs (5%)		9.47 KiB (1%)	202
`["circuitsim", "pftrajectories_union", "q1001_r1"]`	22.602 μs (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q1001_r100"]`	132.558 μs (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q1001_r10000"]`	1.054 ms (5%)		96 bytes (1%)	2
`["circuitsim", "pftrajectories_union", "q101_r1"]`	2.334 μs (5%)		96 bytes (1%)	2
`["clifford", "dense", "cnot250_on_dense500_destab"]`	11.598 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_dense500_stab"]`	5.740 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_diag500_destab"]`	1.123 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot250_on_diag500_stab"]`	560.608 μs (5%)		512 bytes (1%)	8
`["clifford", "dense", "cnot_on_dense500_destab"]`	51.988 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_dense500_stab"]`	24.546 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_diag500_destab"]`	26.549 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "cnot_on_diag500_stab"]`	13.785 μs (5%)		368 bytes (1%)	10
`["clifford", "dense", "dense500_on_dense500_destab"]`	11.541 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_dense500_stab"]`	5.733 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_diag500_destab"]`	1.123 ms (5%)		512 bytes (1%)	8
`["clifford", "dense", "dense500_on_diag500_stab"]`	560.568 μs (5%)		512 bytes (1%)	8
`["clifford", "symbolic", "cnot250_on_dense500_destab"]`	1.449 ms (5%)
`["clifford", "symbolic", "cnot250_on_dense500_stab"]`	687.045 μs (5%)
`["clifford", "symbolic", "cnot250_on_diag500_destab"]`	1.158 ms (5%)
`["clifford", "symbolic", "cnot250_on_diag500_stab"]`	570.737 μs (5%)
`["clifford", "symbolic", "cnot_on_dense500_destab"]`	4.728 μs (5%)
`["clifford", "symbolic", "cnot_on_dense500_stab"]`	2.304 μs (5%)
`["clifford", "symbolic", "cnot_on_diag500_destab"]`	4.659 μs (5%)
`["clifford", "symbolic", "cnot_on_diag500_stab"]`	2.294 μs (5%)
`["ecc", "evaluate_decoder", "shor_bp_comm"]`	1.798 ms (5%)		1.76 MiB (1%)	42343
`["ecc", "evaluate_decoder", "shor_bp_naivesyn"]`	4.143 ms (5%)		3.62 MiB (1%)	81606
`["ecc", "evaluate_decoder", "shor_bp_shorsyn"]`	4.549 ms (5%)		3.70 MiB (1%)	82622
`["ecc", "evaluate_decoder", "shor_pybp_comm"]`	57.878 ms (5%)		4.76 MiB (1%)	144207
`["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]`	128.694 ms (5%)		9.57 MiB (1%)	284018
`["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]`	171.238 ms (5%)		9.66 MiB (1%)	285104
`["ecc", "evaluate_decoder", "shor_table_comm"]`	256.099 μs (5%)		241.42 KiB (1%)	4593
`["ecc", "evaluate_decoder", "shor_table_naivesyn"]`	868.634 μs (5%)		334.20 KiB (1%)	4790
`["ecc", "evaluate_decoder", "shor_table_shorsyn"]`	1.263 ms (5%)		427.73 KiB (1%)	5876
`["ecc", "evaluate_decoder", "toric8_bp_comm"]`	709.219 ms (5%)	13.053 ms	174.38 MiB (1%)	1045311
`["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]`	1.449 s (5%)	28.275 ms	348.76 MiB (1%)	2112901
`["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]`	1.461 s (5%)	29.249 ms	351.42 MiB (1%)	2146183
`["ecc", "evaluate_decoder", "toric8_pybp_comm"]`	454.695 ms (5%)		8.11 MiB (1%)	163696
`["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]`	944.429 ms (5%)		18.04 MiB (1%)	360388
`["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]`	940.861 ms (5%)		20.56 MiB (1%)	392802
`["ecc", "evaluate_decoder", "toric8_pymatch_comm"]`	4.208 ms (5%)		1.95 MiB (1%)	23176
`["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]`	12.468 ms (5%)		5.58 MiB (1%)	79355
`["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]`	20.956 ms (5%)		8.09 MiB (1%)	111769
`["ecc", "evaluate_decoder", "toric8_table_comm"]`	2.622 ms (5%)		1.64 MiB (1%)	23082
`["ecc", "evaluate_decoder", "toric8_table_naivesyn"]`	10.162 ms (5%)		4.93 MiB (1%)	79160
`["ecc", "evaluate_decoder", "toric8_table_shorsyn"]`	18.722 ms (5%)		7.44 MiB (1%)	111574
`["pauli", "mul", "100"]`	14.416 ns (5%)
`["pauli", "mul", "1000"]`	18.189 ns (5%)
`["pauli", "mul", "100000"]`	722.992 ns (5%)
`["pauli", "mul", "20000000"]`	205.243 μs (5%)
`["stabilizer", "canon", "cano500"]`	3.044 ms (5%)
`["stabilizer", "canon", "diag_cano500"]`	708.715 μs (5%)
`["stabilizer", "canon", "diag_gott500"]`	4.078 ms (5%)		5.19 MiB (1%)	34521
`["stabilizer", "canon", "diag_rref500"]`	627.043 μs (5%)
`["stabilizer", "canon", "gott500"]`	6.438 ms (5%)		5.19 MiB (1%)	34531
`["stabilizer", "canon", "md_cano500"]`	1.533 ms (5%)
`["stabilizer", "canon", "md_rref500"]`	1.556 ms (5%)
`["stabilizer", "canon", "rref500"]`	3.120 ms (5%)
`["stabilizer", "project", "destabilizer"]`	18.585 μs (5%)		288 bytes (1%)	5
`["stabilizer", "project", "stabilizer"]`	8.906 μs (5%)		80 bytes (1%)	2
`["stabilizer", "tensor", "diag_pow5_20"]`	1.116 ms (5%)		23.97 MiB (1%)	34
`["stabilizer", "tensor", "pow5_20"]`	2.805 μs (5%)		6.44 KiB (1%)	31
`["stabilizer", "trace", "destabilizer"]`	31.319 μs (5%)		80 bytes (1%)	2
`["stabilizer", "trace", "stabilizer"]`	24.085 μs (5%)		112 bytes (1%)	3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["circuitsim", "mctrajectories"]
["circuitsim", "mctrajectories_sumtype"]
["circuitsim", "mctrajectories_union"]
["circuitsim", "pftrajectories"]
["circuitsim", "pftrajectories_sumtype"]
["circuitsim", "pftrajectories_union"]
["clifford", "dense"]
["clifford", "symbolic"]
["ecc", "evaluate_decoder"]
["pauli", "mul"]
["stabilizer", "canon"]
["stabilizer", "project"]
["stabilizer", "tensor"]
["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.12.0-DEV.339
Commit b9aeafa171e (2024-04-13 20:07 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2282 s 0 s 294 s 13346 s 0 s
 #2 0 MHz 2785 s 0 s 306 s 12841 s 0 s...*[Comment body truncated]*

Apr 14 '24 11:04 github-actions[bot]

QuantumClifford.jl QuantumClifford.jl copied to clipboard

SteaneReedMuller Code

Codecov Report

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Target

Baseline

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Baseline result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Runtime information

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Target

Baseline

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Baseline result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Runtime information

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Target

Baseline

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

QuantumClifford.jl
QuantumClifford.jl copied to clipboard