QuantumClifford.jl
QuantumClifford.jl copied to clipboard
QuantumSavory
`[2ᵐ + 1 - 3, k, 2ᵐ ⁺ ¹ - 3 - k]` shortenedMDS code
- Adding Reed Solomon codes to the ECC Module
- Adding the tests for the reed module
TODO: convert Generator Matrix for Reed Muller codes to Parity check Matrix Add proper references
Edit: Maybe the todo task is not required since in the literature, RS codes are defined in by generating their generator matrix.
Why RS codes? Reed Solomon codes are important subroutines for advanced Quantum codes, this subroutine is going to be helpful when defining advanced QECCs. This PR is based to address that goal.
Classical Reed-Solomon codes are standardly defined in terms of generating their generator matrix, so added a function for generator matrix and also added a generator_matrix function in the ECC.jl
Edit: More precisely, it's code generator polynomial
For documentation, these resources for consultation :
- https://codyplanteen.com/assets/rs/gf256_prim.pdf
- https://codyplanteen.com/notes/rs
- https://core.ac.uk/download/pdf/16697418.pdf
More specifically, Reed-Solomon codes: RS(255, 223) is the common instance that is used in many places
Is this able to generate all RS codes or is there some limitation on the size? It seems 255 is hardcodded as the largest number of physical bits possible.
If I understand correctly, an RS code is unique defined by three integers that have to obey k<n<q (encoded message size, physical message size, and the order of the group we use to generate the code). q has to also obey q = 2^p.
Here q is fixed to 256 so only a particular set of RS codes is available, right?
Is this able to generate all RS codes or is there some limitation on the size? It seems 255 is hardcodded as the largest number of physical bits possible.
Nope, this is only one instance of the ReedSolomon codes namely RS(255, 233). This can be easily extended to include all RS codes though.
Correction: 223, instead of 233 RS(255, 223), where GF(2^8)
I would like to refer a youtube lecture for consultation, That really helped me in understanding math and concept of ReedSolomon codes for consulation when adding papers. Is it okay to add it in the docstring along with papers ?
Link: https://youtu.be/K26Ssr8H3ec?si=DOai-gnjveQMaq-p (Pedagogical lecture)
Improvements:
- I have added three helper functions that people can use to play with RS codes. We are only concerned with generator polynomial, but for pedagogical purposes, one can create a custom message polynomial , and then corresponding parity check polynomial and codeword polynomials.
- Added all known examples in literature for Reed Solmon codes for testing purposes.
- using only Nemo for generation of Reed-Solomon codes
Improvements:
- Adding classical BCH codes as well. These codes are fully implemented in Nemo as well.
- Tested the universal property of BCH codes as well!
- Tested locally as well.
It is impressive that the BCHs are also added now!
Just a quick advice on structuring PRs like this to make it easier to review. Here for instances, BCH is something sufficiently separate from the original contribution. It is a good idea to split such things into two PRs that can depend on each other, something like:
> git checkout -b first_contribution # create a branch for the initial work
> ... do all the implementation work and local testing
> git add .
> git commit -m "implementing first contribution"
> git push # make a pull request
> git branch -b second_contribution # create a branch on top of your current contribution
> ... do all the implementation work and local testing for the second contribution
> git add .
> git commit -m "implementing second contribution that depends on first contribution"
> git push # make a second pull request that depends on the first pull request being merged
This makes reviewing drastically easier as the reviewer needs to keep only one thing in their head. But the contributor needs to be careful to make sure that modifications they make to the first PR get carried over (they need to rebase the second one on top of the first one occasionally).
No need to make any such changes here. Just a note for future reference.
No need to make any such changes here. Just a note for future reference.
Dear Professor, Thank you for this advice and new information! I will keep this in mind for future reference.
Benchmark Result
Judge result
Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl
Job Properties
- Time of benchmarks:
- Target: 9 Apr 2024 - 04:04
- Baseline: 9 Apr 2024 - 04:11
- Package commits:
- Target: 4774a8
- Baseline: 6c184a
- Julia commits:
- Target: 0e28cf
- Baseline: 0e28cf
- Julia command flags:
- Target: None
- Baseline: None
- Environment variables:
- Target: None
- Baseline: None
Results
A ratio greater than 1.0 denotes a possible regression (marked with :x:), while a ratio less
than 1.0 denotes a possible improvement (marked with :white_check_mark:). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).
| ID | time ratio | memory ratio |
|---|---|---|
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] |
0.94 (5%) :white_check_mark: | 1.00 (1%) |
["clifford", "dense", "cnot250_on_diag500_stab"] |
1.14 (5%) :x: | 1.00 (1%) |
["clifford", "dense", "dense500_on_diag500_stab"] |
1.14 (5%) :x: | 1.00 (1%) |
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] |
0.58 (5%) :white_check_mark: | 0.98 (1%) :white_check_mark: |
["ecc", "evaluate_decoder", "shor_pybp_comm"] |
1.36 (5%) :x: | 1.00 (1%) |
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] |
0.95 (5%) :white_check_mark: | 0.98 (1%) :white_check_mark: |
["pauli", "mul", "100"] |
1.06 (5%) :x: | 1.00 (1%) |
["pauli", "mul", "20000000"] |
1.08 (5%) :x: | 1.00 (1%) |
["stabilizer", "canon", "diag_rref500"] |
0.94 (5%) :white_check_mark: | 1.00 (1%) |
["stabilizer", "tensor", "diag_pow5_20"] |
0.93 (5%) :white_check_mark: | 1.00 (1%) |
["stabilizer", "trace", "destabilizer"] |
0.91 (5%) :white_check_mark: | 1.00 (1%) |
Benchmark Group List
Here's a list of all the benchmark groups executed by this job:
["circuitsim", "mctrajectories"]["circuitsim", "mctrajectories_sumtype"]["circuitsim", "mctrajectories_union"]["circuitsim", "pftrajectories"]["circuitsim", "pftrajectories_sumtype"]["circuitsim", "pftrajectories_union"]["clifford", "dense"]["clifford", "symbolic"]["ecc", "evaluate_decoder"]["pauli", "mul"]["stabilizer", "canon"]["stabilizer", "project"]["stabilizer", "tensor"]["stabilizer", "trace"]
Julia versioninfo
Target
Julia Version 1.12.0-DEV.317
Commit 0e28cf6abff (2024-04-08 10:46 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
Ubuntu 22.04.4 LTS
uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
CPU: AMD EPYC 7763 64-Core Processor:
speed user nice sys idle irq
#1 0 MHz 2612 s 0 s 277 s 5786 s 0 s
#2 0 MHz 2383 s 0 s 320 s 5950 s 0 s
#3 0 MHz 2441 s 0 s 269 s 5967 s 0 s
#4 0 MHz 1973 s 0 s 269 s 6439 s 0 s
Memory: 15.606498718261719 GB (13386.1953125 MB free)
Uptime: 873.47 sec
Load Avg: 1.0 1.11 0.82
WORD_SIZE: 64
LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)
Baseline
Julia Version 1.12.0-DEV.317
Commit 0e28cf6abff (2024-04-08 10:46 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
Ubuntu 22.04.4 LTS
uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
CPU: AMD EPYC 7763 64-Core Processor:
speed user nice sys idle irq
#1 0 MHz 4014 s 0 s 363 s 8578 s 0 s
#2 0 MHz 3221 s 0 s 402 s 9304 s 0 s
#3 0 MHz 3522 s 0 s 378 s 9050 s 0 s
#4 0 MHz 3071 s 0 s 375 s 9505 s 0 s
Memory: 15.606498718261719 GB (13231.93359375 MB free)
Uptime: 1302.32 sec
Load Avg: 1.02 1.09 0.94
WORD_SIZE: 64
LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)
Target result
Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl
Job Properties
- Time of benchmark: 9 Apr 2024 - 4:4
- Package commit: 4774a8
- Julia commit: 0e28cf
- Julia command flags: None
- Environment variables: None
Results
Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.
| ID | time | GC time | memory | allocations |
|---|---|---|---|---|
["circuitsim", "mctrajectories", "q1001_r1"] |
14.971 ms (5%) | 500.77 KiB (1%) | 18019 | |
["circuitsim", "mctrajectories", "q101_r1"] |
170.839 μs (5%) | 50.53 KiB (1%) | 1818 | |
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] |
13.095 ms (5%) | 496 bytes (1%) | 9 | |
["circuitsim", "mctrajectories_sumtype", "q101_r1"] |
117.589 μs (5%) | 256 bytes (1%) | 8 | |
["circuitsim", "mctrajectories_union", "q1001_r1"] |
13.001 ms (5%) | 496 bytes (1%) | 9 | |
["circuitsim", "mctrajectories_union", "q101_r1"] |
116.407 μs (5%) | 256 bytes (1%) | 8 | |
["circuitsim", "pftrajectories", "q1001_r1"] |
61.806 μs (5%) | 93.80 KiB (1%) | 2001 | |
["circuitsim", "pftrajectories", "q1001_r100"] |
174.234 μs (5%) | 93.80 KiB (1%) | 2001 | |
["circuitsim", "pftrajectories", "q1001_r10000"] |
1.098 ms (5%) | 93.80 KiB (1%) | 2001 | |
["circuitsim", "pftrajectories", "q101_r1"] |
6.201 μs (5%) | 9.42 KiB (1%) | 201 | |
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] |
44.713 μs (5%) | 93.84 KiB (1%) | 2002 | |
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] |
156.391 μs (5%) | 93.84 KiB (1%) | 2002 | |
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] |
1.085 ms (5%) | 93.84 KiB (1%) | 2002 | |
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] |
5.872 ms (5%) | 93.84 KiB (1%) | 2002 | |
["circuitsim", "pftrajectories_sumtype", "q101_r1"] |
4.488 μs (5%) | 9.47 KiB (1%) | 202 | |
["circuitsim", "pftrajectories_union", "q1001_r1"] |
22.611 μs (5%) | 96 bytes (1%) | 2 | |
["circuitsim", "pftrajectories_union", "q1001_r100"] |
134.300 μs (5%) | 96 bytes (1%) | 2 | |
["circuitsim", "pftrajectories_union", "q1001_r10000"] |
1.057 ms (5%) | 96 bytes (1%) | 2 | |
["circuitsim", "pftrajectories_union", "q101_r1"] |
2.334 μs (5%) | 96 bytes (1%) | 2 | |
["clifford", "dense", "cnot250_on_dense500_destab"] |
11.616 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "cnot250_on_dense500_stab"] |
5.801 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "cnot250_on_diag500_destab"] |
1.121 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "cnot250_on_diag500_stab"] |
560.604 μs (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "cnot_on_dense500_destab"] |
51.095 μs (5%) | 368 bytes (1%) | 10 | |
["clifford", "dense", "cnot_on_dense500_stab"] |
24.175 μs (5%) | 368 bytes (1%) | 10 | |
["clifford", "dense", "cnot_on_diag500_destab"] |
26.339 μs (5%) | 368 bytes (1%) | 10 | |
["clifford", "dense", "cnot_on_diag500_stab"] |
13.566 μs (5%) | 368 bytes (1%) | 10 | |
["clifford", "dense", "dense500_on_dense500_destab"] |
11.606 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "dense500_on_dense500_stab"] |
5.787 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "dense500_on_diag500_destab"] |
980.276 μs (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "dense500_on_diag500_stab"] |
560.614 μs (5%) | 512 bytes (1%) | 8 | |
["clifford", "symbolic", "cnot250_on_dense500_destab"] |
1.436 ms (5%) | |||
["clifford", "symbolic", "cnot250_on_dense500_stab"] |
686.589 μs (5%) | |||
["clifford", "symbolic", "cnot250_on_diag500_destab"] |
1.161 ms (5%) | |||
["clifford", "symbolic", "cnot250_on_diag500_stab"] |
569.611 μs (5%) | |||
["clifford", "symbolic", "cnot_on_dense500_destab"] |
4.749 μs (5%) | |||
["clifford", "symbolic", "cnot_on_dense500_stab"] |
2.294 μs (5%) | |||
["clifford", "symbolic", "cnot_on_diag500_destab"] |
4.648 μs (5%) | |||
["clifford", "symbolic", "cnot_on_diag500_stab"] |
2.304 μs (5%) | |||
["ecc", "evaluate_decoder", "shor_bp_comm"] |
1.801 ms (5%) | 1.77 MiB (1%) | 42581 | |
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] |
4.030 ms (5%) | 3.61 MiB (1%) | 81732 | |
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] |
4.465 ms (5%) | 3.71 MiB (1%) | 82846 | |
["ecc", "evaluate_decoder", "shor_pybp_comm"] |
76.985 ms (5%) | 4.76 MiB (1%) | 144207 | |
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] |
149.934 ms (5%) | 9.57 MiB (1%) | 284018 | |
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] |
134.003 ms (5%) | 9.66 MiB (1%) | 285104 | |
["ecc", "evaluate_decoder", "shor_table_comm"] |
255.566 μs (5%) | 241.42 KiB (1%) | 4593 | |
["ecc", "evaluate_decoder", "shor_table_naivesyn"] |
863.218 μs (5%) | 334.20 KiB (1%) | 4790 | |
["ecc", "evaluate_decoder", "shor_table_shorsyn"] |
1.266 ms (5%) | 427.73 KiB (1%) | 5876 | |
["ecc", "evaluate_decoder", "toric8_bp_comm"] |
693.079 ms (5%) | 13.872 ms | 176.03 MiB (1%) | 1055020 |
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] |
1.409 s (5%) | 30.180 ms | 348.89 MiB (1%) | 2113706 |
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] |
1.365 s (5%) | 30.497 ms | 349.76 MiB (1%) | 2136439 |
["ecc", "evaluate_decoder", "toric8_pybp_comm"] |
469.783 ms (5%) | 8.11 MiB (1%) | 163696 | |
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] |
958.821 ms (5%) | 18.04 MiB (1%) | 360388 | |
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] |
965.414 ms (5%) | 20.56 MiB (1%) | 392802 | |
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] |
4.251 ms (5%) | 1.95 MiB (1%) | 23176 | |
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] |
12.345 ms (5%) | 5.58 MiB (1%) | 79355 | |
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] |
20.777 ms (5%) | 8.09 MiB (1%) | 111769 | |
["ecc", "evaluate_decoder", "toric8_table_comm"] |
2.631 ms (5%) | 1.64 MiB (1%) | 23082 | |
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] |
10.047 ms (5%) | 4.93 MiB (1%) | 79160 | |
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] |
18.564 ms (5%) | 7.44 MiB (1%) | 111574 | |
["pauli", "mul", "100"] |
15.319 ns (5%) | |||
["pauli", "mul", "1000"] |
17.899 ns (5%) | |||
["pauli", "mul", "100000"] |
719.535 ns (5%) | |||
["pauli", "mul", "20000000"] |
193.591 μs (5%) | |||
["stabilizer", "canon", "cano500"] |
3.095 ms (5%) | |||
["stabilizer", "canon", "diag_cano500"] |
752.351 μs (5%) | |||
["stabilizer", "canon", "diag_gott500"] |
4.201 ms (5%) | 5.19 MiB (1%) | 34521 | |
["stabilizer", "canon", "diag_rref500"] |
626.838 μs (5%) | |||
["stabilizer", "canon", "gott500"] |
6.576 ms (5%) | 5.19 MiB (1%) | 34531 | |
["stabilizer", "canon", "md_cano500"] |
1.541 ms (5%) | |||
["stabilizer", "canon", "md_rref500"] |
1.577 ms (5%) | |||
["stabilizer", "canon", "rref500"] |
3.116 ms (5%) | |||
["stabilizer", "project", "destabilizer"] |
18.064 μs (5%) | 288 bytes (1%) | 5 | |
["stabilizer", "project", "stabilizer"] |
9.057 μs (5%) | 80 bytes (1%) | 2 | |
["stabilizer", "tensor", "diag_pow5_20"] |
1.292 ms (5%) | 23.97 MiB (1%) | 34 | |
["stabilizer", "tensor", "pow5_20"] |
2.754 μs (5%) | 6.44 KiB (1%) | 31 | |
["stabilizer", "trace", "destabilizer"] |
31.239 μs (5%) | 80 bytes (1%) | 2 | |
["stabilizer", "trace", "stabilizer"] |
24.205 μs (5%) | 112 bytes (1%) | 3 |
Benchmark Group List
Here's a list of all the benchmark groups executed by this job:
["circuitsim", "mctrajectories"]["circuitsim", "mctrajectories_sumtype"]["circuitsim", "mctrajectories_union"]["circuitsim", "pftrajectories"]["circuitsim", "pftrajectories_sumtype"]["circuitsim", "pftrajectories_union"]["clifford", "dense"]["clifford", "symbolic"]["ecc", "evaluate_decoder"]["pauli", "mul"]["stabilizer", "canon"]["stabilizer", "project"]["stabilizer", "tensor"]["stabilizer", "trace"]
Julia versioninfo
Julia Version 1.12.0-DEV.317
Commit 0e28cf6abff (2024-04-08 10:46 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
Ubuntu 22.04.4 LTS
uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
CPU: AMD EPYC 7763 64-Core Processor:
speed user nice sys idle irq
#1 0 MHz 2612 s 0 s 277 s 5786 s 0 s
#2 0 MHz 2383 s 0 s 320 s 5950 s 0 s
#3 0 MHz 2441 s 0 s 269 s 5967 s 0 s
#4 0 MHz 1973 s 0 s 269 s 6439 s 0 s
Memory: 15.606498718261719 GB (13386.1953125 MB free)
Uptime: 873.47 sec
Load Avg: 1.0 1.11 0.82
WORD_SIZE: 64
LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)
Baseline result
Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl
Job Properties
- Time of benchmark: 9 Apr 2024 - 4:11
- Package commit: 6c184a
- Julia commit: 0e28cf
- Julia command flags: None
- Environment variables: None
Results
Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.
| ID | time | GC time | memory | allocations |
|---|---|---|---|---|
["circuitsim", "mctrajectories", "q1001_r1"] |
15.326 ms (5%) | 500.77 KiB (1%) | 18019 | |
["circuitsim", "mctrajectories", "q101_r1"] |
173.193 μs (5%) | 50.53 KiB (1%) | 1818 | |
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] |
13.936 ms (5%) | 496 bytes (1%) | 9 | |
["circuitsim", "mctrajectories_sumtype", "q101_r1"] |
118.801 μs (5%) | 256 bytes (1%) | 8 | |
["circuitsim", "mctrajectories_union", "q1001_r1"] |
13.592 ms (5%) | 496 bytes (1%) | 9 | |
["circuitsim", "mctrajectories_union", "q101_r1"] |
117.169 μs (5%) | 256 bytes (1%) | 8 | |
["circuitsim", "pftrajectories", "q1001_r1"] |
63.929 μs (5%) | 93.80 KiB (1%) | 2001 | |
["circuitsim", "pftrajectories", "q1001_r100"] |
175.156 μs (5%) | 93.80 KiB (1%) | 2001 | |
["circuitsim", "pftrajectories", "q1001_r10000"] |
1.102 ms (5%) | 93.80 KiB (1%) | 2001 | |
["circuitsim", "pftrajectories", "q101_r1"] |
6.421 μs (5%) | 9.42 KiB (1%) | 201 | |
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] |
45.114 μs (5%) | 93.84 KiB (1%) | 2002 | |
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] |
155.459 μs (5%) | 93.84 KiB (1%) | 2002 | |
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] |
1.084 ms (5%) | 93.84 KiB (1%) | 2002 | |
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] |
5.875 ms (5%) | 93.84 KiB (1%) | 2002 | |
["circuitsim", "pftrajectories_sumtype", "q101_r1"] |
4.478 μs (5%) | 9.47 KiB (1%) | 202 | |
["circuitsim", "pftrajectories_union", "q1001_r1"] |
23.514 μs (5%) | 96 bytes (1%) | 2 | |
["circuitsim", "pftrajectories_union", "q1001_r100"] |
132.898 μs (5%) | 96 bytes (1%) | 2 | |
["circuitsim", "pftrajectories_union", "q1001_r10000"] |
1.054 ms (5%) | 96 bytes (1%) | 2 | |
["circuitsim", "pftrajectories_union", "q101_r1"] |
2.424 μs (5%) | 96 bytes (1%) | 2 | |
["clifford", "dense", "cnot250_on_dense500_destab"] |
11.601 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "cnot250_on_dense500_stab"] |
5.799 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "cnot250_on_diag500_destab"] |
1.123 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "cnot250_on_diag500_stab"] |
490.344 μs (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "cnot_on_dense500_destab"] |
51.315 μs (5%) | 368 bytes (1%) | 10 | |
["clifford", "dense", "cnot_on_dense500_stab"] |
24.435 μs (5%) | 368 bytes (1%) | 10 | |
["clifford", "dense", "cnot_on_diag500_destab"] |
26.289 μs (5%) | 368 bytes (1%) | 10 | |
["clifford", "dense", "cnot_on_diag500_stab"] |
13.715 μs (5%) | 368 bytes (1%) | 10 | |
["clifford", "dense", "dense500_on_dense500_destab"] |
11.581 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "dense500_on_dense500_stab"] |
5.786 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "dense500_on_diag500_destab"] |
980.026 μs (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "dense500_on_diag500_stab"] |
490.223 μs (5%) | 512 bytes (1%) | 8 | |
["clifford", "symbolic", "cnot250_on_dense500_destab"] |
1.435 ms (5%) | |||
["clifford", "symbolic", "cnot250_on_dense500_stab"] |
688.002 μs (5%) | |||
["clifford", "symbolic", "cnot250_on_diag500_destab"] |
1.161 ms (5%) | |||
["clifford", "symbolic", "cnot250_on_diag500_stab"] |
570.193 μs (5%) | |||
["clifford", "symbolic", "cnot_on_dense500_destab"] |
4.739 μs (5%) | |||
["clifford", "symbolic", "cnot_on_dense500_stab"] |
2.304 μs (5%) | |||
["clifford", "symbolic", "cnot_on_diag500_destab"] |
4.679 μs (5%) | |||
["clifford", "symbolic", "cnot_on_diag500_stab"] |
2.284 μs (5%) | |||
["ecc", "evaluate_decoder", "shor_bp_comm"] |
1.839 ms (5%) | 1.77 MiB (1%) | 42595 | |
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] |
4.029 ms (5%) | 3.60 MiB (1%) | 81368 | |
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] |
7.748 ms (5%) | 3.77 MiB (1%) | 84386 | |
["ecc", "evaluate_decoder", "shor_pybp_comm"] |
56.759 ms (5%) | 4.76 MiB (1%) | 144207 | |
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] |
150.946 ms (5%) | 9.57 MiB (1%) | 284018 | |
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] |
131.820 ms (5%) | 9.66 MiB (1%) | 285104 | |
["ecc", "evaluate_decoder", "shor_table_comm"] |
255.095 μs (5%) | 241.42 KiB (1%) | 4593 | |
["ecc", "evaluate_decoder", "shor_table_naivesyn"] |
869.139 μs (5%) | 334.20 KiB (1%) | 4790 | |
["ecc", "evaluate_decoder", "shor_table_shorsyn"] |
1.268 ms (5%) | 427.73 KiB (1%) | 5876 | |
["ecc", "evaluate_decoder", "toric8_bp_comm"] |
700.430 ms (5%) | 14.309 ms | 174.65 MiB (1%) | 1046935 |
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] |
1.382 s (5%) | 26.514 ms | 347.37 MiB (1%) | 2104753 |
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] |
1.438 s (5%) | 29.790 ms | 357.00 MiB (1%) | 2179020 |
["ecc", "evaluate_decoder", "toric8_pybp_comm"] |
469.326 ms (5%) | 8.11 MiB (1%) | 163696 | |
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] |
986.061 ms (5%) | 18.04 MiB (1%) | 360388 | |
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] |
987.792 ms (5%) | 20.56 MiB (1%) | 392802 | |
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] |
4.210 ms (5%) | 1.95 MiB (1%) | 23176 | |
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] |
12.502 ms (5%) | 5.58 MiB (1%) | 79355 | |
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] |
21.254 ms (5%) | 8.09 MiB (1%) | 111769 | |
["ecc", "evaluate_decoder", "toric8_table_comm"] |
2.599 ms (5%) | 1.64 MiB (1%) | 23082 | |
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] |
10.127 ms (5%) | 4.93 MiB (1%) | 79160 | |
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] |
18.735 ms (5%) | 7.44 MiB (1%) | 111574 | |
["pauli", "mul", "100"] |
14.516 ns (5%) | |||
["pauli", "mul", "1000"] |
18.832 ns (5%) | |||
["pauli", "mul", "100000"] |
732.071 ns (5%) | |||
["pauli", "mul", "20000000"] |
179.685 μs (5%) | |||
["stabilizer", "canon", "cano500"] |
3.088 ms (5%) | |||
["stabilizer", "canon", "diag_cano500"] |
752.430 μs (5%) | |||
["stabilizer", "canon", "diag_gott500"] |
4.030 ms (5%) | 5.19 MiB (1%) | 34521 | |
["stabilizer", "canon", "diag_rref500"] |
665.520 μs (5%) | |||
["stabilizer", "canon", "gott500"] |
6.379 ms (5%) | 5.19 MiB (1%) | 34531 | |
["stabilizer", "canon", "md_cano500"] |
1.611 ms (5%) | |||
["stabilizer", "canon", "md_rref500"] |
1.625 ms (5%) | |||
["stabilizer", "canon", "rref500"] |
3.138 ms (5%) | |||
["stabilizer", "project", "destabilizer"] |
18.344 μs (5%) | 288 bytes (1%) | 5 | |
["stabilizer", "project", "stabilizer"] |
9.047 μs (5%) | 80 bytes (1%) | 2 | |
["stabilizer", "tensor", "diag_pow5_20"] |
1.394 ms (5%) | 23.97 MiB (1%) | 34 | |
["stabilizer", "tensor", "pow5_20"] |
2.676 μs (5%) | 6.44 KiB (1%) | 31 | |
["stabilizer", "trace", "destabilizer"] |
34.274 μs (5%) | 80 bytes (1%) | 2 | |
["stabilizer", "trace", "stabilizer"] |
24.495 μs (5%) | 112 bytes (1%) | 3 |
Benchmark Group List
Here's a list of all the benchmark groups executed by this job:
["circuitsim", "mctrajectories"]["circuitsim", "mctrajectories_sumtype"]["circuitsim", "mctrajectories_union"]["circuitsim", "pftrajectories"]["circuitsim", "pftrajectories_sumtype"]["circuitsim", "pftrajectories_union"]["clifford", "dense"]["clifford", "symbolic"]["ecc", "evaluate_decoder"]["pauli", "mul"]["stabilizer", "canon"]["stabilizer", "project"]["stabilizer", "tensor"]["stabilizer", "trace"]
Julia versioninfo
Julia Version 1.12.0-DEV.317
Commit 0e28cf6abff (2024-04-08 10:46 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
Ubuntu 22.04.4 LTS
uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
CPU: AMD EPYC 7763 64-Core Processor:
speed user nice sys idle irq
#1 0 MHz 4014 s 0 s 363 s 8578 s 0 s
#2 0 MHz 3221 s 0 s 402 s 9304 s 0 s
#3 0 MHz 3522 s 0 s 378 s 9050 s 0 s
#4 0 MHz 3071 s 0 s 375 s 9505 s 0 s
Memory: 15.606498718261719 GB (13231.93359375 MB free)
Uptime: 1302.32 sec
Load Avg: 1.02 1.09 0.94
WORD_SIZE: 64
LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)
Runtime information
| Runtime Info | |
|---|---|
| BLAS #threads | 2 |
BLAS.vendor() |
lbt |
Sys.CPU_THREADS |
4 |
lscpu output:
Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 48 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 4 On-line CPU(s) list: 0-3 Vendor ID: AuthenticAMD Model name: AMD EPYC 7763 64-Core Processor CPU family: 25 Model: 1 Thread(s) per core: 2 Core(s) per socket: 2 Socket(s): 1 Stepping: 1 BogoMIPS: 4890.86 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm Virtualization: AMD-V Hypervisor vendor: Microsoft Virtualization type: full L1d cache: 64 KiB (2 instances) L1i cache: 64 KiB (2 instances) L2 cache: 1 MiB (2 instances) L3 cache: 32 MiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0-3 Vulnerability Gather data sampling: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode Vulnerability Spec store bypass: Vulnerable Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected
| Cpu Property | Value |
|---|---|
| Brand | AMD EPYC 7763 64-Core Processor |
| Vendor | :AMD |
| Architecture | :Unknown |
| Model | Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00 |
| Cores | 16 physical cores, 16 logical cores (on executing CPU) |
| No Hyperthreading hardware capability detected | |
| Clock Frequencies | Not supported by CPU |
| Data Cache | Level 1:3 : (32, 512, 32768) kbytes |
| 64 byte cache line size | |
| Address Size | 48 bits virtual, 48 bits physical |
| SIMD | 256 bit = 32 byte max. SIMD vector size |
| Time Stamp Counter | TSC is accessible via rdtsc |
| TSC runs at constant rate (invariant from clock frequency) | |
| Perf. Monitoring | Performance Monitoring Counters (PMC) are not supported |
| Hypervisor | Yes, Microsoft |
Benchmark Result
Judge result
Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl
Job Properties
- Time of benchmarks:
- Target: 9 Apr 2024 - 04:04
- Baseline: 9 Apr 2024 - 04:11
- Package commits:
- Target: 4774a8
- Baseline: 6c184a
- Julia commits:
- Target: 0e28cf
- Baseline: 0e28cf
- Julia command flags:
- Target: None
- Baseline: None
- Environment variables:
- Target: None
- Baseline: None
Results
A ratio greater than 1.0 denotes a possible regression (marked with :x:), while a ratio less
than 1.0 denotes a possible improvement (marked with :white_check_mark:). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).
| ID | time ratio | memory ratio |
|---|---|---|
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] |
0.94 (5%) :white_check_mark: | 1.00 (1%) |
["clifford", "dense", "cnot250_on_diag500_stab"] |
1.14 (5%) :x: | 1.00 (1%) |
["clifford", "dense", "dense500_on_diag500_stab"] |
1.14 (5%) :x: | 1.00 (1%) |
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] |
0.58 (5%) :white_check_mark: | 0.98 (1%) :white_check_mark: |
["ecc", "evaluate_decoder", "shor_pybp_comm"] |
1.36 (5%) :x: | 1.00 (1%) |
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] |
0.95 (5%) :white_check_mark: | 0.98 (1%) :white_check_mark: |
["pauli", "mul", "100"] |
1.06 (5%) :x: | 1.00 (1%) |
["pauli", "mul", "20000000"] |
1.08 (5%) :x: | 1.00 (1%) |
["stabilizer", "canon", "diag_rref500"] |
0.94 (5%) :white_check_mark: | 1.00 (1%) |
["stabilizer", "tensor", "diag_pow5_20"] |
0.93 (5%) :white_check_mark: | 1.00 (1%) |
["stabilizer", "trace", "destabilizer"] |
0.91 (5%) :white_check_mark: | 1.00 (1%) |
Benchmark Group List
Here's a list of all the benchmark groups executed by this job:
["circuitsim", "mctrajectories"]["circuitsim", "mctrajectories_sumtype"]["circuitsim", "mctrajectories_union"]["circuitsim", "pftrajectories"]["circuitsim", "pftrajectories_sumtype"]["circuitsim", "pftrajectories_union"]["clifford", "dense"]["clifford", "symbolic"]["ecc", "evaluate_decoder"]["pauli", "mul"]["stabilizer", "canon"]["stabilizer", "project"]["stabilizer", "tensor"]["stabilizer", "trace"]
Julia versioninfo
Target
Julia Version 1.12.0-DEV.317
Commit 0e28cf6abff (2024-04-08 10:46 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
Ubuntu 22.04.4 LTS
uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
CPU: AMD EPYC 7763 64-Core Processor:
speed user nice sys idle irq
#1 0 MHz 2612 s 0 s 277 s 5786 s 0 s
#2 0 MHz 2383 s 0 s 320 s 5950 s 0 s
#3 0 MHz 2441 s 0 s 269 s 5967 s 0 s
#4 0 MHz 1973 s 0 s 269 s 6439 s 0 s
Memory: 15.606498718261719 GB (13386.1953125 MB free)
Uptime: 873.47 sec
Load Avg: 1.0 1.11 0.82
WORD_SIZE: 64
LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)
Baseline
Julia Version 1.12.0-DEV.317
Commit 0e28cf6abff (2024-04-08 10:46 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
Ubuntu 22.04.4 LTS
uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
CPU: AMD EPYC 7763 64-Core Processor:
speed user nice sys idle irq
#1 0 MHz 4014 s 0 s 363 s 8578 s 0 s
#2 0 MHz 3221 s 0 s 402 s 9304 s 0 s
#3 0 MHz 3522 s 0 s 378 s 9050 s 0 s
#4 0 MHz 3071 s 0 s 375 s 9505 s 0 s
Memory: 15.606498718261719 GB (13231.93359375 MB free)
Uptime: 1302.32 sec
Load Avg: 1.02 1.09 0.94
WORD_SIZE: 64
LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)
Target result
Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl
Job Properties
- Time of benchmark: 9 Apr 2024 - 4:4
- Package commit: 4774a8
- Julia commit: 0e28cf
- Julia command flags: None
- Environment variables: None
Results
Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.
| ID | time | GC time | memory | allocations |
|---|---|---|---|---|
["circuitsim", "mctrajectories", "q1001_r1"] |
14.971 ms (5%) | 500.77 KiB (1%) | 18019 | |
["circuitsim", "mctrajectories", "q101_r1"] |
170.839 μs (5%) | 50.53 KiB (1%) | 1818 | |
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] |
13.095 ms (5%) | 496 bytes (1%) | 9 | |
["circuitsim", "mctrajectories_sumtype", "q101_r1"] |
117.589 μs (5%) | 256 bytes (1%) | 8 | |
["circuitsim", "mctrajectories_union", "q1001_r1"] |
13.001 ms (5%) | 496 bytes (1%) | 9 | |
["circuitsim", "mctrajectories_union", "q101_r1"] |
116.407 μs (5%) | 256 bytes (1%) | 8 | |
["circuitsim", "pftrajectories", "q1001_r1"] |
61.806 μs (5%) | 93.80 KiB (1%) | 2001 | |
["circuitsim", "pftrajectories", "q1001_r100"] |
174.234 μs (5%) | 93.80 KiB (1%) | 2001 | |
["circuitsim", "pftrajectories", "q1001_r10000"] |
1.098 ms (5%) | 93.80 KiB (1%) | 2001 | |
["circuitsim", "pftrajectories", "q101_r1"] |
6.201 μs (5%) | 9.42 KiB (1%) | 201 | |
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] |
44.713 μs (5%) | 93.84 KiB (1%) | 2002 | |
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] |
156.391 μs (5%) | 93.84 KiB (1%) | 2002 | |
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] |
1.085 ms (5%) | 93.84 KiB (1%) | 2002 | |
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] |
5.872 ms (5%) | 93.84 KiB (1%) | 2002 | |
["circuitsim", "pftrajectories_sumtype", "q101_r1"] |
4.488 μs (5%) | 9.47 KiB (1%) | 202 | |
["circuitsim", "pftrajectories_union", "q1001_r1"] |
22.611 μs (5%) | 96 bytes (1%) | 2 | |
["circuitsim", "pftrajectories_union", "q1001_r100"] |
134.300 μs (5%) | 96 bytes (1%) | 2 | |
["circuitsim", "pftrajectories_union", "q1001_r10000"] |
1.057 ms (5%) | 96 bytes (1%) | 2 | |
["circuitsim", "pftrajectories_union", "q101_r1"] |
2.334 μs (5%) | 96 bytes (1%) | 2 | |
["clifford", "dense", "cnot250_on_dense500_destab"] |
11.616 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "cnot250_on_dense500_stab"] |
5.801 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "cnot250_on_diag500_destab"] |
1.121 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "cnot250_on_diag500_stab"] |
560.604 μs (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "cnot_on_dense500_destab"] |
51.095 μs (5%) | 368 bytes (1%) | 10 | |
["clifford", "dense", "cnot_on_dense500_stab"] |
24.175 μs (5%) | 368 bytes (1%) | 10 | |
["clifford", "dense", "cnot_on_diag500_destab"] |
26.339 μs (5%) | 368 bytes (1%) | 10 | |
["clifford", "dense", "cnot_on_diag500_stab"] |
13.566 μs (5%) | 368 bytes (1%) | 10 | |
["clifford", "dense", "dense500_on_dense500_destab"] |
11.606 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "dense500_on_dense500_stab"] |
5.787 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "dense500_on_diag500_destab"] |
980.276 μs (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "dense500_on_diag500_stab"] |
560.614 μs (5%) | 512 bytes (1%) | 8 | |
["clifford", "symbolic", "cnot250_on_dense500_destab"] |
1.436 ms (5%) | |||
["clifford", "symbolic", "cnot250_on_dense500_stab"] |
686.589 μs (5%) | |||
["clifford", "symbolic", "cnot250_on_diag500_destab"] |
1.161 ms (5%) | |||
["clifford", "symbolic", "cnot250_on_diag500_stab"] |
569.611 μs (5%) | |||
["clifford", "symbolic", "cnot_on_dense500_destab"] |
4.749 μs (5%) | |||
["clifford", "symbolic", "cnot_on_dense500_stab"] |
2.294 μs (5%) | |||
["clifford", "symbolic", "cnot_on_diag500_destab"] |
4.648 μs (5%) | |||
["clifford", "symbolic", "cnot_on_diag500_stab"] |
2.304 μs (5%) | |||
["ecc", "evaluate_decoder", "shor_bp_comm"] |
1.801 ms (5%) | 1.77 MiB (1%) | 42581 | |
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] |
4.030 ms (5%) | 3.61 MiB (1%) | 81732 | |
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] |
4.465 ms (5%) | 3.71 MiB (1%) | 82846 | |
["ecc", "evaluate_decoder", "shor_pybp_comm"] |
76.985 ms (5%) | 4.76 MiB (1%) | 144207 | |
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] |
149.934 ms (5%) | 9.57 MiB (1%) | 284018 | |
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] |
134.003 ms (5%) | 9.66 MiB (1%) | 285104 | |
["ecc", "evaluate_decoder", "shor_table_comm"] |
255.566 μs (5%) | 241.42 KiB (1%) | 4593 | |
["ecc", "evaluate_decoder", "shor_table_naivesyn"] |
863.218 μs (5%) | 334.20 KiB (1%) | 4790 | |
["ecc", "evaluate_decoder", "shor_table_shorsyn"] |
1.266 ms (5%) | 427.73 KiB (1%) | 5876 | |
["ecc", "evaluate_decoder", "toric8_bp_comm"] |
693.079 ms (5%) | 13.872 ms | 176.03 MiB (1%) | 1055020 |
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] |
1.409 s (5%) | 30.180 ms | 348.89 MiB (1%) | 2113706 |
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] |
1.365 s (5%) | 30.497 ms | 349.76 MiB (1%) | 2136439 |
["ecc", "evaluate_decoder", "toric8_pybp_comm"] |
469.783 ms (5%) | 8.11 MiB (1%) | 163696 | |
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] |
958.821 ms (5%) | 18.04 MiB (1%) | 360388 | |
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] |
965.414 ms (5%) | 20.56 MiB (1%) | 392802 | |
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] |
4.251 ms (5%) | 1.95 MiB (1%) | 23176 | |
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] |
12.345 ms (5%) | 5.58 MiB (1%) | 79355 | |
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] |
20.777 ms (5%) | 8.09 MiB (1%) | 111769 | |
["ecc", "evaluate_decoder", "toric8_table_comm"] |
2.631 ms (5%) | 1.64 MiB (1%) | 23082 | |
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] |
10.047 ms (5%) | 4.93 MiB (1%) | 79160 | |
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] |
18.564 ms (5%) | 7.44 MiB (1%) | 111574 | |
["pauli", "mul", "100"] |
15.319 ns (5%) | |||
["pauli", "mul", "1000"] |
17.899 ns (5%) | |||
["pauli", "mul", "100000"] |
719.535 ns (5%) | |||
["pauli", "mul", "20000000"] |
193.591 μs (5%) | |||
["stabilizer", "canon", "cano500"] |
3.095 ms (5%) | |||
["stabilizer", "canon", "diag_cano500"] |
752.351 μs (5%) | |||
["stabilizer", "canon", "diag_gott500"] |
4.201 ms (5%) | 5.19 MiB (1%) | 34521 | |
["stabilizer", "canon", "diag_rref500"] |
626.838 μs (5%) | |||
["stabilizer", "canon", "gott500"] |
6.576 ms (5%) | 5.19 MiB (1%) | 34531 | |
["stabilizer", "canon", "md_cano500"] |
1.541 ms (5%) | |||
["stabilizer", "canon", "md_rref500"] |
1.577 ms (5%) | |||
["stabilizer", "canon", "rref500"] |
3.116 ms (5%) | |||
["stabilizer", "project", "destabilizer"] |
18.064 μs (5%) | 288 bytes (1%) | 5 | |
["stabilizer", "project", "stabilizer"] |
9.057 μs (5%) | 80 bytes (1%) | 2 | |
["stabilizer", "tensor", "diag_pow5_20"] |
1.292 ms (5%) | 23.97 MiB (1%) | 34 | |
["stabilizer", "tensor", "pow5_20"] |
2.754 μs (5%) | 6.44 KiB (1%) | 31 | |
["stabilizer", "trace", "destabilizer"] |
31.239 μs (5%) | 80 bytes (1%) | 2 | |
["stabilizer", "trace", "stabilizer"] |
24.205 μs (5%) | 112 bytes (1%) | 3 |
Benchmark Group List
Here's a list of all the benchmark groups executed by this job:
["circuitsim", "mctrajectories"]["circuitsim", "mctrajectories_sumtype"]["circuitsim", "mctrajectories_union"]["circuitsim", "pftrajectories"]["circuitsim", "pftrajectories_sumtype"]["circuitsim", "pftrajectories_union"]["clifford", "dense"]["clifford", "symbolic"]["ecc", "evaluate_decoder"]["pauli", "mul"]["stabilizer", "canon"]["stabilizer", "project"]["stabilizer", "tensor"]["stabilizer", "trace"]
Julia versioninfo
Julia Version 1.12.0-DEV.317
Commit 0e28cf6abff (2024-04-08 10:46 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
Ubuntu 22.04.4 LTS
uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
CPU: AMD EPYC 7763 64-Core Processor:
speed user nice sys idle irq
#1 0 MHz 2612 s 0 s 277 s 5786 s 0 s
#2 0 MHz 2383 s 0 s 320 s 5950 s 0 s
#3 0 MHz 2441 s 0 s 269 s 5967 s 0 s
#4 0 MHz 1973 s 0 s 269 s 6439 s 0 s
Memory: 15.606498718261719 GB (13386.1953125 MB free)
Uptime: 873.47 sec
Load Avg: 1.0 1.11 0.82
WORD_SIZE: 64
LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)
Baseline result
Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl
Job Properties
- Time of benchmark: 9 Apr 2024 - 4:11
- Package commit: 6c184a
- Julia commit: 0e28cf
- Julia command flags: None
- Environment variables: None
Results
Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.
| ID | time | GC time | memory | allocations |
|---|---|---|---|---|
["circuitsim", "mctrajectories", "q1001_r1"] |
15.326 ms (5%) | 500.77 KiB (1%) | 18019 | |
["circuitsim", "mctrajectories", "q101_r1"] |
173.193 μs (5%) | 50.53 KiB (1%) | 1818 | |
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] |
13.936 ms (5%) | 496 bytes (1%) | 9 | |
["circuitsim", "mctrajectories_sumtype", "q101_r1"] |
118.801 μs (5%) | 256 bytes (1%) | 8 | |
["circuitsim", "mctrajectories_union", "q1001_r1"] |
13.592 ms (5%) | 496 bytes (1%) | 9 | |
["circuitsim", "mctrajectories_union", "q101_r1"] |
117.169 μs (5%) | 256 bytes (1%) | 8 | |
["circuitsim", "pftrajectories", "q1001_r1"] |
63.929 μs (5%) | 93.80 KiB (1%) | 2001 | |
["circuitsim", "pftrajectories", "q1001_r100"] |
175.156 μs (5%) | 93.80 KiB (1%) | 2001 | |
["circuitsim", "pftrajectories", "q1001_r10000"] |
1.102 ms (5%) | 93.80 KiB (1%) | 2001 | |
["circuitsim", "pftrajectories", "q101_r1"] |
6.421 μs (5%) | 9.42 KiB (1%) | 201 | |
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] |
45.114 μs (5%) | 93.84 KiB (1%) | 2002 | |
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] |
155.459 μs (5%) | 93.84 KiB (1%) | 2002 | |
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] |
1.084 ms (5%) | 93.84 KiB (1%) | 2002 | |
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] |
5.875 ms (5%) | 93.84 KiB (1%) | 2002 | |
["circuitsim", "pftrajectories_sumtype", "q101_r1"] |
4.478 μs (5%) | 9.47 KiB (1%) | 202 | |
["circuitsim", "pftrajectories_union", "q1001_r1"] |
23.514 μs (5%) | 96 bytes (1%) | 2 | |
["circuitsim", "pftrajectories_union", "q1001_r100"] |
132.898 μs (5%) | 96 bytes (1%) | 2 | |
["circuitsim", "pftrajectories_union", "q1001_r10000"] |
1.054 ms (5%) | 96 bytes (1%) | 2 | |
["circuitsim", "pftrajectories_union", "q101_r1"] |
2.424 μs (5%) | 96 bytes (1%) | 2 | |
["clifford", "dense", "cnot250_on_dense500_destab"] |
11.601 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "cnot250_on_dense500_stab"] |
5.799 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "cnot250_on_diag500_destab"] |
1.123 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "cnot250_on_diag500_stab"] |
490.344 μs (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "cnot_on_dense500_destab"] |
51.315 μs (5%) | 368 bytes (1%) | 10 | |
["clifford", "dense", "cnot_on_dense500_stab"] |
24.435 μs (5%) | 368 bytes (1%) | 10 | |
["clifford", "dense", "cnot_on_diag500_destab"] |
26.289 μs (5%) | 368 bytes (1%) | 10 | |
["clifford", "dense", "cnot_on_diag500_stab"] |
13.715 μs (5%) | 368 bytes (1%) | 10 | |
["clifford", "dense", "dense500_on_dense500_destab"] |
11.581 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "dense500_on_dense500_stab"] |
5.786 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "dense500_on_diag500_destab"] |
980.026 μs (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "dense500_on_diag500_stab"] |
490.223 μs (5%) | 512 bytes (1%) | 8 | |
["clifford", "symbolic", "cnot250_on_dense500_destab"] |
1.435 ms (5%) | |||
["clifford", "symbolic", "cnot250_on_dense500_stab"] |
688.002 μs (5%) | |||
["clifford", "symbolic", "cnot250_on_diag500_destab"] |
1.161 ms (5%) | |||
["clifford", "symbolic", "cnot250_on_diag500_stab"] |
570.193 μs (5%) | |||
["clifford", "symbolic", "cnot_on_dense500_destab"] |
4.739 μs (5%) | |||
["clifford", "symbolic", "cnot_on_dense500_stab"] |
2.304 μs (5%) | |||
["clifford", "symbolic", "cnot_on_diag500_destab"] |
4.679 μs (5%) | |||
["clifford", "symbolic", "cnot_on_diag500_stab"] |
2.284 μs (5%) | |||
["ecc", "evaluate_decoder", "shor_bp_comm"] |
1.839 ms (5%) | 1.77 MiB (1%) | 42595 | |
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] |
4.029 ms (5%) | 3.60 MiB (1%) | 81368 | |
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] |
7.748 ms (5%) | 3.77 MiB (1%) | 84386 | |
["ecc", "evaluate_decoder", "shor_pybp_comm"] |
56.759 ms (5%) | 4.76 MiB (1%) | 144207 | |
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] |
150.946 ms (5%) | 9.57 MiB (1%) | 284018 | |
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] |
131.820 ms (5%) | 9.66 MiB (1%) | 285104 | |
["ecc", "evaluate_decoder", "shor_table_comm"] |
255.095 μs (5%) | 241.42 KiB (1%) | 4593 | |
["ecc", "evaluate_decoder", "shor_table_naivesyn"] |
869.139 μs (5%) | 334.20 KiB (1%) | 4790 | |
["ecc", "evaluate_decoder", "shor_table_shorsyn"] |
1.268 ms (5%) | 427.73 KiB (1%) | 5876 | |
["ecc", "evaluate_decoder", "toric8_bp_comm"] |
700.430 ms (5%) | 14.309 ms | 174.65 MiB (1%) | 1046935 |
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] |
1.382 s (5%) | 26.514 ms | 347.37 MiB (1%) | 2104753 |
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] |
1.438 s (5%) | 29.790 ms | 357.00 MiB (1%) | 2179020 |
["ecc", "evaluate_decoder", "toric8_pybp_comm"] |
469.326 ms (5%) | 8.11 MiB (1%) | 163696 | |
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] |
986.061 ms (5%) | 18.04 MiB (1%) | 360388 | |
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] |
987.792 ms (5%) | 20.56 MiB (1%) | 392802 | |
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] |
4.210 ms (5%) | 1.95 MiB (1%) | 23176 | |
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] |
12.502 ms (5%) | 5.58 MiB (1%) | 79355 | |
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] |
21.254 ms (5%) | 8.09 MiB (1%) | 111769 | |
["ecc", "evaluate_decoder", "toric8_table_comm"] |
2.599 ms (5%) | 1.64 MiB (1%) | 23082 | |
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] |
10.127 ms (5%) | 4.93 MiB (1%) | 79160 | |
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] |
18.735 ms (5%) | 7.44 MiB (1%) | 111574 | |
["pauli", "mul", "100"] |
14.516 ns (5%) | |||
["pauli", "mul", "1000"] |
18.832 ns (5%) | |||
["pauli", "mul", "100000"] |
732.071 ns (5%) | |||
["pauli", "mul", "20000000"] |
179.685 μs (5%) | |||
["stabilizer", "canon", "cano500"] |
3.088 ms (5%) | |||
["stabilizer", "canon", "diag_cano500"] |
752.430 μs (5%) | |||
["stabilizer", "canon", "diag_gott500"] |
4.030 ms (5%) | 5.19 MiB (1%) | 34521 | |
["stabilizer", "canon", "diag_rref500"] |
665.520 μs (5%) | |||
["stabilizer", "canon", "gott500"] |
6.379 ms (5%) | 5.19 MiB (1%) | 34531 | |
["stabilizer", "canon", "md_cano500"] |
1.611 ms (5%) | |||
["stabilizer", "canon", "md_rref500"] |
1.625 ms (5%) | |||
["stabilizer", "canon", "rref500"] |
3.138 ms (5%) | |||
["stabilizer", "project", "destabilizer"] |
18.344 μs (5%) | 288 bytes (1%) | 5 | |
["stabilizer", "project", "stabilizer"] |
9.047 μs (5%) | 80 bytes (1%) | 2 | |
["stabilizer", "tensor", "diag_pow5_20"] |
1.394 ms (5%) | 23.97 MiB (1%) | 34 | |
["stabilizer", "tensor", "pow5_20"] |
2.676 μs (5%) | 6.44 KiB (1%) | 31 | |
["stabilizer", "trace", "destabilizer"] |
34.274 μs (5%) | 80 bytes (1%) | 2 | |
["stabilizer", "trace", "stabilizer"] |
24.495 μs (5%) | 112 bytes (1%) | 3 |
Benchmark Group List
Here's a list of all the benchmark groups executed by this job:
["circuitsim", "mctrajectories"]["circuitsim", "mctrajectories_sumtype"]["circuitsim", "mctrajectories_union"]["circuitsim", "pftrajectories"]["circuitsim", "pftrajectories_sumtype"]["circuitsim", "pftrajectories_union"]["clifford", "dense"]["clifford", "symbolic"]["ecc", "evaluate_decoder"]["pauli", "mul"]["stabilizer", "canon"]["stabilizer", "project"]["stabilizer", "tensor"]["stabilizer", "trace"]
Julia versioninfo
Julia Version 1.12.0-DEV.317
Commit 0e28cf6abff (2024-04-08 10:46 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
Ubuntu 22.04.4 LTS
uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
CPU: AMD EPYC 7763 64-Core Processor:
speed user nice sys idle irq
#1 0 MHz 4014 s 0 s 363 s 8578 s 0 s
#2 0 MHz 3221 s 0 s 402 s 9304 s 0 s
#3 0 MHz 3522 s 0 s 378 s 9050 s 0 s
#4 0 MHz 3071 s 0 s 375 s 9505 s 0 s
Memory: 15.606498718261719 GB (13231.93359375 MB free)
Uptime: 1302.32 sec
Load Avg: 1.02 1.09 0.94
WORD_SIZE: 64
LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)
Runtime information
| Runtime Info | |
|---|---|
| BLAS #threads | 2 |
BLAS.vendor() |
lbt |
Sys.CPU_THREADS |
4 |
lscpu output:
Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 48 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 4 On-line CPU(s) list: 0-3 Vendor ID: AuthenticAMD Model name: AMD EPYC 7763 64-Core Processor CPU family: 25 Model: 1 Thread(s) per core: 2 Core(s) per socket: 2 Socket(s): 1 Stepping: 1 BogoMIPS: 4890.86 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm Virtualization: AMD-V Hypervisor vendor: Microsoft Virtualization type: full L1d cache: 64 KiB (2 instances) L1i cache: 64 KiB (2 instances) L2 cache: 1 MiB (2 instances) L3 cache: 32 MiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0-3 Vulnerability Gather data sampling: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode Vulnerability Spec store bypass: Vulnerable Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected
| Cpu Property | Value |
|---|---|
| Brand | AMD EPYC 7763 64-Core Processor |
| Vendor | :AMD |
| Architecture | :Unknown |
| Model | Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00 |
| Cores | 16 physical cores, 16 logical cores (on executing CPU) |
| No Hyperthreading hardware capability detected | |
| Clock Frequencies | Not supported by CPU |
| Data Cache | Level 1:3 : (32, 512, 32768) kbytes |
| 64 byte cache line size | |
| Address Size | 48 bits virtual, 48 bits physical |
| SIMD | 256 bit = 32 byte max. SIMD vector size |
| Time Stamp Counter | TSC is accessible via rdtsc |
| TSC runs at constant rate (invariant from clock frequency) | |
| Perf. Monitoring | Performance Monitoring Counters (PMC) are not supported |
| Hypervisor | Yes, Microsoft |
Benchmark Result
Judge result
Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl
Job Properties
- Time of benchmarks:
- Target: 9 Apr 2024 - 04:04
- Baseline: 9 Apr 2024 - 04:11
- Package commits:
- Target: 4774a8
- Baseline: 6c184a
- Julia commits:
- Target: 0e28cf
- Baseline: 0e28cf
- Julia command flags:
- Target: None
- Baseline: None
- Environment variables:
- Target: None
- Baseline: None
Results
A ratio greater than 1.0 denotes a possible regression (marked with :x:), while a ratio less
than 1.0 denotes a possible improvement (marked with :white_check_mark:). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).
| ID | time ratio | memory ratio |
|---|---|---|
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] |
0.94 (5%) :white_check_mark: | 1.00 (1%) |
["clifford", "dense", "cnot250_on_diag500_stab"] |
1.14 (5%) :x: | 1.00 (1%) |
["clifford", "dense", "dense500_on_diag500_stab"] |
1.14 (5%) :x: | 1.00 (1%) |
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] |
0.58 (5%) :white_check_mark: | 0.98 (1%) :white_check_mark: |
["ecc", "evaluate_decoder", "shor_pybp_comm"] |
1.36 (5%) :x: | 1.00 (1%) |
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] |
0.95 (5%) :white_check_mark: | 0.98 (1%) :white_check_mark: |
["pauli", "mul", "100"] |
1.06 (5%) :x: | 1.00 (1%) |
["pauli", "mul", "20000000"] |
1.08 (5%) :x: | 1.00 (1%) |
["stabilizer", "canon", "diag_rref500"] |
0.94 (5%) :white_check_mark: | 1.00 (1%) |
["stabilizer", "tensor", "diag_pow5_20"] |
0.93 (5%) :white_check_mark: | 1.00 (1%) |
["stabilizer", "trace", "destabilizer"] |
0.91 (5%) :white_check_mark: | 1.00 (1%) |
Benchmark Group List
Here's a list of all the benchmark groups executed by this job:
["circuitsim", "mctrajectories"]["circuitsim", "mctrajectories_sumtype"]["circuitsim", "mctrajectories_union"]["circuitsim", "pftrajectories"]["circuitsim", "pftrajectories_sumtype"]["circuitsim", "pftrajectories_union"]["clifford", "dense"]["clifford", "symbolic"]["ecc", "evaluate_decoder"]["pauli", "mul"]["stabilizer", "canon"]["stabilizer", "project"]["stabilizer", "tensor"]["stabilizer", "trace"]
Julia versioninfo
Target
Julia Version 1.12.0-DEV.317
Commit 0e28cf6abff (2024-04-08 10:46 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
Ubuntu 22.04.4 LTS
uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
CPU: AMD EPYC 7763 64-Core Processor:
speed user nice sys idle irq
#1 0 MHz 2612 s 0 s 277 s 5786 s 0 s
#2 0 MHz 2383 s 0 s 320 s 5950 s 0 s
#3 0 MHz 2441 s 0 s 269 s 5967 s 0 s
#4 0 MHz 1973 s 0 s 269 s 6439 s 0 s
Memory: 15.606498718261719 GB (13386.1953125 MB free)
Uptime: 873.47 sec
Load Avg: 1.0 1.11 0.82
WORD_SIZE: 64
LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)
Baseline
Julia Version 1.12.0-DEV.317
Commit 0e28cf6abff (2024-04-08 10:46 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
Ubuntu 22.04.4 LTS
uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
CPU: AMD EPYC 7763 64-Core Processor:
speed user nice sys idle irq
#1 0 MHz 4014 s 0 s 363 s 8578 s 0 s
#2 0 MHz 3221 s 0 s 402 s 9304 s 0 s
#3 0 MHz 3522 s 0 s 378 s 9050 s 0 s
#4 0 MHz 3071 s 0 s 375 s 9505 s 0 s
Memory: 15.606498718261719 GB (13231.93359375 MB free)
Uptime: 1302.32 sec
Load Avg: 1.02 1.09 0.94
WORD_SIZE: 64
LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)
Target result
Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl
Job Properties
- Time of benchmark: 9 Apr 2024 - 4:4
- Package commit: 4774a8
- Julia commit: 0e28cf
- Julia command flags: None
- Environment variables: None
Results
Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.
| ID | time | GC time | memory | allocations |
|---|---|---|---|---|
["circuitsim", "mctrajectories", "q1001_r1"] |
14.971 ms (5%) | 500.77 KiB (1%) | 18019 | |
["circuitsim", "mctrajectories", "q101_r1"] |
170.839 μs (5%) | 50.53 KiB (1%) | 1818 | |
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] |
13.095 ms (5%) | 496 bytes (1%) | 9 | |
["circuitsim", "mctrajectories_sumtype", "q101_r1"] |
117.589 μs (5%) | 256 bytes (1%) | 8 | |
["circuitsim", "mctrajectories_union", "q1001_r1"] |
13.001 ms (5%) | 496 bytes (1%) | 9 | |
["circuitsim", "mctrajectories_union", "q101_r1"] |
116.407 μs (5%) | 256 bytes (1%) | 8 | |
["circuitsim", "pftrajectories", "q1001_r1"] |
61.806 μs (5%) | 93.80 KiB (1%) | 2001 | |
["circuitsim", "pftrajectories", "q1001_r100"] |
174.234 μs (5%) | 93.80 KiB (1%) | 2001 | |
["circuitsim", "pftrajectories", "q1001_r10000"] |
1.098 ms (5%) | 93.80 KiB (1%) | 2001 | |
["circuitsim", "pftrajectories", "q101_r1"] |
6.201 μs (5%) | 9.42 KiB (1%) | 201 | |
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] |
44.713 μs (5%) | 93.84 KiB (1%) | 2002 | |
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] |
156.391 μs (5%) | 93.84 KiB (1%) | 2002 | |
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] |
1.085 ms (5%) | 93.84 KiB (1%) | 2002 | |
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] |
5.872 ms (5%) | 93.84 KiB (1%) | 2002 | |
["circuitsim", "pftrajectories_sumtype", "q101_r1"] |
4.488 μs (5%) | 9.47 KiB (1%) | 202 | |
["circuitsim", "pftrajectories_union", "q1001_r1"] |
22.611 μs (5%) | 96 bytes (1%) | 2 | |
["circuitsim", "pftrajectories_union", "q1001_r100"] |
134.300 μs (5%) | 96 bytes (1%) | 2 | |
["circuitsim", "pftrajectories_union", "q1001_r10000"] |
1.057 ms (5%) | 96 bytes (1%) | 2 | |
["circuitsim", "pftrajectories_union", "q101_r1"] |
2.334 μs (5%) | 96 bytes (1%) | 2 | |
["clifford", "dense", "cnot250_on_dense500_destab"] |
11.616 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "cnot250_on_dense500_stab"] |
5.801 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "cnot250_on_diag500_destab"] |
1.121 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "cnot250_on_diag500_stab"] |
560.604 μs (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "cnot_on_dense500_destab"] |
51.095 μs (5%) | 368 bytes (1%) | 10 | |
["clifford", "dense", "cnot_on_dense500_stab"] |
24.175 μs (5%) | 368 bytes (1%) | 10 | |
["clifford", "dense", "cnot_on_diag500_destab"] |
26.339 μs (5%) | 368 bytes (1%) | 10 | |
["clifford", "dense", "cnot_on_diag500_stab"] |
13.566 μs (5%) | 368 bytes (1%) | 10 | |
["clifford", "dense", "dense500_on_dense500_destab"] |
11.606 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "dense500_on_dense500_stab"] |
5.787 ms (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "dense500_on_diag500_destab"] |
980.276 μs (5%) | 512 bytes (1%) | 8 | |
["clifford", "dense", "dense500_on_diag500_stab"] |
560.614 μs (5%) | 512 bytes (1%) | 8 | |
["clifford", "symbolic", "cnot250_on_dense500_destab"] |
1.436 ms (5%) | |||
["clifford", "symbolic", "cnot250_on_dense500_stab"] |
686.589 μs (5%) | |||
["clifford", "symbolic", "cnot250_on_diag500_destab"] |
1.161 ms (5%) | |||
["clifford", "symbolic", "cnot250_on_diag500_stab"] |
569.611 μs (5%) | |||
["clifford", "symbolic", "cnot_on_dense500_destab"] |
4.749 μs (5%) | |||
["clifford", "symbolic", "cnot_on_dense500_stab"] |
2.294 μs (5%) | |||
["clifford", "symbolic", "cnot_on_diag500_destab"] |
4.648 μs (5%) | |||
["clifford", "symbolic", "cnot_on_diag500_stab"] |
2.304 μs (5%) | |||
["ecc", "evaluate_decoder", "shor_bp_comm"] |
1.801 ms (5%) | 1.77 MiB (1%) | 42581 | |
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] |
4.030 ms (5%) | 3.61 MiB (1%) | 81732 | |
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] |
4.465 ms (5%) | 3.71 MiB (1%) | 82846 | |
["ecc", "evaluate_decoder", "shor_pybp_comm"] |
76.985 ms (5%) | 4.76 MiB (1%) | 144207 | |
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] |
149.934 ms (5%) | 9.57 MiB (1%) | 284018 | |
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] |
134.003 ms (5%) | 9.66 MiB (1%) | 285104 | |
["ecc", "evaluate_decoder", "shor_table_comm"] |
255.566 μs (5%) | 241.42 KiB (1%) | 4593 | |
["ecc", "evaluate_decoder", "shor_table_naivesyn"] |
863.218 μs (5%) | 334.20 KiB (1%) | 4790 | |
["ecc", "evaluate_decoder", "shor_table_shorsyn"] |
1.266 ms (5%) | 427.73 KiB (1%) | 5876 | |
["ecc", "evaluate_decoder", "toric8_bp_comm"] |
693.079 ms (5%) | 13.872 ms | 176.03 MiB (1%) | 1055020 |
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] |
1.409 s (5%) | 30.180 ms | 348.89 MiB (1%) | 2113706 |
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] |
1.365 s (5%) | 30.497 ms | 349.76 MiB (1%) | 2136439 |
["ecc", "evaluate_decoder", "toric8_pybp_comm"] |
469.783 ms (5%) | 8.11 MiB (1%) | 163696 | |
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] |
958.821 ms (5%) | 18.04 MiB (1%) | 360388 | |
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] |
965.414 ms (5%) | 20.56 MiB (1%) | 392802 | |
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] |
4.251 ms (5%) | 1.95 MiB (1%) | 23176 | |
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] |
12.345 ms (5%) | 5.58 MiB (1%) | 79355 | |
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] |
20.777 ms (5%) | 8.09 MiB (1%) | 111769 | |
["ecc", "evaluate_decoder", "toric8_table_comm"] |
2.631 ms (5%) | 1.64 MiB (1%) | 23082 | |
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] |
10.047 ms (5%) | 4.93 MiB (1%) | 79160 | |
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] |
18.564 ms (5%) | 7.44 MiB (1%) | 111574 | |
["pauli", "mul", "100"] |
15.319 ns (5%) | |||
["pauli", "mul", "1000"] |
17.899 ns (5%) | |||
["pauli", "mul", "100000"] |
719.535 ns (5%) | |||
["pauli", "mul", "20000000"] |
193.591 μs (5%) | |||
["stabilizer", "canon", "cano500"] |
3.095 ms (5%) | |||
["stabilizer", "canon", "diag_cano500"] |
752.351 μs (5%) | |||
["stabilizer", "canon", "diag_gott500"] |
4.201 ms (5%) | 5.19 MiB (1%) | 34521 | |
["stabilizer", "canon", "diag_rref500"] |
626.838 μs (5%) | |||
["stabilizer", "canon", "gott500"] |
6.576 ms (5%) | 5.19 MiB (1%) | 34531 | |
["stabilizer", "canon", "md_cano500"] |
1.541 ms (5%) | |||
["stabilizer", "canon", "md_rref500"] |
1.577 ms (5%) | |||
["stabilizer", "canon", "rref500"] |
3.116 ms (5%) | |||
["stabilizer", "project", "destabilizer"] |
18.064 μs (5%) | 288 bytes (1%) | 5 | |
["stabilizer", "project", "stabilizer"] |
9.057 μs (5%) | 80 bytes (1%) | 2 | |
["stabilizer", "tensor", "diag_pow5_20"] |
1.292 ms (5%) | 23.97 MiB (1%) | 34 | |
["stabilizer", "tensor", "pow5_20"] |
2.754 μs (5%) | 6.44 KiB (1%) | 31 | |
["stabilizer", "trace", "destabilizer"] |
31.239 μs (5%) | 80 bytes (1%) | 2 | |
["stabilizer", "trace", "stabilizer"] |
24.205 μs (5%) | 112 bytes (1%) | 3 |
Benchmark Group List
Here's a list of all the benchmark groups executed by this job:
["circuitsim", "mctrajectories"]["circuitsim", "mctrajectories_sumtype"]["circuitsim", "mctrajectories_union"]["circuitsim", "pftrajectories"]["circuitsim", "pftrajectories_sumtype"]["circuitsim", "pftrajectories_union"]["clifford", "dense"]["clifford", "symbolic"]["ecc", "evaluate_decoder"]["pauli", "mul"]["stabilizer", "canon"]["stabilizer", "project"]["stabilizer", "tensor"]["stabilizer", "trace"]
Julia versioninfo
Julia Version 1.12.0-DEV.317
Commit 0e28cf6abff (2024-04-08 10:46 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
Ubuntu 22.04.4 LTS
uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
CPU: AMD EPYC 7763 64-Core Processor:
speed user nice sys idle irq
#1 0 MHz 2612 s 0 s 277 s 5786 s 0 s
#2 0 MHz 2383 s 0 s 320 s 5950 s 0 s
#3 0 MHz 2441 s 0 s 269 s 5967 s 0 s
#4 0 MHz 1973 s 0 s 269 s 6439 s 0 s
Memory: 15.606498718261719 GB (13386.1953125 MB free)
Uptime: 873.47 sec
Load Avg: 1.0 1.11 0.82
WORD_SIZE: 64
LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)
Baseline result
Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl
Job Properties
- Time of benchmark: 9 Apr 2024 - 4:11
- Package commit: 6c184a
- Julia commit: 0e28cf
- Julia command flags: None
- Environment variables: None
Results
Below is a table of this job's resul...[Comment body truncated]
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Dear Prof, I request your attention to review this one, so that I can fully focus on more abstract convolutional.
Also, if you don't mind, I want give these slides a mention in the docstring: they are so nice: https://web.ntpu.edu.tw/~yshan/BCH_code.pdf
I will probably try to review the previous PR first, which wont happen until maybe the weekend.
Since this PR is fully implemented in Nemo with universal code properties proven, it would be have been easy to review this PR first since convolutional is also based on Nemo.
Convolutional is extending the concepts used here, so I wanted to discuss one approach with you.
I will revisit Steane-Reed-Muller as well for verification and correctness with paper as well.
My apologies, since you are very occupied, I don't think I have the opportunity to discuss to seek some guidance the about Direct sums in convolutional case at this time. I will try to work on my end to polish convolutional PR as well!
Set proper bounds for BCH codes instead of arbitraty max 500 limit.
Some outputs n: block length t: t bit error that can be corrected, t is error correction capability
julia> generator_polynomial(BCH(15,2))
x^8 + x^7 + x^6 + x^4 + 1
julia> generator_polynomial(BCH(15,3))
x^10 + x^8 + x^5 + x^4 + x^2 + x + 1
julia> generator_polynomial(BCH(15,1))
x^4 + x + 1
julia> generator_polynomial(BCH(7,1))
x^3 + x + 1
julia> generator_polynomial(BCH(7,2))
x^6 + x^5 + x^4 + x^3 + x^2 + x + 1
julia> generator_polynomial(BCH(7,3))
x^6 + x^5 + x^4 + x^3 + x^2 + x + 1
julia> generator_polynomial(BCH(7,4))
x^7 + 1
BCH
- Lowest value of n is 7 since lowest order of finite field is r = 3. n is defined here as 2^r - 1. Also, t must not be > 2^(r - 1) as well.
The double and triple BCH codes when t =2, and t =3 respectively for n = 15 can be verified from the outputs as well. Their derivation is on Page4 and Page 5 of the slides here: https://web.ntpu.edu.tw/~yshan/BCH_code.pdf
julia> generator_polynomial(BCH(15,2)) x^8 + x^7 + x^6 + x^4 + 1
julia> generator_polynomial(BCH(15,3)) x^10 + x^8 + x^5 + x^4 + x^2 + x + 1
BTW, Stefan, Happy World Quantum Day!
something seems to be missing: how is one supposed to actually get a parity_check matrix here?
Reed Solomon Case
We can check the parity check polynomial. I have created two functions, namelyfunction _codeword_polynomial_rs(message, ck)and function _parity_check_polynomial_rs(n, k, x, message, gx) that takes the generator polynomial and then using the equations given in the Giesel paper, find out the corresponding parity check polynomial.
BCH case This generates the generator polynomial, and then is method to get to parity check matrix. Kindly please see the https://web.ntpu.edu.tw/~yshan/BCH_code.pdf (Pages 14 to 16). Please let me know your insights.
Actually, I ask the same question myself when writing the codes and asked the Nemo developers this question as well. From their response, it seemed there is no clear cut way to go from rings to matrices as per their description. I will forward their exact response.
BCH case This generates the generator polynomial, and then is method to get to parity check matrix. Kindly please see the https://web.ntpu.edu.tw/~yshan/BCH_code.pdf (Pages 14 to 16). Please let me know your insights.
I do not really have much insight here, I have not worked with these codes before. That resource seems to list explicitly how to construct the parity_check matrix, so that is good. This library just needs the parity check matrix -- for now it is not trying to provide much more.
Page 14 talks about some alphas that define the code. Do you have them?
Page 17 talks about taking powers of alpha and then expressing them as binary column vectors. Can you do that?
It seems as long as these terms are defined, the operation is pretty easy, but I do not know what these terms are.
I will mark this as a draft for now, until the parity check matrices are figured out.
When I was working on these codes, I asked the Nemo Developers this question:
It's nice that we can generate generator polynomials via these methods in nemo toolkit when working with FiniteFieldsIt would be great to have some guidance about how to utilize these g(x) in the matrix form.
Is there any way to represent these generator polynomials in the matrix form. If I have them (gx) in matrix for, I can convert them to parity check matrices
In summary, I am not sure whether there is a method in Nemo or AA to actually write generator polynomial as full matrix form with coefficients being of the type FiniteField and Polynomials rings.
To get around this, we can do operations on the g(x), and then get to parity check polynomial that that is still in the datatypes of Fields and Rings, not a binary parity check matrix.
This workaround is used in the paper: https://ntrs.nasa.gov/api/citations/19900019023/downloads/19900019023.pdf section 3.3.2 . Here, the author construct the message polynomial, and then the parity check polynomial for Reed Solomon Codes. For this reason, the two helper functions are described.
So, in the Reed-Solomon case, we can define the parity check polynomial. But I am not sure how to convert the polynomial representation into the corresponding matrix representation since in the doc of Nemo, there is no method for this.
Your Question
Page 14 talks about some alphas that define the code. Do you have them?
The alphas are the roots of the generator polynomial. In the Reed Solomon case, the alphas are represented by a
For example:
g(x) = x^4 + a^6*x^3 + a^6*x^2 + a^3 + a`
g(x) = x^4 + (a^2 + 1)*x^3 + (a^2 + 1)*x^2 + (a + 1) + a `
Your Question
Page 17 talks about taking powers of alpha and then expressing them as binary column vectors. Can you do that?
My guess is that we can perform operations on g(x) of BCH to get the roots/zeros of the generator polynomial and multiply it with the message to get the parity check polynomial. But since the datatypes of alpha will be finite field and rings, of that form, my guess is that we can take the powers of the zeros of the polynomial.
I will investigate it further.
Page 17 talks about taking powers of alpha and then expressing them as binary column vectors. Can you do that?
My guess is that we can perform operations on g(x) of BCH to get the roots/zeros of the generator polynomial and multiply it with the message to get the parity check polynomial. But since the datatypes of alpha will be finite field and rings, of that form, my guess is that we can take the powers of the zeros of the polynomial.
I will investigate it further.
I have the impression it is much simpler than this. If you already have alpha and you can already compute powers of alpha, it seems the only thing left is to figure out what the following means:
If you already have alpha and you can already compute powers of alpha, it seems the only thing left is to figure out what the following means:
Sounds interesting, but doesn't this assumes that we have constructed H as a matrix not as a polynomial, right? And that H matrix has zeros of the generator polynomial g(x) as elements which are of the types FiniteField Element/Polynomial Ring Element.
So, here we see this H matrix, but the elements of the matrix are of the type FiniteField Element/Polynomial Ring Element since we are working with polynomial rings/finite fields.
In Nemo/AA, since we can't represent the generator polynomial in a matrix form, so my question is that the above suggested method assumes we have the H in this matrix form after which we can perform the suggested operation, right?
In the Reed-Solomon case, we have alphas/zeros but Giesel handled the g(x) which is generator polynomial in polynomial notation, so after doing some operations such as taking mod of the polynomial which we can do in Nemo, we arrived at the parity check matrix.
Here, we are trying to represent/get the H in a matrix representation, but I believe, with Nemo, we can play with generator polynomials in polynomial representation.
Kindly please see Giesel https://ntrs.nasa.gov/api/citations/19900019023/downloads/19900019023.pdf section 3.3.2 and please let me know your thoughts. Thank you.
I will ponder about this as well. Thanks for your comments!
Sounds interesting, but doesn't this assumes that we have constructed H as a matrix not as a polynomial, right?
Not really. You can put whatever you wish in a two-axis array an call it a matrix. Sure, it will not have some fancy features that come from Nemo, but we do not want fancy features that come from Nemo, we just want a two-axis array of booleans.
Again, the one important thing to figure out is what does it mean to represent these alphas as binary vectors.
At the end, a parity check matrix for a classical code is a matrix of zeros and ones, not of fancier objects (although those fancier objects might be necessary to derive the matrix).
I skimmed through the Giesel reference, but I do not really know the topics they are discussing.
I agree. We have to leave the fancy stuff behind and come back to 1s and 0s!
Again, the one important thing to figure out is what does it mean to represent these alphas as binary vectors.
I figured it out! Actually, It's pretty simple. I am not sure that method is actually present in Nemo/AA to convert polynomial notation into tuple notation, so I will ask demo devs and then we can easily do the following
Page 14 talks about some alphas that define the code. Do you have them?
Yes, it's done now.
> parity_checks(BCH(15,2))
FqFieldElem[1 a a^2 a^3 a + 1 a^2 + a a^3 + a^2 a^3 + a + 1 a^2 + 1 a^3 + a a^2 + a + 1 a^3 + a^2 + a a^3 + a^2 + a + 1 a^3 + a^2 + 1 a^3 + 1;
1 a a^2 a^3 a + 1 a^2 + a a^3 + a^2 a^3 + a + 1 a^2 + 1 a^3 + a a^2 + a + 1 a^3 + a^2 + a a^3 + a^2 + a + 1 a^3 + a^2 + 1 a^3 + 1]
Page 17 talks about taking powers of alpha and then expressing them as binary column vectors. Can you do that?
Yes, for the corresponding Field Element, we can extract their coefficient vector as Bool as at the end of the day, we want a Boolean Matrix.
What is shown below are m-tuples in column form corresponding to each Field Element from the fancy H matrix!
Bool[1 0 0 0]
Bool[0 1 0 0]
Bool[0 0 1 0]
Bool[0 0 0 1]
Bool[1 1 0 0]
Bool[0 1 1 0]
Bool[0 0 1 1]
Bool[1 1 0 1]
Bool[1 0 1 0]
Bool[0 1 0 1]
Bool[1 1 1 0]
Bool[0 1 1 1]
Bool[1 1 1 1]
Bool[1 0 1 1]
Bool[1 0 0 1]
Bool[1 0 0 0]
Bool[0 1 0 0]
Bool[0 0 1 0]
Bool[0 0 0 1]
Bool[1 1 0 0]
Bool[0 1 1 0]
Bool[0 0 1 1]
Bool[1 1 0 1]
Bool[1 0 1 0]
Bool[0 1 0 1]
Bool[1 1 1 0]
Bool[0 1 1 1]
Bool[1 1 1 1]
Bool[1 0 1 1]
Bool[1 0 0 1]