nvbench
nvbench copied to clipboard
Allow partial comparison in `nvbench_compare.py`
Fixes: #295
Can compare the files in #295:
['base.json', 'base_bif.json']
# mul
## [0] NVIDIA H200
| T{ct} | OffsetT{ct} | Elements{io} | Ref Time | Ref Noise | Cmp Time | Cmp Noise | Diff | %Diff | Status |
|---------|---------------|----------------|------------|-------------|------------|-------------|----------|---------|----------|
| I16 | I32 | 2^28 | 253.092 us | 0.42% | 259.197 us | 0.35% | 6.104 us | 2.41% | SLOW |
| I16 | I64 | 2^28 | 253.069 us | 0.42% | 259.680 us | 0.37% | 6.612 us | 2.61% | SLOW |
# add
## [0] NVIDIA H200
| T{ct} | OffsetT{ct} | Elements{io} | Ref Time | Ref Noise | Cmp Time | Cmp Noise | Diff | %Diff | Status |
|---------|---------------|----------------|------------|-------------|------------|-------------|----------|---------|----------|
| I16 | I32 | 2^28 | 371.395 us | 0.22% | 374.366 us | 0.24% | 2.972 us | 0.80% | SLOW |
| I16 | I64 | 2^28 | 371.890 us | 0.21% | 374.705 us | 0.25% | 2.815 us | 0.76% | SLOW |
# triad
## [0] NVIDIA H200
| T{ct} | OffsetT{ct} | Elements{io} | Ref Time | Ref Noise | Cmp Time | Cmp Noise | Diff | %Diff | Status |
|---------|---------------|----------------|------------|-------------|------------|-------------|----------|---------|----------|
| I16 | I32 | 2^28 | 373.139 us | 0.21% | 374.479 us | 0.24% | 1.340 us | 0.36% | SLOW |
| I16 | I64 | 2^28 | 373.672 us | 0.21% | 374.310 us | 0.24% | 0.638 us | 0.17% | SAME |
# nstream
## [0] NVIDIA H200
| T{ct} | OffsetT{ct} | Elements{io} | Ref Time | Ref Noise | Cmp Time | Cmp Noise | Diff | %Diff | Status |
|---------|---------------|----------------|------------|-------------|------------|-------------|-----------|---------|----------|
| I16 | I32 | 2^28 | 492.228 us | 0.08% | 488.943 us | 0.15% | -3.285 us | -0.67% | FAST |
| I16 | I64 | 2^28 | 492.258 us | 0.13% | 489.330 us | 0.16% | -2.928 us | -0.59% | FAST |
# Summary
- Total Matches: 8
- Pass (diff <= min_noise): 1
- Unknown (infinite noise): 0
- Failure (diff > min_noise): 7