nvbench icon indicating copy to clipboard operation
nvbench copied to clipboard

Allow partial comparison in `nvbench_compare.py`

Open bernhardmgruber opened this issue 2 months ago • 0 comments

Fixes: #295

Can compare the files in #295:

['base.json', 'base_bif.json']
# mul

## [0] NVIDIA H200

|  T{ct}  |  OffsetT{ct}  |  Elements{io}  |   Ref Time |   Ref Noise |   Cmp Time |   Cmp Noise |     Diff |   %Diff |  Status  |
|---------|---------------|----------------|------------|-------------|------------|-------------|----------|---------|----------|
|   I16   |      I32      |      2^28      | 253.092 us |       0.42% | 259.197 us |       0.35% | 6.104 us |   2.41% |   SLOW   |
|   I16   |      I64      |      2^28      | 253.069 us |       0.42% | 259.680 us |       0.37% | 6.612 us |   2.61% |   SLOW   |

# add

## [0] NVIDIA H200

|  T{ct}  |  OffsetT{ct}  |  Elements{io}  |   Ref Time |   Ref Noise |   Cmp Time |   Cmp Noise |     Diff |   %Diff |  Status  |
|---------|---------------|----------------|------------|-------------|------------|-------------|----------|---------|----------|
|   I16   |      I32      |      2^28      | 371.395 us |       0.22% | 374.366 us |       0.24% | 2.972 us |   0.80% |   SLOW   |
|   I16   |      I64      |      2^28      | 371.890 us |       0.21% | 374.705 us |       0.25% | 2.815 us |   0.76% |   SLOW   |

# triad

## [0] NVIDIA H200

|  T{ct}  |  OffsetT{ct}  |  Elements{io}  |   Ref Time |   Ref Noise |   Cmp Time |   Cmp Noise |     Diff |   %Diff |  Status  |
|---------|---------------|----------------|------------|-------------|------------|-------------|----------|---------|----------|
|   I16   |      I32      |      2^28      | 373.139 us |       0.21% | 374.479 us |       0.24% | 1.340 us |   0.36% |   SLOW   |
|   I16   |      I64      |      2^28      | 373.672 us |       0.21% | 374.310 us |       0.24% | 0.638 us |   0.17% |   SAME   |

# nstream

## [0] NVIDIA H200

|  T{ct}  |  OffsetT{ct}  |  Elements{io}  |   Ref Time |   Ref Noise |   Cmp Time |   Cmp Noise |      Diff |   %Diff |  Status  |
|---------|---------------|----------------|------------|-------------|------------|-------------|-----------|---------|----------|
|   I16   |      I32      |      2^28      | 492.228 us |       0.08% | 488.943 us |       0.15% | -3.285 us |  -0.67% |   FAST   |
|   I16   |      I64      |      2^28      | 492.258 us |       0.13% | 489.330 us |       0.16% | -2.928 us |  -0.59% |   FAST   |

# Summary

- Total Matches: 8
  - Pass    (diff <= min_noise): 1
  - Unknown (infinite noise):    0
  - Failure (diff > min_noise):  7

bernhardmgruber avatar Dec 10 '25 10:12 bernhardmgruber