omniperf
omniperf copied to clipboard
ROCm
Metric filtering does not work with baseline compare
Describe the bug Metric filtering does not work with baseline compare
Development Environment:
- Linux Distribution: RHEL8
- Omniperf Version: Latest 2.x
- GPU: MI100
- Custer (if applicable): HPCFund
To Reproduce Steps to reproduce the behavior:
- Run omniperf analyze -p tests/workloads/vcopy/MI100/ -p tests/workloads/vcopy/MI200 --block 11
- See error numpy.core._exceptions._UFuncNoLoopError: ufunc 'add' did not contain a loop with signature matching types (dtype('float64'), dtype('<U2')) -> None
- Also returns error when passing same path
Screenshots
Additional context Add any other context about the problem here.
I couldn't reproduce the issue with the commands you attached. I suspect you may have incorrectly pulled - I'll schedule an appointment to review over the phone
I hit the same issue today on:
commit 62221383a701f13fac63bdbf54b8fc6fb87ebc5e (HEAD -> 2.x, origin/2.x)
Author: colramos-amd <[email protected]>
Date: Mon Mar 18 14:28:14 2024 -0500
Fixing visualization for Compute Pipeline SOL. Need to consider 'Pct of Peak' rather than 'Avg'
Signed-off-by: colramos-amd <[email protected]>
will put details in our chat
If I provide the same path twice, error is correctly caught, i.e.
$ omniperf analyze -p workloads/mix/MI200/ -p workloads/mix/MI200/
___ _ __
/ _ \ _ __ ___ _ __ (_)_ __ ___ _ __ / _|
| | | | '_ ` _ \| '_ \| | '_ \ / _ \ '__| |_
| |_| | | | | | | | | | | |_) | __/ | | _|
\___/|_| |_| |_|_| |_|_| .__/ \___|_| |_|
|_|
Analysis mode = cli
ERROR You cannot provide the same path twice.
If I provide two different workloads and block filter, data correctly loads, i.e.
$ omniperf analyze -p workloads/mix/MI200/ -p workloads/mix_k_filt/MI200/ -b 11
___ _ __
/ _ \ _ __ ___ _ __ (_)_ __ ___ _ __ / _|
| | | | '_ ` _ \| '_ \| | '_ \ / _ \ '__| |_
| |_| | | | | | | | | | | |_) | __/ | | _|
\___/|_| |_| |_|_| |_|_| .__/ \___|_| |_|
|_|
Analysis mode = cli
[analysis] deriving Omniperf metrics...
...
--------------------------------------------------------------------------------
11. Compute Units - Compute Pipeline
11.1 Speed-of-Light
╒═════════════╤═══════════════════╤═════════╤═══════════════════╤════════════╤════════╤═══════════╤═════════════════╤═══════════════╤════════════════╕
│ Metric_ID │ Metric │ Avg │ Avg │ Abs Diff │ Unit │ Peak │ Peak │ Pct of Peak │ Pct of Peak │
╞═════════════╪═══════════════════╪═════════╪═══════════════════╪════════════╪════════╪═══════════╪═════════════════╪═══════════════╪════════════════╡
│ 11.1.0 │ VALU FLOPs │ 7497.89 │ 10053.16 (34.08%) │ 11.29 │ Gflop │ 22630.40 │ 22630.4 (0.0%) │ 33.13 │ 44.42 (34.08%) │
├─────────────┼───────────────────┼─────────┼───────────────────┼────────────┼────────┼───────────┼─────────────────┼───────────────┼────────────────┤
│ 11.1.1 │ VALU IOPs │ 2325.84 │ 1132.67 (-51.3%) │ -5.27 │ Giop │ 22630.40 │ 22630.4 (0.0%) │ 10.28 │ 5.01 (-51.28%) │
├─────────────┼───────────────────┼─────────┼───────────────────┼────────────┼────────┼───────────┼─────────────────┼───────────────┼────────────────┤
│ 11.1.2 │ MFMA FLOPs (BF16) │ 0.00 │ 0.0 (0.0%) │ 0.00 │ Gflop │ 181043.20 │ 181043.2 (0.0%) │ 0.00 │ 0.0 (0.0%) │
├─────────────┼───────────────────┼─────────┼───────────────────┼────────────┼────────┼───────────┼─────────────────┼───────────────┼────────────────┤
│ 11.1.3 │ MFMA FLOPs (F16) │ 0.00 │ 0.0 (0.0%) │ 0.00 │ Gflop │ 181043.20 │ 181043.2 (0.0%) │ 0.00 │ 0.0 (0.0%) │
├─────────────┼───────────────────┼─────────┼───────────────────┼────────────┼────────┼───────────┼─────────────────┼───────────────┼────────────────┤
│ 11.1.4 │ MFMA FLOPs (F32) │ 0.00 │ 0.0 (0.0%) │ 0.00 │ Gflop │ 45260.80 │ 45260.8 (0.0%) │ 0.00 │ 0.0 (0.0%) │
├─────────────┼───────────────────┼─────────┼───────────────────┼────────────┼────────┼───────────┼─────────────────┼───────────────┼────────────────┤
│ 11.1.5 │ MFMA FLOPs (F64) │ 0.00 │ 0.0 (0.0%) │ 0.00 │ Gflop │ 45260.80 │ 45260.8 (0.0%) │ 0.00 │ 0.0 (0.0%) │
├─────────────┼───────────────────┼─────────┼───────────────────┼────────────┼────────┼───────────┼─────────────────┼───────────────┼────────────────┤
│ 11.1.6 │ MFMA IOPs (INT8) │ 0.00 │ 0.0 (0.0%) │ 0.00 │ Giop │ 181043.20 │ 181043.2 (0.0%) │ 0.00 │ 0.0 (0.0%) │
╘═════════════╧═══════════════════╧═════════╧═══════════════════╧════════════╧════════╧═══════════╧═════════════════╧═══════════════╧════════════════╛
11.2 Pipeline Stats
...
I may need some extra help understanding where this issue is occuring
