oneapi-src
BenchDNN error handling issue
Error prompt information processing is too general, not specific enough in benchdnn. And no --help option supported in command mode. Just a doc in page https://github.com/oneapi-src/oneDNN/tree/master/tests/benchdnn.
e.g.
(base) huang@mlp:~/oneDNN/build$ ./tests/benchdnn/benchdnn --DRIVER --ip --cfg=f32 --dir=FWD_I
err: unknown driver
tests:0 passed:0 skipped:0 mistrusted:0 unimplemented:0 failed:0 listed:0
Based on this error tip, I'm really confused about what options to change, and what are the correct options? Some infos like this would be better.
Expected behavior
./benchdnn --help
Usage:
benchdnn --DRIVER
[--engine=ENGINE_KIND]
[--mode=MODE] [--reset]
[--max-ms-per-prb=INT] [--fix-times-per-prb=INT]
[-vINT|--verbose=INT]
[--skip-impl=SKIP_IMPL]
[--allow-unimpl=BOOL]
[--perf-template=PERF_TEMPLATE]
[DRIVER-OPTS] PROBLEM-DESCRIPTION
[--batch=FILE]
...
Hi @huangzhiyuan, Command-line help support is something that we have in our backlog but it's not in the top priority list. But I guess it will land into the product eventually. Error messages are not quite clear, can't agree more on this. We also have an item in our backlog to improve error handling/messages for benchdnn (at least for user's input) and it will improved rather sooner than later. Thanks for sharing your feedback with us.
P.S. The doc page you've sent has recently updated documentation which should bring more clarity than the previous version. Tool usage may be more transparent if using desired driver doc page directly. For Inner Product it will be this one: https://github.com/oneapi-src/oneDNN/blob/master/tests/benchdnn/doc/driver_ip.md. Hope this will slightly reduce the complexity.
"err: unknown driver" is the correct error log. https://github.com/oneapi-src/oneDNN/blob/master/tests/benchdnn/benchdnn.cpp#L142
Code is the best document if you have time to review code. You can add "-DCMAKE_BUILD_TYPE=Debug" in your source cmake build, and use gdb to trace: $ gdb --args ./benchdnn YOUR_ARGS
