Benchmark: add H100 suite

Open simon-mo opened this issue 1 year ago • 2 comments

I have recently added an H100 agent which will be online for 12 hours per day. Let's test it out.

Successful build: https://buildkite.com/vllm/performance-benchmark/builds/4258

Jul 01 '24 20:07 simon-mo

Can I use this for some fp8 test - especially Mixtral

Jul 02 '24 02:07 robertgshaw2-redhat

@KuntaiDu can you please review this? I think I got it working (see link in the description) by adding bunch of clean up in the shell script

Jul 09 '24 16:07 simon-mo

this is awesome, thanks for adding

Jul 11 '24 22:07 cadedaniel