foundation-model-benchmarking-tool
foundation-model-benchmarking-tool copied to clipboard
Reports only show information per instance instead of per endpoint
In this configuration, I wanted to compare the performance of two different hosting images (LMI and TGI) on the same endpoint. (I added the g5 just to get past the statistics issue). So the config includes 2 inf2 endpoints and on g5 endpoint.
However, the graph only shows the metrics for the instance (presumably the best run) instead of a different circle for each endpoint.
We might also do the same thing if we were testing out different configurations, like batch sizes on inf2.