Matthew Kotila

@rarzumanyan are you able to add testing for this feature?

@onqtam any update on this?

You can't specify both concurrency and request rate because they affect each other. Here's a simple counterexample: Imagine you specify concurrency of 2 and a request rate of 4 requests...

> @NikeNano: Does pref_analyzer throughput also include transfering data back to CPU when cuda share memory is used? I'm not sure if I understand. The calculation for throughput simply counts...

@tanmayv25 in case you have any quick ideas

@tanmayv25 can you review this PR? I'm not familiar with the `geventhttpclient` library or the proxy concept.

> @matthewkotila The change looks quite straightforward. We need to add the CI test coverage for the use case before merging this PR. You can create a ticket to extend...

I've created a ticket for us to add testing in case the PR creator does not do so themselves.

@jbkyang-nvi can you review this code? I think you were the one who mainly worked on the java client. If I'm mistaken, can you recommend someone who is familiar with...

> @kzelias: ... But I still don't understand how to get this to work on multiple files. Could you elaborate? If your model has multiple inputs that you want to...