Mark Saroufim

@jamessocure would you like to make the PR to improve our batching documentation?

Could you please add some detail as to how you verified this fix worked?

Hi which configuration server are you referring to and could you elaborate on what you'd like better if we did integrate it?

Which one would be most useful to you? Realistically we can only prioritize one at first

Hi @jack-gits I think it's an interesting ask but I'm not sure how to prioritize it. If you'd like to make a PR proposal for it by all means I...

Just a quick update, the main challenge for this PR will be correctly packaging setup.py with all the correct package versions and I'm seeing all sorts of basic issues even...

And here's an example of an inference running from my logs. Repro is in `test/pytest/test_onnx.py` ``` (base) ubuntu@ip-172-31-17-70:~$ curl -X POST http://127.0.0.1:8080/predictions/onnx --data-binary '1' [ -0.28003010153770447 ](base) ubuntu@ip-172-31-17-70:~$ ```

Thank you! Just triggered CI to see if we missed anything

Thank @hgong-snap I'm trying to assess how high of a priority this ask should be. Do you mind elaborating on your integrations point. cc: @maaquib @lxning for visibility

@HamidShojanazeri are you still interested in merging this PR?