Jeehyeok Kim

I'm not sure this guide also applies to YoloV5, but I am able to run bytetrack/yolox by following this patch https://github.com/mrtj/yolox-panorama-tutorial/blob/main/yolox-torchscript.ipynb

Here is my approach for neo compile and tested through simulator 1. Use compatible version of torch and vision pip install torch==1.8.1 torchvision==0.9.1 2. Use yolov5 latest release to export...

For me, I changed run_command or subprocess.run to include shell=True or change command gcloud to gcloud.cmd can find the path correctly.

I think this is really important feature for vllm production. Currently when client sends requests, vllm queues everything. There should have way to limit of it like rate limiting, queue...