Nipi64310

an error will occur when the input shape dimension is greater than 2

1

Hi @nebuly-ai Thanks for sharing this! When I test @accelerate_model() in bert, the input size is (batch_size * text_length * hidden_size). there will be an error `RuntimeError: self must be...

bug recurrence: acm watcher script with code "from gevent import monkey; monkey.patch_all()" or gunicorn start service with "-k gevent" Traceback (most recent call last): File "/root/anaconda3/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker...

Can I deploy the service using Lorax without using lorax-launcher to start?

2

Can I deploy the service using Lorax without using lorax-launcher to start, and instead load the model in the code? Similar to HF and VLLM, I can use the following...