worker-vllm icon indicating copy to clipboard operation
worker-vllm copied to clipboard

Feature request: environment variable for `--task`

Open nFunctor opened this issue 9 months ago • 1 comments

Hello!

We are trying to use serverless for embedders like BAAI/bge-multilingual-gemma2 that work very well on your pods. For this embedder, one has to pass explicitly the argument --task embed for it not to default to generate. I seem to be unable to do that with the runpod worker configuration.

Is it possible to make it an environment variable in a future release?

nFunctor avatar Mar 21 '25 10:03 nFunctor

Sure. We can look into this.

pandyamarut avatar Mar 26 '25 18:03 pandyamarut