worker-vllm
worker-vllm copied to clipboard
Feature request: environment variable for `--task`
Hello!
We are trying to use serverless for embedders like BAAI/bge-multilingual-gemma2 that work very well on your pods. For this embedder, one has to pass explicitly the argument --task embed for it not to default to generate. I seem to be unable to do that with the runpod worker configuration.
Is it possible to make it an environment variable in a future release?
Sure. We can look into this.