vllm-project
[New Model]: Cogagent
The model to consider.
https://huggingface.co/THUDM/CogAgent
The closest model vllm already supports.
No response
What's your difficulty of supporting the model you want?
Vision models
Need to Wait for / help with #4888 and #4942 before this can be implemented. Maybe even some more stuff.
Need to Wait for / help with #4888 and #4942 before this can be implemented. Maybe even some more stuff.
Quick update:
#4888 is landed, enabling the xFormers backend to support encoder attention, decoder self-attention, and decoder cross-attention. #4837 and #4888 (both of which have been landed) were prerequisites for #4942 . #4942 completes end-to-end support for encoder/decoder models with the xFormers backend & also introduces the BART model into vLLM. #4942 is still WIP but hoping to complete it soon.
@robertgshaw2-neuralmagic
