aikitoria
aikitoria
Btw, would it be possible to open source the kernels for FMHA, XQA, and "trtllmGen" ? Since MMHA is already open source, that would expose the whole library for people...
@b8zhong I don't believe any special code is required? As the model architecture is supported by vLLM, you can simply launch the model with vLLM and it should work. Though...
This also occurs with trtllm-serve, not just run.py, which is much worse!
Interesting, that's very useful! Still think this would be a nice addition though, where you could swipe to switch out images on individual messages that generated some
> Regarding CogVideoX1.5, it supports 768-1360 (long edge) and 768 short edge Is vertical video (i.e. 768x1360) meant to be supported? It always becomes blurry when I try.
Hi guys, has there been any progress towards multi-NUMA tensor parallel? I see sglang recently implemented this feature ( https://lmsys.org/blog/2025-07-14-intel-xeon-optimization/ ) but they do not have the partial offload solution...
Awesome, looking forward to it being released!
When you say in future, do you already have an estimate when this update will come?
This would be awesome indeed! It seems the architecture is quite similar to flux aswell?
Since HunyuanVideo I2V seemingly didn't turn out as good as expected, how about focusing the video support on Wan 2.1? 👀