DefTruth comments

Results 256 comments of


                                            DefTruth

Outlines doesn't install on Collab because Rust isn't available to compile outlines core

downgrade to outlines==0.0.46 is work for me.

[Bug]: qwen2-vl 7b, on vllm 0.8.1 & 0.8.2, sometimes (not deterministically but depends on data) I got: ValueError: Attempted to assign 702 = 702 multimodal tokens to 703 placeholders

i got the same error

[Bug]: qwen2-vl 7b, on vllm 0.8.1 & 0.8.2, sometimes (not deterministically but depends on data) I got: ValueError: Attempted to assign 702 = 702 multimodal tokens to 703 placeholders

seems we have to disable chunked-prefill on V0 mode, V1 work fine with chunked-prefill, but V0 failed. ```bash ERROR 03-31 16:21:41 [engine.py:160] ValueError('Attempted to assign 5460 = 5460 multimodal tokens...

torchao release compatibility table

still have this warning message for torch 2.9.0: ```bash Skipping import of cpp extensions due to incompatible torch version 2.9.0+cu128 for torchao version 0.14.0 Please see GitHub issue #2919 for...

[Feature] Ulysses Attention for any sequence length w/o padding

> How does this fare to unified attention? I believe UAA would be a better implementation of Ulysses attention for any sequence length. It offers zero padding, near-zero communication overhead,...

[Feature] Ulysses Attention for any sequence length w/o padding

> Sounds like a good option to me. [@DefTruth](https://github.com/DefTruth) would you like to work on adding it? My pleasure. However, I've been quite busy lately—I'll submit the implementation of UAA...