Jin Shang comments

Results 32 comments of


                                            Jin Shang

clangd does not include C++ system headers in the search path for files with a ".h" extension

> Or to try to track things down from the other end: Does the following, super minimal example, modified from @NomiChirps', reproduce the issue for you, @js8544, or you, @NomiChirps,...

clangd does not include C++ system headers in the search path for files with a ".h" extension

Oh I see. My system has clang 13.0.1, which doesn't include the fix @cpsauer mentioned above. So recognizing headers as c files is expected.

clangd does not include C++ system headers in the search path for files with a ".h" extension

> Just to be clear, clang_d_ 13, right? If so, good news, since clangd 14 should resolve that issue. (Seems likely that we're dealing with two different root causes here.)...

Add Encoder-decoder model support and T5 Model support

I don't mind having two separate block tables. I chose to have them in one because it would make minimal change to existing components. In fact it only adds a...

Add Encoder-decoder model support and T5 Model support

I agree. The current indexing scheme (by computing paddings each time) is painful and hard to read.

Add Encoder-decoder model support and T5 Model support

> The issue of NaN may arise due to numerical overflow when using FP16 for computation in these models(T5-Large). Yeah I noticed that transformers's original implementation of T5 also suffers...

Add Encoder-decoder model support and T5 Model support

@zhuohan123 @WoosukKwon Would you guys mind taking a look at this PR? T5 seems to be working now.

Adding support for encoder-decoder models, like T5 or BART

@zhuohan123 Hi, I'm interested in implementing support for encode-decoder models. Does it require any changes other than what's listed in https://vllm.readthedocs.io/en/latest/models/adding_model.html?

Adding support for encoder-decoder models, like T5 or BART

@WoosukKwon @zhuohan123 Hi, my team plans to work on T5 support. We would like to ask a few questions before we start. 1. Is the vLLM team currently working or...

Adding support for encoder-decoder models, like T5 or BART

Update: I'm very close to finishing this. I've run T5 with vllm successfully on my local machine. I think I will be able to submit a PR in the coming...