libra

Look for the feature +1

Is there any new progress or update for this ?

@knqyf263 what's your advice to contribute, which `dir` should we put in, since there is no bazel rules now.

+1 for this feature

I am working on this PR, and will soon submit the draft.

You're right. I've just discovered a discussion about T5 https://github.com/vllm-project/vllm/issues/187#issuecomment-1825244021 , where there are differing opinions on the encoder-decoder model. Perhaps it will improve after that PR is merged?

> Would it be possible to implement this so realtime data can be sent from any source (e.g. websockets, reading large file, etc...) glad to see to support from websockets...

I also care about this. Btw I suggest you to using the bazel 4.0+ version from my experences. it can reduce lots of problem. @dinukadev