Zhong Qishuai issues

Results 3 issues of


                                            Zhong Qishuai

Possibility of Passing Prompts as List[str] to AsyncEngine.generate()

Hi! Thank you for your amazing framework! I have tried serving a GPT BigCode model using vllm together with ray following the example: https://github.com/ray-project/ray/blob/3d3183d944424a960a2c6ce048abd1316c901c1e/doc/source/serve/doc_code/vllm_example.py And in my use case the...

good first issue

feature request

text_generation_launcher: Method Warmup encountered an error: `RuntimeError: CUDA error: operation not permitted when stream is capturing` with v0.8.2

### System Info OS version: Ubantu 18.04.1 GPU: Rtx 2080 Nvidia & Cuda: +-----------------------------------------------------------------------------+ | NVIDIA-SMI 460.91.03 Driver Version: 460.91.03 CUDA Version: 11.2 | |-------------------------------+----------------------+----------------------+ ### Information - [X] Docker...

[Frontend] re-enable multi-modality input in the new beam search implementation

**Changes in this PR:** This PR introduces the following changes based on the updated beam search implementation: * Re-enable multi-modality input: Support for multi-modality input has been re-enabled for beam...