Cyrus Leung
Cyrus Leung
Not supported yet. If you need this functionality now, check out #4200.
@simon-mo The `models-test` keeps getting interrupted, causing the CI to fail.
> @DarkLight1337 @ywang96 @rkooo567 Is this PR ready for merge? Yes.
This is likely the result of #4355, which made `ChatCompletionRequest.messages` more strict to avoid unrecognized attributes. Would this interface be sufficient for defining custom roles? ```py class CustomChatCompletionContentPartParam(TypedDict, total=False): __pydantic_config__...
Have you checked whether this would affect the usage of the example templates in vLLM? I don't think that the examples include BOS tokens.
I am referring to the ones that are in vLLM repo, not those on HuggingFace.
> @DarkLight1337 I got your point. I'm not familiar with all these models, but falcon-instruct doesn't use the BOS token, so this PR won't affect it. It would be great...
The CI tests fail so those have to be addressed first.
> Thanks for pointing that out. I tried to fix it but still encountered issues. I don't quite understand the error since "Any" has been imported. It passed on my...
Seems that the existing tests assume the addition of bos token. Can you update them accordingly?