teis-e comments

Repositories
Issues
Comments

Results 24 comments of


                                            teis-e

[Feature Request] llama v3 support

> @teis-e I was able to get Llama 3 70B-Instruct with TensorRT-LLM v0.9.0 working with: > > 1. In `tokenizer_config.json`, change line 2055 to `"eos_token": "",` > > ``` >...

[Bug]: OutputParserException: Failed to parse pydantic object from guidance program

Did you mention to fix this? I got this with both stabilityai/stablelm-zephyr-3b and zephyr 7b

Confusion about Deepspeed Inference

Hello, did you find an answer?

(request): Support for streaming without mpv

Hello, great work on the code. Can you show me an example of how you would use it? I'm having issues on getting the stream from elevenlabs: ``` response =...