teis-e
teis-e
> @teis-e I was able to get Llama 3 70B-Instruct with TensorRT-LLM v0.9.0 working with: > > 1. In `tokenizer_config.json`, change line 2055 to `"eos_token": "",` > > ``` >...
Did you mention to fix this? I got this with both stabilityai/stablelm-zephyr-3b and zephyr 7b
Hello, did you find an answer?
Hello, great work on the code. Can you show me an example of how you would use it? I'm having issues on getting the stream from elevenlabs: ``` response =...
I get this error using `from deepspeed.compression.inference.quantization import _init_group_wise_weight_quantization `: `ModuleNotFoundError: No module named 'deepspeed.compression.inference'` And this with 'from deepspeed.inference.quantization import _init_group_wise_weight_quantization': ` File "/home/sw/bulldozer/code/deepspeed/DeepSpeedExamples/inference/huggingface/zero_inference/deep.py", line 1, in from deepspeed.inference.quantization...
Yes waiting for this as welll
I did exactly as above described however it is not working in RDR2 for me .I've followed all the steps and tried a lot of different solutions. For assasin screed...