Maziyar Panahi
Maziyar Panahi
> @maziyarpanahi Please test the following 2 (marlin+non-marlin) quants instead. The previous quants had calibration issues. > > 1. https://huggingface.co/LnL-AI/dbrx-base-converted-v2-4bit-gptq-marlin-v2 > 2. https://huggingface.co/LnL-AI/dbrx-base-converted-v2-4bit-gptq-gptq-v2 My review of `LnL-AI/dbrx-base-converted-v2-4bit-gptq-gptq-v2` model: - speed:...
> @maziyarpanahi You can try [turboderp/exllamav2#388 (comment)](https://github.com/turboderp/exllamav2/issues/388#issuecomment-2028517860) I'll try to add those to the tokenizer config, but apart from the stop, the quality of the response is solid
This would be great! I would say the `TextStreamer` is just a stdout, so probably`TextIteratorStreamer` is a better choice: https://huggingface.co/docs/transformers/main/en/internal/generation_utils#transformers.TextIteratorStreamer
You guys really need to start reading either Twitter announcements or the other issues before posting new ones! There is no User stream anymore! Twitter has removed it entirely. We...
Me too, I just cannot seem to convert and quantized https://huggingface.co/deepseek-ai/deepseek-math-7b-instruct Tried on a latest `main` branch and still fails with: ``` terminate called after throwing an instance of 'std::out_of_range'...
You need to use the v2 APIs: https://twittercommunity.com/t/deprecation-announcement-removing-compliance-messages-from-statuses-filter-and-retiring-statuses-sample-from-the-twitter-api-v1-1/170500 The 1.1 still works (I have many of them) and the warning is just a warning to let you know. I highly...
Hi @LaaZa I have started the quantization based on this PR. It's at 27/40 now. Shall I also test https://github.com/AutoGPTQ/AutoGPTQ/pull/625 or this PR will be have the lats changes?
> > Hi @LaaZa > > I have started the quantization based on this PR. It's at 27/40 now. Shall I also test #625 or this PR will be have...
Hi @Qubitium I like both models, unfortunately, I cannot say which one was better. Thank you for your hard work, and please let me know if I can test any...
Twitter has changed something in its DM. You should follow twitter api and twitter developer accounts on Twitter for upcoming changes. This library is not maintained at the moment so...