drbh

closing as https://github.com/huggingface/text-generation-inference/pull/3046 was merge. Thank you @aW3st for the changes! 🙏 *note: please see https://github.com/huggingface/text-generation-inference/pull/3046#issuecomment-2729926714 for example usage

Hi @ashwincv0112 thank you for opening this issue, it appears that the starcoder2 modeling code has not been updated to handle multi lora correctly. I've started a PR here with...

Hi @ashwincv0112, the PR should be merged soon - just working on adding some tests today. Regarding the versioning once the changes are merged - the changes will not be...

Hi @BugsBuggy thanks for opening this issue, I believe the issue is related to attempting to load a model that is saved in `bnb` format. Regarding `bnb`, TGI supports loading...

This PR adds support for phi 3.5 moe, and improves the chat endpoint to assume greedy generation unless the temp is explicitly set by the user in the request (this...

Hey @aymeric-roucher thanks for pointing this out, I believe there was a couple issues with the regex expression I originally added to the docs. I think the `\\d?` notion may...

closing as https://github.com/huggingface/text-generation-inference/pull/2468 was merged and is available here https://huggingface.co/docs/text-generation-inference/en/basic_tutorials/using_guidance#constrain-with-pydantic

Hi @varad0309 thanks for opening this issue, `v2.2.0` was released ~3 weeks ago and TGI since has had some bug fixes and improvements that are available on latest. Specifically, a...

oh apologies I must have misunderstood the issue, it sounds that tools responses have regressed starting at version 2.2.0 and onwards? Would you be able to share an example of...

Hi @varad0309 I believe these issues should be resolved by the recent improvements/bug fixes to grammars and tool calling (https://github.com/huggingface/text-generation-inference/pull/2463, https://github.com/huggingface/text-generation-inference/pull/2454, https://github.com/huggingface/text-generation-inference/pull/2391, etc...) Would you kindly try the most recent...