Nicolas Patry
Nicolas Patry
@sywangyi is this expected ?
Closing this for now. Feel free to reopen.
We cannot really accept this. This is a bug in Baichuan weights, not in our code. The issue with your proposed fix is that we support tensor parallelism (TP), which...
My reviews were not sent for 1 month at least, nice :(
Closing as stale (more coming with new benchmarking tools !)
Merged in https://github.com/huggingface/text-generation-inference/pull/2665
It seems to break something else without this specific configuration. https://github.com/huggingface/text-generation-inference/actions/runs/9856277290/job/27212925674#step:13:453 `no process-level CryptoProvider available -- call CryptoProvider::install_default() before this point` I haven't understood yet what exactly is causing the...
@jerrinot Thanks for the link I saw that issue, but it seems slightly odd that every binary now has to change the code to choose a rustls implementation for the...
I'm sorry but I do not understand, how does it downloads 2 models ? I cannot reproduce on my side. Can you provide steps to reproduce, it seems the candle...
I can now reproduce. The trust-remote-code behavior of this model is suprising to me, looking into what's going on.