IncognitoPilot
IncognitoPilot copied to clipboard
Add support for code llama
Would be great to support code llama. As of now, there is no huggingface-version, which prevents a trivial integration.
There is a huggingface-version now, but I always have issues when I run it. Very often, the model starts to just generate whitespace and doesn't stop anymore. Might be an issue with the text generation inference service, or something else, not clear yet.
Added instructions to work with code llama.
On replicate, it works, but the largest model (34B) is not available instruction-finetuned.
With TGI, I still get an error, see https://github.com/huggingface/text-generation-inference/issues/973