Grant Celley
Grant Celley
So this is the code I got to get it working. It is just a hack but if you want to playwith it just use this code ```python3 from transformers...
There is a few PRs over the past year. PR #565 Last updated: December 19th, 2024 The only thing I can tell is it opens up `grammar` from llama.cpp to...
I just saw that pull request #8 does that
Here is my code ```python tokenizer = AutoTokenizer.from_pretrained(MODEL) model = AutoModelForCausalLM.from_pretrained(MODEL, device_map='auto', load_in_8bit=True) llm = guidance.llms.Transformers(model, tokenizer) ``` I am going to close this