Aidan San
Aidan San
I started doing a bit of work towards this, but the code is still very untested. https://github.com/ucbds-infra/otter-grader/commit/4b4fcceac8a9e093d738521d32c06bcd2780d8c1#diff-5d0eda1bcc776b13f0532e94251c777a561f7a81e51bec8821b27feb6ca412e2R66
Also my current hack is including it as one of the `autograder_files` and then deleting the .py file from the zip file.
I initialized a sampler and passed it into a regex generator. ```python sampler = outlines.samplers.MultinomialSampler(temperature=1.5, top_k=100, samples=5) generator = outlines.generate.regex( model, r"I really love (\S*) cake with ice cream.", sampler=sampler...
I'm not sure about `ExLlamaV2`, but maybe? This was my model initialization code `model = outlines.models.transformers("meta-llama/Meta-Llama-3-8B")`