Austin
Austin
You need to extract it from a currently existing model. The models are available in the `llama.cpp/models` path following the prefix format of `ggml-vocab-`. If any of the existing vocabs...
This is interesting. The only issue I see with this is that it doesn't account for FIM (Fill-in-the-Middle). Other than that, it seems alright. Something to note is that this,...
> Or are you meaning coding related models and I dont know, if they have some fill-in-the-blank or is it fill-in-the-middle Yes, [this is what I meant](https://arxiv.org/abs/2207.14255). One of the...
I have a [pattern](https://github.com/teleprint-me/text-extraction/blob/main/text_extraction/logger.py#L13) we can use to centralize the logger instantiation with. The implementation is really simple and flexible. ```python LOGGER_FORMAT = "%(asctime)s - %(filename)s:%(lineno)d - %(levelname)s - %(message)s"...
No worries. It was just a suggestion. No need to use it if undesired. As for your questions. This is builtin. No added dependency. You're already using it. ```python import...
Maybe this one should be prioritized for now?
`` is [End of Turn](https://github.com/meta-llama/llama3/blob/main/llama/tokenizer.py#L70). Meta always includes the [templates in their source code](https://github.com/meta-llama/llama3/blob/main/llama/tokenizer.py#L202). Should always reference it as a guide. > The end of each message is marked by...
Yeah, this is why I said templates should be the responsibility of the user. It's why I always use the completions endpoint and avoid any chat template enforcement. The problem...
I think this is the middle of the road solution which is good. I just keep reiterating it because the tokens are dictated by the tokenizer and the settings used...
I'd love to have it automated, it would be great. I forgot where I stated it, but I remember reiterating that this is similar to _"Hilbert's paradox of the Grand...