Alternative offline LLMs

[adnanPBI]

Hi, Your code used llma2 chat offline LLM. But, I wanted to use alternative offline LLMs such as huggingface's distilbert or roberta or albert. Do you have any suggestion for those LLMs to apply on python base?

[Backendmagier]

i think you would just need to update the download_models function to download the model you want... and if it is not a Llama model then you need to change the get_tokenizer function and lot the right tokenizer

[Shahin-rmz]

get_tokenizer function and lot the right tokenizer

Hi, yes I like to use my own local LLM, but it is a bit tricky. I just need to adjust the tokenizer ?