Local-GPU LLM model (e.g. from huggingface) support

[michaelthwan]

We support some API call (OpenAI / GooseAI) but it would be nice if we can select a model small but good enough to do the summarization work in local GPU.

• [ ] Study models
• [ ] Code the local GPU call LLM module

[michaelthwan]

Examples

• https://huggingface.co/docs/transformers/model_doc/gpt_neox

[eren23]

@karabC if you are going to make like a document or something of your model study may I kindly take a look at it, there are many models and it's super easy to get lost so I just want it for learning stuff, if I missed anything when checking those models before etc.

[michaelthwan]

@eren23 Great idea !

[spergware]

+1