spliit icon indicating copy to clipboard operation
spliit copied to clipboard

categorisation via locally hosted llm service such as ollama

Open ChristopherJohnston opened this issue 8 months ago • 2 comments

Adding support to use a non-chatGPT service for category extract from title. For example, https://github.com/ollama/ollama provides an openai-like REST API for running against downloaded models such as llama3.

Modified the expense form category extraction method and added LOCAL_AI_MODEL and LOCAL_AI_URL env variables, e.g.:

NEXT_PUBLIC_ENABLE_CATEGORY_EXTRACT=true
LOCAL_AI_URL=http://127.0.0.1:11434/api/chat
LOCAL_AI_MODEL=llama3

When set, the messages are sent to this service instead of OpenAI/ChatGPT.

ChristopherJohnston avatar Jun 04 '24 13:06 ChristopherJohnston