obsidian-local-llm

obsidian-local-llm copied to clipboard

Nice work!

1. Automate starting llama server
2. Incorporate the Obsidian vault text? The llama-hub has a loader that could be useful? https://github.com/emptycrown/llama-hub/blob/main/loader_hub/obsidian/base.py

@jakepog perhaps you've been using chatbots too much and forgot that you're communicating with a human - if it were my repo I would also probably ignore you 🤣

That said, we do share a use case of interest, feeding the vault to a local offline LLM with a prompt interface. I'm a developer too and this is a problem I am interested in. This solution looks good because we can swap out models with the many available at huggingface.

@jakepog I'm guessing you have some python experience, insights on this problem?

@zatevakhin Thanks for creating this plugin, I'm sure interest will rise for sure, I was just wondering if you could share any quick insights into what's involved with feeding vault data into Llama?

Seems like LangChain might be the way? https://docs.langchain.com/docs/

I'm new to this domain but thinking through briefly, exposing the vault content for consumption by a model seems like quite a large problem with considerable performance implications. Exposing contents presumably via obsidian APIs, mapping/annotating contents+entities, caching(?), preparing for consumption, implementing langchain, configuration, tests, benchmarking... Probably out of scope for what this repo is, but an important problem perhaps other projects are a bit further along, I will do some research

EDIT: Did some research, Khoj has an implementation that covers this use-case, and it is indeed quite complex

@jakepog, @replete

Thanks!

1. I've considered maybe using a JS wrapper for LLAMA to integrate it with Obsidian. But thinking about creating a separate Python service with API for this case, and not sure that it will be a good idea to run it automatically.

2. That was the first thing I've tried to do with local LLM :) But there still are a few problems with context length, which is insufficient to properly post-process data. I used LangChain for this around one month ago. Didn't have some success in this area.

Maybe in the near time will have some time to experiment with this.

There can be an even better way to implement this using LangChain JS, but not sure, still didn't have time to investigate this.

This might be interesting to you @jakepog: https://shishirpatil.github.io/gorilla/