bug: RAG Service does not properly store / retrieve embeddings from index between container instances

[retarpt]

Container does not re-use existing embeddings and wastes tokens

If I need to restart the RAG service container, or I close it, the next time I start Avante, the RAG service re-processes every file in the directory and sends calls to the /embeddings endpoint of my LLM provider. This is obviously a major problem as it wastes tokens.

I see this error when Avante starts a new container:

2025-03-18 17:41:58,217 - ERROR - Failed to load index from storage: No index in storage context, check if you specified the right persist_dir.

To reproduce

1. Start Neovim in a directory that exists in your project root defined in the RAG service config (the container starts up and files are indexed)

2. Close Neovim and docker kill the container

3. Open Neovim

4. Tail the logs of the running RAG service container and note that it is re-embedding every file in the directory (wasting tokens)

Expected behavior

I expect that the files' embeddings would be retrieved from storage between container restarts.

Installation method

lazy.nvim

Environment

NVIM v0.10.2

Repro

rag_service = {
    enabled = true,                         -- Enables the RAG service
    host_mount = "/home/fakeuser/workspace",              -- Host mount path for the rag service
    provider = "openai",                    -- The provider to use for RAG service service
    llm_model = "o3-mini-2025-01-31",       -- The LLM model to use for RAG service
    embed_model = "text-embedding-ada-002", -- The embedding model to use for RAG service
    endpoint = "https://internal-company-llm-that-emulates-openai.com",
    docker_extra_args =
    "-v /etc/ssl/certs:/etc/ssl/certs:ro -v /etc/pki/:/etc/pki:ro -e SSL_CERT_FILE=/etc/ssl/certs/ca-bundle.crt -e REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-bundle.crt -e CURL_CA_BUNDLE=/etc/ssl/certs/ca-bundle.crt -e HTTPX_CA_CERTS=/etc/ssl/certs/ca-bundle.crt",
  }

[omarcresp]

Hi, im working in some fixes for the RAG feature and my current WIP covers on that. The changes will with this PR but im full of work rn hope to resume it on weekend https://github.com/yetone/avante.nvim/pull/1565

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[jbigler]

It appears that the referenced PR was already merged, but I am still experiencing this problem. I let it run for hours to finish indexing my repo, then when I start my computer up the next day it gives the "Failed to load index from storage" error and starts reindexing the entire project. (I'm using Docker)

~/.local/share/nvim/avante/rag_service/sqlite/indexing_history.db is around 37MB ~/.local/share/nvim/avante/rag_service/chroma_db/chroma.sqlite3 is around 700MB.

Additionally, it does not have git installed in the Docker image. When I tried adding it and building the image myself, it was used by the rag service, but complained about it being an unsafe repository.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.

[NathanPB]

Did you ever figure this out? I'm stuck with the same issue.

[jbigler]

No, I've been trying some other plugins to see what options there are.