Codebase retrieval mode doesn't properly answer using the context

[logancyang]

Before submitting your bug report

• [X] I believe this is a bug. I'll try to join the Continue Discord for questions
• [X] I'm not able to find an open issue that reports the same bug
• [X] I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS: MacOS 14.3.1
- Continue: v0.8.17
- IDE: VS Code 1.87.1

Description

First time trying continue with @codebase, it seems successful at retrieval but the answer still appear to see nothing from the retrieved files.

I've been trying different questions with @Codebase but none worked. This one above was with GPT-4 and the default transformers.js embedding.

Really wanted it to work. (I was planning to make a YT video to share it because I think this is the current best way to integrate local coding LLMs)

I also tried using openai embedding small by setting the embedding provider as below and refreshing the index, but got this error Error getting context items from codebase: TypeError: Cannot read properties of undefined (reading '0'). Could be a separate issue though.

{
  "embeddingsProvider": {
    "provider": "openai",
    "model": "text-embedding-3-small"
  }
}

To reproduce

My config.json

{
  "models": [
    {
      "title": "GPT-4 (Free Trial)",
      "provider": "free-trial",
      "model": "gpt-4"
    },
    {
      "title": "GPT-4 Vision (Free Trial)",
      "provider": "free-trial",
      "model": "gpt-4-vision-preview"
    },
    {
      "title": "Gemini Pro (Free Trial)",
      "provider": "free-trial",
      "model": "gemini-pro"
    },
    {
      "title": "Codellama 70b (Free Trial)",
      "provider": "free-trial",
      "model": "codellama-70b"
    },
    {
      "title": "Phind Codellama",
      "provider": "ollama",
      "model": "phind-codellama",
      "num_ctx": 16384
    }
  ],
  "slashCommands": [
    {
      "name": "edit",
      "description": "Edit selected code"
    },
    {
      "name": "comment",
      "description": "Write comments for the selected code"
    },
    {
      "name": "share",
      "description": "Download and share this session"
    },
    {
      "name": "cmd",
      "description": "Generate a shell command"
    }
  ],
  "customCommands": [
    {
      "name": "test",
      "prompt": "Write a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.",
      "description": "Write unit tests for highlighted code"
    }
  ],
  "contextProviders": [
    {
      "name": "diff",
      "params": {}
    },
    {
      "name": "open",
      "params": {}
    },
    {
      "name": "terminal",
      "params": {}
    },
    {
      "name": "problems",
      "params": {}
    },
    {
      "name": "codebase",
      "params": {
        "nRetrieve": 45,
        "nFinal": 15,
        "useReranking": true
      }
    }
  ],
  "embeddingsProvider": {
    "provider": "transformers.js"
  },
  "allowAnonymousTelemetry": false
}

Log output

No response

[sestinj]

@logancyang it looks like you're doing nothing wrong! Properties of undefined does seem like a separate error and I'll check this out.

But for the main error, I'm wondering if perhaps the aiState file just got pruned out of the context (this seems particularly likely since it's at the top of the list). A couple of things might help debug:

• Check the "Output" window in VS Code (next to Terminal), go to the dropdown, and select "Continue - LLM Prompt/Completion". This will show the exact prompt sent to the LLM where you'd be able to verify that anything was/wasn't included
• If it turns out that this is just getting cut out, then you can adjust the number of results returned to make sure that none of them are cut out: https://continue.dev/docs/walkthroughs/codebase-embeddings#nfinal

Assuming this is the case there are a few really obvious fixes I'll make on our end:

1. Set the default nFinal to something that would fit inside of the default GPT-4 context window
2. Fix the docs because they say default nFinal is 5. It's actually 10 right now
3. Return the results in reverse order because that top result should be the closest match, but is the first to be truncated