Autocomplete - Git diff is issued at each keystroke - no cache is implemented

[ferenci84]

Before submitting your bug report

• [x] I believe this is a bug. I'll try to join the Continue Discord for questions
• [x] I'm not able to find an open issue that reports the same bug
• [x] I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS: MacOS
- Continue version: 0.9.264, 0.9.261
- IDE version: 1.97.1
- config.json:

  "tabAutocompleteModel": {
    "title": "Qwen2.5-Coder 1.5B",
    "provider": "ollama",
    "model": "qwen2.5-coder:1.5b-base",
    "completionOptions": {
      "maxTokens": 100
    }
  }

Description

Git diff command is issued at each keystroke. No cache is applied to this, potentially long, operation.

My IDE and my whole system became unresponsive, basically all actions that require any file operation was delayed. Under the hood, git also used the 'rg' command. The problem disappeared if I did any of this:

• Reloaded the VS code window
• Restarted Extensions

When I disabled the Continue extension, the problem didn't came back. When I reenabled, it came back again immediately, even after a fresh system restart.

Interesting detail is that if I issue 'git diff' or 'git diff --cached', on the same repo, it responds quickly. There may be a locking mechanism or race condition that causes problem when two instances of this git command is issued at the same time.

Few screenshots:

I believe it's caused by this code:

// core/autocomplete/snippets/getAllSnippets.ts
const getDiffSnippets = async (
  ide: IDE,
): Promise<AutocompleteDiffSnippet[]> => {
  let diff: string[] = [];
  try {
    diff = await ide.getDiff(true);
  } catch (e) {
    console.error("Error getting diff for autocomplete", e);
  }

  return diff.map((item) => {
    return {
      content: item,
      type: AutocompleteSnippetType.Diff,
    };
  });
};

I checked that it's called from getAllSnippets that is called directly from provideInlineCompletionItems and no cache is preventing any of those calls to be made at each call of provideInlineCompletionItems.

To reproduce

Open git log to see that each keystroke causes git diff.

On my system possibly a large or uncommonly structured git repository caused long delays in this command, and the multiple command issued caused the file-system to be unresponsive, I'm not sure how this can be reproduced, however the core problem is apparent just by looking at the git log.

Additional note

This is an other, less problematic project where git diff do not have such a huge delay and we can see what happens. The red rectangle is where I pressed save. Before and after there are commands that are triggered when autocomplete was triggered. I think the save action, where the rectangle is, where git diff should be issued (however IMHO, the whole 'git diff' stuff could be avoided in favour of last-visited-ranges and last-edited-ranges that is already implemented for autocomplete).