microsoft
Claude models (4.5 series) never use codebase search but only fileSearch or textSearch
- Copilot Chat Extension Version: 0.33.2
- VS Code Version: 1.106.2
- OS Version: Windows 11/WSL2
- Feature (e.g. agent/edit/ask mode): Agent mode
- Selected model (e.g. GPT 4.1, Claude 3.7 Sonnet): Claude Haiku 4.5, Sonnet 4.5, Opus 4.5
I have tested these models since their releases in GitHub Copilot and noticed this behavior. The semantic search tool (codebase) never get invoked even after a 2 hours session of code migration task.
I haven't had enough time with GPT 5/5.1 family but I could tell the new Gemini 3 Pro has the same behavior.
They especially use so much regex search. This means massive token waste when big model like Opus run for 5 mins just to collect the context.
This may come from the tool description that is too general and the models are trained to use the grep-like tools instead. I think we should give more priority to the semantic search tool.
The only fix I could think of right now is specifically mentioning the codebase tool which is not ideal.
I can also see the web search tool being overlooked. The agent hardly triggered it unless explicitly mentioned. On the other hand, context7 appears to be favored a lot.
I’ve noticed a fascinating pattern: the longer the tool description, the more frequently it’s utilized by the model.
I agree, given the improvement to the embedding model here (https://github.blog/news-insights/product-news/copilot-new-embedding-model-vs-code/) not using the codebase search is a wasted potential. The tool description or the system prompt should be rewritten to encourage the model to use it more.
And this is not just Claude models, I rarely see any model use the codebase search tool on its own unless I tell it to.
If you prefer that tool, you can instruct the agent to use it via custom instructions or prompt. They will use it but they do prefer text searching tools since they are RL'd on these tools. I don't want to push it a particular direction, it's hard to know what the impact will be
