Enable repo map with gpt 3.5 turbo-16k?

[IkuStudies]

From what I've read, repo map is disabled outside gpt-4 because 3.5 cant handle the token load. But with 3.5-16k released has anyone tried with 16k?

I'm going to dive into aider codebase and try to enable for 3.5 16k but thought I would ask for insight first

[paul-gauthier]

Thanks for trying aider and for your interest in experimenting with the repo map.

Here is the code that determines whether to use a repo map:

https://github.com/paul-gauthier/aider/blob/166ecdcbd0b86847088dc195ede37f7b77227859/aider/coders/base_coder.py#L167

So to enable it for gpt-3.5-turbo-16k, you could add a self.use_repo_map=True here, so that it is enabled on the model:

https://github.com/paul-gauthier/aider/blob/166ecdcbd0b86847088dc195ede37f7b77227859/aider/models.py#L58

You also need update the prompts used by the wholefile coder. To use a repo-map, it will need a repo_content_prefix:

https://github.com/paul-gauthier/aider/blob/166ecdcbd0b86847088dc195ede37f7b77227859/aider/coders/wholefile_prompts.py#L42

You can consult the editblock coder for an example of the repo_content_prefix. But you probably also need to introduce the concepts of "read-only" and "read-write" files in the other prompts, similar to the editblock coder.

https://github.com/paul-gauthier/aider/blob/166ecdcbd0b86847088dc195ede37f7b77227859/aider/coders/editblock_prompts.py#L53

I worry that GPT-3.5 won't be able to understand that the repo map is read-only content. It will probably try and "edit" the repo-map files.

I plan to eventually explore this, but I am not optimistic based on my prior experience with prompting 3.5 for code editing.

[ramAdam]

@IkuStudies have you tried to make any changes, please share the results if you did ? thanks

[IkuStudies]

ok excellent, thank you for the insight @paul-gauthier

I'll have some time this afternoon to experiment, and @ramAdam will report the results here in this thread.

I'll be working with a python codebase for a game with about 20 class files, I'm aiming to refactor for more modular maintainable architecture, GPT4 would be ideal but maybe I can coax 3.5-16k to perform. I have universal ctags rigged up, and done some basic experiments with aider, so with pauls advice I should be able to at least perform some experiments with my codebase. will post any findings tonight

[Yami-Bitshark]

@IkuStudies Any update from you ?

[Yami-Bitshark]

I made some changes to get gpt35turbo to work with repo map. it's actually working quite impressively, but still gpt3 reasoning, but much much much better experience (for a difficult logic, it won't be any different since you will be using gpt as a doc resource instead of your main code builder) .

The only problem is editing file using "diff" instead of "whole", sometimes it works, sometimes it doesnt, sometimes it loops forever (since i tried to quickly force the "gpt3 turbo" condition always true and set the use repo map to true.) Still can't make it stable. Can you Paul give me any clue? (I will find a solution today, just help me sleep earlier haha).

[paul-gauthier]

Unfortunately 3.5 is not able to work well with the diff edit format. I did a fairly extensive benchmark:

https://aider.chat/docs/benchmarks.html

[Yami-Bitshark]

fair enough, i give up.

[paul-gauthier]

Ya, these LLMs can be frustrating. That's why I decided to build a benchmark, to have a fairly solid objective way to measure the impact of prompt and algorithm changes.

[paul-gauthier]

I've been doing some more experiments and they confirm my belief that 3.5 is really not capable of handling a repository map. Information about the rest of the repository seems to confuse it, and it's unable to work on the code for the files that it does have access to.

I'm gonna close this issue for now, but feel free to reopen it or file another issue if you find evidence that 3.5 could use a repository map.