Add fallback to gpt-3.5-turbo-16k

[w3bdesign]

We should fall back to gpt-3.5-turbo-16k instead of the default gpt-3.5-turbo model, as it can handle larger code outputs.

[jebarpg]

You will also need to update the gpt-engineer\scripts\rerun_edited_message_logs.py model line 16 https://github.com/AntonOsika/gpt-engineer/blob/28cb9dfeaf6e764c5795bd0e3675d2fd1eaf6243/scripts/rerun_edited_message_logs.py#L16

otherwise there will not be any files created when running main.py, only all_output.txt will have the chat text in it and you will have to manually strip out the individual files.

[w3bdesign]

You will also need to update the gpt-engineer\scripts\rerun_edited_message_logs.py model line 16

https://github.com/AntonOsika/gpt-engineer/blob/28cb9dfeaf6e764c5795bd0e3675d2fd1eaf6243/scripts/rerun_edited_message_logs.py#L16

otherwise there will not be any files created when running main.py, only all_output.txt will have the chat text in it and you will have to manually strip out the individual files.

But that string there is hardcoded, and doesn't include the new fallback system.

It should implement the new fallback system as well.

[jebarpg]

You will also need to update the gpt-engineer\scripts\rerun_edited_message_logs.py model line 16 https://github.com/AntonOsika/gpt-engineer/blob/28cb9dfeaf6e764c5795bd0e3675d2fd1eaf6243/scripts/rerun_edited_message_logs.py#L16

otherwise there will not be any files created when running main.py, only all_output.txt will have the chat text in it and you will have to manually strip out the individual files.

But that string there is hardcoded, and doesn't include the new fallback system.

It should implement the new fallback system as well.

Sorry that's exactly what I'm trying to say.

[jebarpg]

actually it would probably solve the issue generically if the user was prompted with a list of available models to choose from for fall back instead of forcing a specific fall back. But when prompted with the fallback options have the (default ) option selected as (gpt-3.5-turbo-16k) as you mentioned above. Maybe sprinkle a comment next to each model name stating (recommended) (standard) (low quality) or something to this effect for quick comparison of what each model's use cases are in comparsion to each other: gpt-3.5-turbo (standard) gpt-3.5-turbo-0301 (something) gpt-3.5-turbo-0613 (something) gpt-3.5-turbo-16k (recommended) gpt-3.5-turbo-16k-0613 (something)

[w3bdesign]

actually it would probably solve the issue generically if the user was prompted with a list of available models to choose from for fall back instead of forcing a specific fall back. But when prompted with the fallback options have the (default ) option selected as (gpt-3.5-turbo-16k) as you mentioned above. Maybe sprinkle a comment next to each model name stating (recommended) (standard) (low quality) or something to this effect for quick comparison of what each model's use cases are in comparsion to each other: gpt-3.5-turbo (standard) gpt-3.5-turbo-0301 (something) gpt-3.5-turbo-0613 (something) gpt-3.5-turbo-16k (recommended) gpt-3.5-turbo-16k-0613 (something)

But what about the planned model deprecations in the future?

[patillacode]

Hey @w3bdesign

I agree that using the 16k model as the fallback makes sense. I also think this should be taken care of in the file @jebarpg has mentioned.

It is true it is not as easy, maybe this is a good task if you want to tackle it out and adding it to your PR?

[w3bdesign]

@jebarpg @patillacode

I changed the model to gpt-3.5-turbo-16k-0613 instead and added the fallback check to the rerun_edited_message_logs.py

Can you check and see if it is better now?

[patillacode]

Thanks @w3bdesign for the effort.

Let's get a final decision in #135

Please be patient! Thanks again!

[AntonOsika]

Thanks @patillacode for following up here!

I don't see the need for 16k tokens. Has anyone had problems with this?

Also, duplicate code not needed?

[jebarpg]

Thanks @patillacode for following up here!

I don't see the need for 16k tokens. Has anyone had problems with this?

Also, duplicate code not needed?

@AntonOsika Check out my PR https://github.com/AntonOsika/gpt-engineer/pull/144 for model fallback. I centralized where the model fallback needs to be updated into one file in the main repo, and it will update both files (main.py and rerun_edited_message_logs.py) with the same fallback model. This makes it less of an issue for keeping two files updated with the same thing. Feel free to critique the mode_select.py file and add what ever default model should be fallen back to.

[patillacode]

Closing for now in favor of keeping the project easy to navigate.

Will address if discussion #135 concludes so.