GPT-3.5-turbo & made up actions

[jakenuts]

I know the model isn't ideal, but I'm having trouble convincing the agent to stop using 'browser.extract' despite ruling that out in my prompt and in the actions prompt as well. Any guidance on how to nail down that requirement?

My prompt:

Scan gbase for 5 new telecasters and summarize their details in a markdown file using one of the actions I will list. NEVER use the "browser.extract" action, IT DOES NOT EXIST. The search url is https://www.gbase.com/gear?Q=telecaster&page={page}

Updated prompts text

What will the next action you will take be, from the actions provided above? Using the functions above, give me a single action to take next, like: {"action":"some.action","optionName":"optionValue"}

If an action isn't explicitly listed here, it doesn't exist. For example, there is no "browser.extract" action, but there is a "browser.navigate" action. Do not return an action that doesn't exist.

[steve8708]

Ah yes 3.5 is quite bad at doing things like this. Unfortunately I have no suggestions top of mind, but appreciate you raising this

[samijaber]

@steve8708 one potential solution is to provide, similar to what @jakenuts suggested:

In the initial prompt for GPT, start by providing a TypeScript interface/type that lists all of the possible actions along with their types, and then tell GPT to restrict its answers to function calls that exist in that interface.

[steve8708]

this is what we already do, we just user a JSON interface instead of TS, but same approach

the problem is that GPT 3.5 tends to ignore instructions like this vs GPT 4 doesn't have this issue. this project isn't really compatible with 3.5 anyway given the large context windows needed for HTML (it's mentioned that GPT 4 is a requirement in the readme)