postgres-new icon indicating copy to clipboard operation
postgres-new copied to clipboard

feat: remove initial tool call messages

Open gregnr opened this issue 1 year ago • 1 comments
trafficstars

Background

When this project was first created as a POC, chat message history did not persist, but your DB tables did. So we added a hidden initial tool call message at the beginning of every conversation that told the LLM which tables were available. This allowed the LLM to know your schema right away without asking.

Since making the POC, we added the ability to persist chat messages which makes this feature unnecessary (DB will always have no tables at the start of the conversation).

Change

Remove these initial hidden tool call messages. They add a decent amount of tokens every request, so if they don't have a good purpose anymore lets remove them.

gregnr avatar Aug 15 '24 03:08 gregnr