hash icon indicating copy to clipboard operation
hash copied to clipboard

H-2950: Simplify coordinator progress report and entity submission

Open CiaranMn opened this issue 8 months ago β€’ 1 comments

🌟 What is the purpose of this PR?

This PR refactors the coordinator agent to simplify its task and reduce token usage by:

  1. Replace the current process whereby it:
  • (1) receives facts, (2) decides which facts to turn into entities with a tool call, (3) decides which entities to submit as 'for return' with a tool call, (4) decides when to declare the task complete with a tool call (at which point the previous submitted entities are returned) with a process whereby it
  • (1) receives entities which have automaticallly been created from the available facts, and (2) declares the job complete with a tool call, specifying which of the available entities to return at the same time

The intention of this is to:

  • reduce chance for error by reducing the number of decisions it has to make
  • reduce token usage by it not needing it to see the raw facts
  • reduce token usage and time by reducing the number of iterations it goes through to conclude a task
  1. Give it the history of things done and discovered in a single progress report message, instead of having those represented as a history of tool calls made and responses received – the only tool calls it now sees in the history are those which have just errored.

Drive-bys:

  1. Move any Q&A made to the user to the user's first message instead of the system prompt
  2. Add retry to calls to the internal API so that the research task doesn't error out if there's a dropped call to it
  3. Some tweaks to prompts to account for regressions in fact gathering performance when switching from Opus to Haiku
  4. Don't throw an error if the fact gathering request reaches its max retries (which kills the whole task), instead return any valid facts gathered to that point
  5. add the task name to the end of the individual LLM response log to make it easier to scan for logs of interest

Pre-Merge Checklist πŸš€

🚒 Has this modified a publishable library?

This PR:

  • [x] does not modify any publishable blocks or libraries, or modifications do not need publishing

πŸ“œ Does this require a change to the docs?

The changes in this PR:

  • [x] are internal and do not require a docs change

πŸ•ΈοΈ Does this require a change to the Turbo Graph?

The changes in this PR:

  • [x] do not affect the execution graph

πŸ›‘ What tests cover this?

  • None yet

❓ How to test this?

  1. Run a Flow

CiaranMn avatar Jun 25 '24 18:06 CiaranMn