marvin icon indicating copy to clipboard operation
marvin copied to clipboard

create context preprocessor

Open zzstoatzz opened this issue 2 years ago • 0 comments

the list of messages provided to the LLM for a given generation should only contain the most relevant messages to producing that generation:

  • most recent messages
  • system messages from top of the thread dictating bot attrs
  • select previous plugin outputs and other messages deemed "relevant" to the actor the generation is for

primarily this would solve the problem of overflowing the model-specific context window limit, but in general a context preprocessor could enable arbitrary actors to "hook into" thread content that is relevant to them

this could potentially involve embedding the thread content so we could get top k messages in a thread

zzstoatzz avatar Apr 08 '23 01:04 zzstoatzz