marvin
marvin copied to clipboard
create context preprocessor
the list of messages provided to the LLM for a given generation should only contain the most relevant messages to producing that generation:
- most recent messages
- system messages from top of the thread dictating bot attrs
- select previous plugin outputs and other messages deemed "relevant" to the actor the generation is for
primarily this would solve the problem of overflowing the model-specific context window limit, but in general a context preprocessor could enable arbitrary actors to "hook into" thread content that is relevant to them
this could potentially involve embedding the thread content so we could get top k messages in a thread