feat(agents): add RequirementAgent

[Tomas2D]

Which issue(s) does this pull-request address?

Ref: #846

Description

Checklist

General

• [ ] I have read the appropriate contributor guide: Python / TypeScript
• [ ] I have signed off on my commit: Python instructions / TypeScript instructions
• [ ] Commit messages and PR title follow conventional commits
• [ ] Appropriate label(s) added to PR: Python for Python changes, TypeScript for TypeScript changes

Code quality checks

• [ ] Linting passes: Python poe lint or poe lint --fix / TypeScript yarn lint or yarn lint:fix
• [ ] Formatting is applied: Python poe format or poe format --fix / TypeScript: yarn format or yarn format:fix
• [ ] Static type checks pass: Python poe type-check

Testing

• [ ] Unit tests pass: Python poe test --type unit / TypeScript yarn test:unit
• [ ] E2E tests pass: Python poe test --type e2e / TypeScript: yarn test:e2e
• [ ] Integration tests pass: Python poe test --type integration
• [ ] Tests are included (for bug fixes or new features)

Documentation

• [ ] Documentation is updated
• [ ] Embedme embeds code examples in docs. To update after edits, run: Python poe docs --type build

[terylt]

Nice work Tomas! Here are just some early comments/questions. Take them with a grain of salt, or maybe they will help with some of the documentation:

1. Do requirements require a tool? If so, I'm wondering if requirements could be an extension on the tool list itself: tools = [ThinkTool, Requirement(SearchTool...,),..]
2. Does each tool have to have a requirement if there are requirements? What's the default behavior if a tool doesn't have a requirement? It's just left to the LLM to decide?
3. Any thoughts on enabling few shot prompting on this agent?
4. Another thought is it might be interesting to have the requirements laid out in order having the position of the requirement in the list have meaning, and then could specify whether each tool in the requirements is required or optional, how many times it would be called in a row, etc. This might make it a bit easier for debugging and visualizing. Like a little tool pipeline. We've worked with a few folks that have use cases where they want their agent to call specific functions in a very defined order. May be good to have a way to render an ASCII diagram of the requirements for debugging.
5. How does the developer use the output of one tool to make determination on the next? This might be a nice example for the documentation. Also noting what attributes are available in the state object for a custom check, and things like what is in a Rule object for a custom requirement, and what's in the RequirementAgentRunState object would help the user for writing their own.
6. Thoughts on combining AskPermission into ConditionalRequirement, or giving it as another option in ConditionalRequirement? Could be a boolean or a enum.

Overall, great work! Nice feature addition.

[araujof]

Nice job, indeed!

I made some minor edits to the docs here: #853

Adding to what @terylt said, a couple of points that could be considered for this agent:

1. Thoughts on exposing some selected, additional context about the agent and tools execution as part of RequirementAgentRunOutput? For example, in addition to steps (trajectories), we could also have provisions for things like retrieval context, security guardrails context (e.g., the filters or sanitizers output), etc. These could be quite useful for building composable GUIs and debugging (similarly to how you already handle logging of the steps, which looks great, btw!).

2. This agent abstraction could be a great foundation to building supervision patterns. One aspect I think we should discuss is session management and state sharing strategies between the agent and its tools (which can also be agents). In a supervision scenario, it would be desirable to be able to customize strategies for how the agent shares state with its tools/agents. Maybe this is a topic that we need to discuss separately, but we were thinking about things like global, local, and transactional (ephemeral, scoped to an agent workflow or run) state, where one could define easily how state is managed, shared, and/or propagated.

[Tomas2D]

@terylt Thank you for the thorough review.

Do requirements require a tool? If so, I'm wondering if requirements could be an extension on the tool list itself: tools = [ThinkTool, Requirement(SearchTool...,),..]

• Yes and no. Built-in requirements (AskPermissionRequirement and ConditionalRequirement) are bound to a tool or a set of tools. However, if you examine the documentation (specifically, the last section on building to your own requirements), you will see that the requirement is not tied to any particular tool, but at the end of the day, it produces a rule that is bound to a tool.

Does each tool have to have a requirement if there are requirements? What's the default behavior if a tool doesn't have a requirement? It's just left to the LLM to decide?

• Requirements produce rules, and a rule applies to a single tool. If there is no rule for a tool, then the tool is in a default state (LLM can use it, it is visible in the system prompt, nothing changes).

Any thoughts on enabling few shot prompting on this agent?

• Few-shot prompting is not related to requirements/rules. You can just add those "few-shot" examples either to the system prompt or add them as standard messages to the agent's memory.

Another thought is it might be interesting to have the requirements laid out in order having the position of the requirement in the list have meaning, and then could specify whether each tool in the requirements is required or optional, how many times it would be called in a row, etc. This might make it a bit easier for debugging and visualizing. Like a little tool pipeline. We've worked with a few folks that have use cases where they want their agent to call specific functions in a very defined order. May be good to have a way to render an ASCII diagram of the requirements for debugging.

• Good point. If you run the demo script, you will see all traces in your terminal. Including all calls to "requirements" and a set of rules they produce. To see what the final resolution was, you can inspect the 'request' attribute of the agent's start event that gets emitted on every iteration (iteration = LLM call + related tool invocations).

How does the developer use the output of one tool to make determination on the next? This might be a nice example for the documentation. Also noting what attributes are available in the state object for a custom check, and things like what is in a Rule object for a custom requirement, and what's in the RequirementAgentRunState object would help the user for writing their own.

• Good point. Let me update the documentation so it is clearer.

Thoughts on combining AskPermissionRequirement into ConditionalRequirement, or giving it as another option in ConditionalRequirement? Could be a boolean or a enum.

• They are both doing something else. Additionally, the "ask the user" part of AskPermissionRequirement executes before a tool's handler is run. However, I see that in some cases, you may want to disable the AskPermission if, for example, the previous message contains a specific substring or other content. However, allowing wrapping requirements in other requirements can cause extra confusion, and it's not clear to me how it should work. Ideas?