microsoft
Implement a drag and drop team builder interface for AGS
The current rewrite of AutoGen studio has focused on representing teams declaratively in JSON. It would be great to add a flexible GUI to let users modify team configurations. A team is made of up compontents - agents (models, tools, etc), termination conditions etc.
- [x] Visualize team config (what are the agents, what are the tools, models etc)
- [x] Add/remove components (agents, teams, models) either directly in JSON or via UI
Component Serialization Decisions
The team (which defines the main workflow/orchestration behaviors) is the primary component in AGS.
There are two potential approaches for storing declarative team (and other component) specifications in the same internal ags database:
Fully Linked Components
In this approach, each component (model, agent, team, memory etc) maintains relationships at the database level. For example, an Agent table row would have foreign keys to its models and tools. Fetching a team's complete configuration requires traversing these relationships to assemble the full picture.
Pros:
- Components stay synchronized - when a model or tool is updated, all agents using it automatically see the change
- Referential integrity is enforced by the database
- Storage efficiency since components are stored once and referenced many times
Cons:
- Complex fetching logic requiring multiple joins to assemble full configurations
- Updates must cascade through relationships
- Schema changes needed to support new component types (a deployment challenge each time AGS users update their app)
- Harder to version control since components can change underneath a team
- More complex deployment as schema updates needed for new features
Independent Components
This approach serializes the complete configuration of each component. Rather than maintaining relationships, a team's configuration includes the full definition of all its agents, including their tools, models, and other settings. When you store a team using the weather_api tool, that tool's entire configuration is part of the team definition.
Pros:
- Configurations are self-contained and portable
- Simple updates - just modify a single record, test it and reuse elsewhere (helps with trust)
- Easy versioning since each team captures a complete snapshot
- Schema flexibility - new component types don't require database changes
- Simpler deployment with less database coordination
Cons:
- Storage duplication when components are reused across teams (not really an issue, configurations are very light).
- No automatic propagation of component updates (we can mitigate this by providing UX that helps users easily validate/test all components to build trust)
- UX needs to clearly communicate the independence of configurations
- Potential confusion if users expect linked behavior (UX can help here)
Key Decision Factors
The choice depends heavily on use case. For experimentation and reproducibility, Independent Components shines - each team captures a precise snapshot of its configuration at a point in time. For production systems prioritizing consistency, Fully Linked Components ensures changes propagate automatically.
A hybrid approach storing both relationships and snapshots is possible but adds complexity. The benefits of Independent Components - reproducibility, schema flexibility, and configuration portability - likely outweigh the storage overhead from duplication for most use cases.
Happy to get thoughts here. @husseinmozannar , @ekzhu , @JingyaChen
Thanks @victordibia for the analysis.
Thoughts:
- Storage cost decreases faster than compute, so storage overhead is not an issue.
- Important to associate class version number (e.g., a hash code) with each serialized component, and ability to retrieve the provider for each version. When an irreversible breaking change in the underlying agent/tool/team class happens in the future we can use the version number to resolve or migrate the stored component to the latest version.
- A provider class maps (serialized component, version) -> (component). On store, always migrate to the latest version.
- Independent component with loose, un-linked, key-value like schema is good to start. Otherwise, the overhead of schema migration will be burdensome when code changes frequently.
I think we also want to separate these two concepts for component:
- Component definition: a JSON serialized config that can be used to create a new component instance.
- Component state: a JSON serialized state that can be used to bring a component back to an existing state.
Separating these two allows us to ease the breaking changes: if a change impacts just the definition, we don't need to worry about also updating the state, and vice versa. This also allows us to run analytics on the state without the analytic code to reconstruct the components, as some components can be heavy and doing so consumes a lot of compute.
More regarding the state.
I think in the beginning we might want to consider supporting only "ephemeral" components: that is their states exists in memory only. We still store their definitions. So, user can retrieve their designed components, but every time they do the components are all new.
We can just have an export function to export all the state in JSON, if user want to see. But no import to start.
Good ideas and points @ekzhu
- Agreed that we should start with independent components each with a version number. Current spec has version numbers
- Agree that state is different from definition/spec. The current work in AGS has focused on definition/spec. #4222 will look at how best to persist exchanged messages. If we do this right, the structure of state should not be far from the actual messages exchanged. This should allow us some lattitude to "restore" agent state by loading messages they sent previously?
Current spec has version numbers
👍 👍
This should allow us some lattitude to "restore" agent state by loading messages they sent previously?
Yep. I think that's right.