Test Magentic One in AGS, Add Documentation

[victordibia]

• [ ] Test and verify support for MagneticOne in studio .Part of this might be work to migrate filesurfer and to be declarative #5607 . There might be work to verify the UI parameters or MagenticOne and its agents are well supported beyond just the json file
• [ ] Update AGS documentaion with a guide.

[victordibia]

As part of the work to fully support ⁠Magentic one teams in AGS. (and Ideally add an example impl as a default agent in AGS).

Supporting the MagenticOneTeam is already a go. Now, one of the issues I am seeing is that supporting the agents will require creating interface modifications specific for filesurfer, codeexecutionagent and magenticone coder.

All of these feel like perhaps not the right thing to do as it increases the maintenance surface and perhaps could be done by reusing AssistantAgent. AssistantAgent is sufficiently modular that we can do this now.

I am thinking of the following:

• filesurfer - fs = AssistantAgent(tool=[..mardownfilebrowser])
• code_execution - code_executor = AssistantAgent(tool=[python_code_executor])
• magenticone_coder = AssistantAgent(system_message="....")

Note: All these changes are straightforward. Only place where work is needed to wrap file browser as tool that can be added to any AssistantAgent.

This does not require any changes to the MagenticOne implementation, just that the example magentic one that will be in AGS will be implemented only using the AssistantAgent (as described above). The one drawback I see is having yet another version of magenticone. But on the bright side, its also a way to highlight the modularity of AssistantAgent and could illustrate how to attach markitdown tool abilities to any agent.

Happy to get feedback @ekzhu @afourney @husseinmozannar @gagb @jackgerrits .

[afourney]

I think there is much we can do to make this easier/composable, but my thoughts at this moment are:

• Coder - No problem.

• ComputerTerminal - This agent is deterministic. It's not backed by a model. It unconditionally executes code when present, and instructs the agent how to provide/format code, when not. So this proposal would be a rather large change in behavior and introduces an LLM call where the model needs to correctly copy, escape / string encode Python from the previous messages as parameters to a function call. This might introduce chances for hallucination -- especially for smaller models. Significant testing would be needed. There's also some important logic to consider on which messages from the history are considered for execution -- it's not just the last message.... it's every message received since last called.

• FileSurfer - Of all M1 agents, this one needs the most TLC, and I am open to ways to improve it. But, right now, FileSurfer is about a half-dozen tools that all manage the same state that persists between calls (e.g., virtual viewport, etc.) We would need to make very sure that things like resetting the agent correctly resets that state. That the state is serialized. Etc. Etc. It's much closer to WebSurfer in this way, honestly.

[victordibia]

Thanks @afourney . I think your input on ComputerTerminal makes a good case for supporting codeexecutor agent directly. I agree that filesurfer may need more design on a filesurfer tool.

[zytoh0]

@victordibia Thanks for working on this. How did Microsoft create the demo videos here with the UI? Any source code or updates on this? https://www.microsoft.com/en-us/research/articles/magentic-one-a-generalist-multi-agent-system-for-solving-complex-tasks/