Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test Magentic One in AGS, Add Documentation #5751

Open
2 tasks
Tracked by #4006
victordibia opened this issue Feb 28, 2025 · 3 comments
Open
2 tasks
Tracked by #4006

Test Magentic One in AGS, Add Documentation #5751

victordibia opened this issue Feb 28, 2025 · 3 comments
Assignees
Labels
proj-studio Related to AutoGen Studio.
Milestone

Comments

@victordibia
Copy link
Collaborator

victordibia commented Feb 28, 2025

  • Test and verify support for MagneticOne in studio .Part of this might be work to migrate filesurfer and to be declarative Make FileSurfer support component config serialization #5607 . There might be work to verify the UI parameters or MagenticOne and its agents are well supported beyond just the json file
  • Update AGS documentaion with a guide.
@victordibia victordibia added the proj-studio Related to AutoGen Studio. label Feb 28, 2025
@victordibia victordibia self-assigned this Feb 28, 2025
@victordibia victordibia added this to the 0.4.x-studio milestone Mar 1, 2025
@victordibia
Copy link
Collaborator Author

As part of the work to fully support ⁠Magentic one teams in AGS. (and Ideally add an example impl as a default agent in AGS).

Supporting the MagenticOneTeam is already a go.
Now, one of the issues I am seeing is that supporting the agents will require creating interface modifications specific for filesurfer, codeexecutionagent and magenticone coder.

All of these feel like perhaps not the right thing to do as it increases the maintenance surface and perhaps could be done by reusing AssistantAgent. AssistantAgent is sufficiently modular that we can do this now.

I am thinking of the following:

  • filesurfer - fs = AssistantAgent(tool=[..mardownfilebrowser])
  • code_execution - code_executor = AssistantAgent(tool=[python_code_executor])
  • magenticone_coder = AssistantAgent(system_message="....")

Note: All these changes are straightforward. Only place where work is needed to wrap file browser as tool that can be added to any AssistantAgent.

This does not require any changes to the MagenticOne implementation, just that the example magentic one that will be in AGS will be implemented only using the AssistantAgent (as described above).
The one drawback I see is having yet another version of magenticone. But on the bright side, its also a way to highlight the modularity of AssistantAgent and could illustrate how to attach markitdown tool abilities to any agent.

Happy to get feedback @ekzhu @afourney @husseinmozannar @gagb @jackgerrits .

@afourney
Copy link
Member

afourney commented Mar 1, 2025

I think there is much we can do to make this easier/composable, but my thoughts at this moment are:

  • Coder - No problem.

  • ComputerTerminal - This agent is deterministic. It's not backed by a model. It unconditionally executes code when present, and instructs the agent how to provide/format code, when not. So this proposal would be a rather large change in behavior and introduces an LLM call where the model needs to correctly copy, escape / string encode Python from the previous messages as parameters to a function call. This might introduce chances for hallucination -- especially for smaller models. Significant testing would be needed. There's also some important logic to consider on which messages from the history are considered for execution -- it's not just the last message.... it's every message received since last called.

  • FileSurfer - Of all M1 agents, this one needs the most TLC, and I am open to ways to improve it. But, right now, FileSurfer is about a half-dozen tools that all manage the same state that persists between calls (e.g., virtual viewport, etc.) We would need to make very sure that things like resetting the agent correctly resets that state. That the state is serialized. Etc. Etc. It's much closer to WebSurfer in this way, honestly.

@victordibia
Copy link
Collaborator Author

Thanks @afourney .
I think your input on ComputerTerminal makes a good case for supporting codeexecutor agent directly.
I agree that filesurfer may need more design on a filesurfer tool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
proj-studio Related to AutoGen Studio.
Projects
None yet
Development

No branches or pull requests

2 participants