async tapeagents

[rizar]

Here's one way:

• we add coroutines:
  • LLM.agenerate
  • Node.agenerate_steps
  • Agent.arun_iteration
  • Agent.arun
• the root agent remains different from other agents in the organization
  • the application launches root.arun() with asyncio.create_task
  • Agent.arun starts by identifying which nodes in which agents can run concurrently and creates asyncio tasks for them (Agent.run_iteration)
  • Agent.arun then runs a queue-based loop. The application provides a message queue.
  • when the root receives a message that a node of a subagent A finished, it does the following
    • updates the tape
    • identifies which new nodes of which new agents should be run (looks at Call and other things).
    • identifies actions and send them back to the application through the queue
  • the root can also receives observations from the application through the queue, the processing is overall same as the above
• important remarks
  • we need to improve tape parsing to handle interleaving steps from different agents working concurrently
  • TapeViewStack should no longer be a stack, it should be a set of tasks with dependencies. We can refactor it and call TapeViews and use the same one for both asynchronous and asynchronous orchestration
  • we can make it so that every agent naturally can be run in both sync and async mode, as long as it is build from nodes that support both sync and async
  • this will let us use all the current tapes in tests to verify the async implementation, though perhaps we will need to update the metadata

TL;DR:

• the above is a minimal plan that replace synchronous function calls with asynchronous messaging through a queue
• the communication topology remains the same: all actions and observations must go through the root agent in order to be noted on the tape

[vasudevanubrolu]

Hi @rizar can I take this up ?