Graph run_stream

[asaf]

Hey,

Graphs may take long time to execute, especially when dealing with multi agents with iterations,

Would love to see streaming graph execition

Effects #704 #695

[samuelcolvin]

I think you want Graph.next.

If you want to implement streaming within a node, you can just implement that as a new method on the node. I'll try to add an example soon.

[dmontagu]

Also, if you want within-node streaming, I think our current plan is to combine the Graph.next referenced by Samuel above with a user-defined method on the node that does streaming, and then manually handling the streaming in the code that loops over the nodes via Graph.next. I am planning to take this approach to handling streaming in the Graph-based refactoring of pydantic_ai.Agent, so hopefully there will be an example soon.

[asaf]

I guess having stream() on the graph level could be nice but raises more complexity such when graphs should be interrupted (e.g., when human in the loop is required), so I see your point of the Graph.next.

Next() moves the execution flow entirely to the control of the user, which probably would work with any usecase.

Currently the Question Graph blocks until the entire result is returned.

@samuelcolvin invoking next() executes run of the current node, I assume we can:

1. Add a new method on the node such stream()
2. Invoke the stream() method manually and await for it to complete, which may modify the current node's object local state with the result of the stream.
3. Invoke next() that invokes the run method on the current node, which can use the local object state set by the stream.
4. Decide where to go next based on the object state plus optionally copy some state from local object to the graph's state.

[dmontagu]

Yes, this is precisely what we had in mind. It's reassuring to see someone else independently working out the details in the same way!

If you run into any issues achieving your goals while taking this approach, we'll definitely want to address them — please let us know.

[asaf]

I see your point :)
I'll keep this issue opened as a reminder to create a graph sample with streaming! 10x guys!

[github-actions]

This issue is stale, and will be closed in 3 days if no reply is received.

[xflashxx]

Eagerly waiting for this! ❤️