Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logic node post miso #58

Merged
merged 56 commits into from
Dec 25, 2024
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
bbdbdcb
Refactor node handling and add RouterNode support in flow logic
preet-bhadra Dec 20, 2024
39b784b
Add source_handle and target_handle to workflow link schemas and upda…
preet-bhadra Dec 20, 2024
bbae012
Update example_router.json coordinates and enhance DynamicNode title …
preet-bhadra Dec 20, 2024
70460bd
Enhance RouterNode and DynamicSchemaNode with dynamic model generatio…
preet-bhadra Dec 20, 2024
8018578
Merge branch 'main' of https://github.com/sevn-ai/pyspur_dev into Log…
preet-bhadra Dec 20, 2024
a302941
fix: improve type handling in DynamicSchemaNode to raise errors for u…
preet-bhadra Dec 20, 2024
68ed3f1
refactor: enhance dynamic schema handling and input validation in Rou…
preet-bhadra Dec 21, 2024
ebe1fa3
Merge branch 'main' of https://github.com/sevn-ai/pyspur_dev into Log…
preet-bhadra Dec 21, 2024
cf76bad
fix: use inputSchema variables instead of input nodes inside expressions
JeanKaddour Dec 21, 2024
92110c5
Merge remote-tracking branch 'origin/main' into LogicNodePostMISO
srijanpatel Dec 21, 2024
5c6eaac
make router node use BaseNode as base
srijanpatel Dec 22, 2024
faf226f
Refactor WorkflowExecutor to handle router nodes and optional outputs
srijanpatel Dec 22, 2024
408565f
refactor: rename IfElseNode to RouterNode and update related configur…
preet-bhadra Dec 23, 2024
472abd9
Merge branch 'LogicNodePostMISO' of https://github.com/sevn-ai/pyspur…
preet-bhadra Dec 23, 2024
e0ec32a
feat: add debug print statements to WorkflowExecutor and RouterNode f…
preet-bhadra Dec 23, 2024
34596a7
feat: enhance RouterNode configuration and add nested variable support
preet-bhadra Dec 24, 2024
28f5261
feat: enhance RouterNode configuration to include route_map for impro…
preet-bhadra Dec 24, 2024
184ad9f
refactor: remove duplicate edges and prevent duplicate connections in…
srijanpatel Dec 24, 2024
f4e3237
use source node title as default target handle name
srijanpatel Dec 24, 2024
b9e2d96
feat: update RouterNode configuration to use route_map and enhance ed…
preet-bhadra Dec 24, 2024
91cdd14
feat: enhance SingleLLMCallNode to allow extra fields in input and im…
preet-bhadra Dec 24, 2024
5092f6e
chore: remove example_router.json as it is no longer needed
preet-bhadra Dec 24, 2024
2ce0a14
Refactor: Update useSaveWorkflow to improve handle naming consistency
srijanpatel Dec 24, 2024
9a362d6
Refactor: Update useSaveWorkflow to use route_map for RouterNode conf…
srijanpatel Dec 24, 2024
a0635b3
ellipsis fix in frontend/src/hooks/useSaveWorkflow.ts
srijanpatel Dec 24, 2024
473cbb5
bug fix for ellipsis fix
srijanpatel Dec 24, 2024
5256b1a
Refactor: Update useSaveWorkflow to use typed routeMap in RouterNode …
srijanpatel Dec 24, 2024
f2362ef
Refactor: Move router-related schemas to a new file and clean up Rout…
preet-bhadra Dec 24, 2024
729a705
Refactor route condition types
srijanpatel Dec 24, 2024
d27cc26
Refactor: Initialize route_map with default route in RouterNodeConfig
srijanpatel Dec 25, 2024
94877cd
special handling not needed for router in JSPydantic, include require…
srijanpatel Dec 25, 2024
ed51d2d
Refactor: Remove commented out code for initializing routes in Router…
srijanpatel Dec 25, 2024
fa1f1ed
Refactor: Deduplicate nodes and edges in DynamicNode and flowSlice
srijanpatel Dec 25, 2024
c54c721
Refactor: Update DynamicNode to include connection fromNode details i…
srijanpatel Dec 25, 2024
dbd90c0
Refactor: Update inputVariables in RouterNode to include connected no…
srijanpatel Dec 25, 2024
a830b98
Refactor: Initialize route_map with previously set values
srijanpatel Dec 25, 2024
d5558b1
Refactor: Update RouterNode to dynamically calculate node width based…
srijanpatel Dec 25, 2024
08abe4f
Refactor: Update RouterNode to display input variables as multiline o…
srijanpatel Dec 25, 2024
68c23c5
use dot notation for router handles
srijanpatel Dec 25, 2024
df21b43
switch of verbose mode in litellm and fix formatting
srijanpatel Dec 25, 2024
40addef
fix warning of variable used before declaration
srijanpatel Dec 25, 2024
e8a8530
fix type
srijanpatel Dec 25, 2024
949d6da
bug fix rerender loop
srijanpatel Dec 25, 2024
f3e74d3
Refactor useSaveWorkflow to handle missing title in node data
srijanpatel Dec 25, 2024
5b60c10
Refactor flowSlice to update target routerNode's output schema with s…
srijanpatel Dec 25, 2024
4ce7f69
Refactor get_nested_value function in RouterNode to use dot notation …
srijanpatel Dec 25, 2024
b90608c
Refactor BaseNode's input handling to support dictionaries of BaseNod…
srijanpatel Dec 25, 2024
3660928
skip executing nodes that have predecessor output None
srijanpatel Dec 25, 2024
860f779
Refactor SingleLLMCallNode to improve code organization and error han…
srijanpatel Dec 25, 2024
f1d5e53
fix output models or router node
srijanpatel Dec 25, 2024
9778e44
Refactor flowSlice.ts to update connected RouterNodes' output schema …
srijanpatel Dec 25, 2024
0208202
fix router nodes status and output display
srijanpatel Dec 25, 2024
efa241f
Merge remote-tracking branch 'origin/main' into LogicNodePostMISO
srijanpatel Dec 25, 2024
6c1efa7
Refactor RouterNode.tsx to add aria-label attributes to Select compon…
srijanpatel Dec 25, 2024
4951d70
memoize incoming edges selector
srijanpatel Dec 25, 2024
2b8c1a8
comment merge node for now
srijanpatel Dec 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 52 additions & 8 deletions backend/app/execution/workflow_executor.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import asyncio
from datetime import datetime
from typing import Any, Dict, Iterator, List, Optional, Set
from typing import Any, Dict, Iterator, List, Optional, Set, Tuple

from ..nodes.base import BaseNodeOutput
from ..nodes.factory import NodeFactory
Expand Down Expand Up @@ -35,11 +35,13 @@ def __init__(
self.context = context
self._node_dict: Dict[str, WorkflowNodeSchema] = {}
self._dependencies: Dict[str, Set[str]] = {}
self._node_tasks: Dict[str, asyncio.Task[BaseNodeOutput]] = {}
self._node_tasks: Dict[str, asyncio.Task[Optional[BaseNodeOutput]]] = {}
self._initial_inputs: Dict[str, Dict[str, Any]] = (
{}
) # <node_id, <input for the node>>
self._outputs: Dict[str, BaseNodeOutput] = {} # <node_id, < node output>>
self._outputs: Dict[str, Optional[BaseNodeOutput]] = (
{}
) # <node_id, < node output>>
self._build_node_dict()
self._build_dependencies()

Expand All @@ -54,9 +56,22 @@ def _build_dependencies(self):
dependencies[link.target_id].add(link.source_id)
self._dependencies = dependencies

def _get_source_handles(self) -> Dict[Tuple[str, str], str]:
"""Build a mapping of (source_id, target_id) -> source_handle for router nodes only"""
source_handles: Dict[Tuple[str, str], str] = {}
for link in self.workflow.links:
source_node = self._node_dict[link.source_id]
if source_node.node_type == "RouterNode":
if not link.source_handle:
raise ValueError(
f"Missing source_handle in link from router node {link.source_id} to {link.target_id}"
)
source_handles[(link.source_id, link.target_id)] = link.source_handle
return source_handles

def _get_async_task_for_node_execution(
self, node_id: str
) -> asyncio.Task[BaseNodeOutput]:
) -> asyncio.Task[Optional[BaseNodeOutput]]:
if node_id in self._node_tasks:
return self._node_tasks[node_id]
# Start task for the node
Expand All @@ -68,14 +83,14 @@ def _get_async_task_for_node_execution(
self.task_recorder.create_task(node_id, {})
return task

async def _execute_node(self, node_id: str) -> BaseNodeOutput:
async def _execute_node(self, node_id: str) -> Optional[BaseNodeOutput]:
if node_id in self._outputs:
return self._outputs[node_id]
node = self._node_dict[node_id]

# Wait for dependencies
dependency_ids = self._dependencies.get(node_id, set())
predecessor_outputs: List[BaseNodeOutput] = []
predecessor_outputs: List[Optional[BaseNodeOutput]] = []
if dependency_ids:
predecessor_outputs = await asyncio.gather(
*(
Expand All @@ -84,12 +99,36 @@ async def _execute_node(self, node_id: str) -> BaseNodeOutput:
)
)

node_input = dict(zip(dependency_ids, predecessor_outputs))
# Get source handles mapping
source_handles = self._get_source_handles()

# Build node input, handling router outputs specially
node_input = {}
for dep_id, output in zip(dependency_ids, predecessor_outputs):
source_node = self._node_dict[dep_id]
if source_node.node_type == "RouterNode":
# For router nodes, we must have a source handle
source_handle = source_handles.get((dep_id, node_id))
if not source_handle:
raise ValueError(
f"Missing source_handle in link from router node {dep_id} to {node_id}"
)
# Get the specific route's output from the router
route_output = getattr(output, source_handle, None)
if route_output is not None:
node_input[dep_id] = route_output
else:
node_input[dep_id] = output

# Special handling for InputNode - use initial inputs
if node.node_type == "InputNode":
node_input = self._initial_inputs.get(node_id, {})

# if any of the inputs are None, return None
if any([v is None for v in node_input.values()]):
self._outputs[node_id] = None
return None

node_instance = NodeFactory.create_node(
node_name=node.title, node_type_name=node.node_type, config=node.config
)
Expand Down Expand Up @@ -174,7 +213,12 @@ async def run(
# Wait for all tasks to complete
await asyncio.gather(*self._node_tasks.values())

return self._outputs
# return the non-None outputs
return {
node_id: output
for node_id, output in self._outputs.items()
if output is not None
}

async def __call__(
self,
Expand Down
33 changes: 31 additions & 2 deletions backend/app/nodes/dynamic_schema.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,14 @@
"""
This module defines classes and methods for handling dynamic schemas
using Pydantic models. It includes configurations for nodes with dynamic
input/output schemas and a base class for such nodes.
"""

from abc import ABC
from typing import Dict
from typing import Dict, Any

from pydantic import BaseModel, create_model
from .base import BaseNode
from pydantic import BaseModel


class DynamicSchemaNodeConfig(BaseModel):
Expand All @@ -15,6 +21,29 @@ class DynamicSchemaNodeConfig(BaseModel):
class DynamicSchemaNode(BaseNode, ABC):
"""Base class for nodes with dynamic input/output schemas."""

@staticmethod
def get_model_for_schema_dict(schema: Dict[str, str], schema_name: str) -> Any:
"""
Create and return a Pydantic model based on a schema dictionary.
"""
type_mapping: Dict[str, Any] = {
"Any": Any,
"str": str,
"int": int,
"float": float,
"bool": bool,
"any": Any, # Handle lowercase "any" by mapping it to Any
}

fields = {}
for key, value in schema.items():
# If the type is not recognized, raise an error instead of using eval
if value not in type_mapping:
raise ValueError(f"Unsupported type '{value}' in schema for field '{key}'")
fields[key] = (type_mapping[value], ...)

return create_model(schema_name, **fields) # type: ignore

def setup(self) -> None:
"""Set up dynamic input/output models based on configuration."""
# check if the config is an instance of DynamicSchemaNodeConfig
Expand Down
157 changes: 0 additions & 157 deletions backend/app/nodes/logic/if_else.py

This file was deleted.

Loading