SDK - Components - Creating graph components from python pipeline function #2273
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
`create_graph_component_from_pipeline_func` converts python pipeline function to a graph component object that can be saved, shared, composed or submitted for execution.
Example:
producer_op = load_component(component_with_0_inputs_and_2_outputs)
processor_op = load_component(component_with_2_inputs_and_2_outputs)
def pipeline1(pipeline_param_1: int):
producer_task = producer_op()
processor_task = processor_op(pipeline_param_1, producer_task.outputs['Output 2'])
return OrderedDict([
('Pipeline output 1', producer_task.outputs['Output 1']),
('Pipeline output 2', processor_task.outputs['Output 2']),
])
graph_component = create_graph_component_from_pipeline_func(pipeline1)
Non-public create_graph_component_spec_from_pipeline_func creates ComponentSpec Public create_graph_component_from_pipeline_func creates component and writes it to file.
…gnature Stopped humanizing the input names for now. I think it's benefitial to extract interface from function signature the same way for both container and graph python components.
Remove the component spec from component reference unless it will make the reference empty or unless explicitly asked by the user
Ark-kun
force-pushed
the
SDK---Components---Creating-graph-components-from-python-pipeline-function---2
branch
from
1fa60fb to
e410b4d
Compare
numerology
reviewed
Oct 2, 2019
Ark-kun
force-pushed
the
SDK---Components---Creating-graph-components-from-python-pipeline-function---2
branch
from
34f1431 to
66ebd74
Compare
|
/lgtm |
Ark-kun
force-pushed
the
SDK---Components---Creating-graph-components-from-python-pipeline-function---2
branch
from
66ebd74 to
1e59cfc
Compare
Looks like I did |
|
/lgtm |
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: Ark-kun The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The experimental
create_graph_component_from_pipeline_funccreates graph component definition from a python pipeline function. The component file can be published for sharing.Pipeline function is a function that only calls component functions and passes outputs to inputs.
This feature is experimental and lacks support for some of the DSL features like conditions and loops.
Only pipelines consisting of loaded components or python components are supported at this moment(no manually created ContainerOps or ResourceOps).
Example:
This change is