azureml-examples
azureml-examples copied to clipboard
Provide example of best practice for passing objects (e.g. azureml.core.Dataset) between custom components in pipelines
What example? Describe it
In the AMLS Designer it appears that components can have a Dataset output type. What's the equivalent to be able to pass a Dataset object from the output of a custom Component A to the input of custom Component B? The only examples I see about dealing with inputs/outputs of custom components is with writing out data to paths and using the path as an input to the next component.
I guess this is simply a limitation of custom components vs components in Designer? That custom components by nature are registered separately from the pipeline run they are composed within, so there's no way to configure passing objects between them, since their scripts are registered separately, other than writing out a pickle file and picking it up in the next component via the output path of it's component? Since Designer is ran from the AMLS workspace runtime, this allows for Dataset objects to be passed between?
So is the best practice for passing Dataset objects (and other objects) between AML Custom Components within an AML Pipeline via setting an output path for a pickle file?