data-prep-kit icon indicating copy to clipboard operation
data-prep-kit copied to clipboard

pipeline transform

Open blublinsky opened this issue 1 year ago • 4 comments

Why are these changes needed?

Initial implementation of pipeline transform

Related issue number (if any).

blublinsky avatar Sep 19 '24 20:09 blublinsky

@daw3rd , I looked over this PR and it is OK for me, but leave it for your final verdict.

roytman avatar Sep 23 '24 05:09 roytman

Can we run a pipeline outside of the runtime/launcher? If so, I'd like to see a test or other example of how this is done? If not, we need it.

For running a pipeline outside a runtime we have kfp. We can not pass values in memory outside of runtime

blublinsky avatar Sep 23 '24 14:09 blublinsky

Can we run a pipeline outside of the runtime/launcher? If so, I'd like to see a test or other example of how this is done? If not, we need it.

For running a pipeline outside a runtime we have kfp. We can not pass values in memory outside of runtime

This is a problem and does not meet requirements defined in the issue. https://github.com/IBM/data-prep-kit/issues/374#issuecomment-2353447356

daw3rd avatar Sep 23 '24 14:09 daw3rd

Can we run a pipeline outside of the runtime/launcher? If so, I'd like to see a test or other example of how this is done? If not, we need it.

For running a pipeline outside a runtime we have kfp. We can not pass values in memory outside of runtime

This is a problem and does not meet requirements defined in the issue. #374 (comment)

The requirement does not specify the ability to run across runtimes. Besides, where is this requirement coming from?

blublinsky avatar Sep 23 '24 14:09 blublinsky

added as transform chain in #1329

swith005 avatar Jun 26 '25 15:06 swith005