Cannot pass a list as a parameter to a step

[Mathis-Z]

I would like to run some jobs that return a string each. Then I want to pass these as a list to another step. This does not work though because the list is reportedly not JSON-serializable. The problem is that validate_input in zenml.steps.entrypoint_function_utils exits early if the passed parameter is a StepArtifact but when a list of StepArtifact is passed, the function does not exit early. The same goes for other collections.

[schustmi]

The underlying problem is not the input validation but rather that passing collections of artifacts (=outputs of other steps) is currently not supported. It's on our list of todos in the near future though.

[Mathis-Z]

Ok, that is unfortunate. Is there some kind of workaround available?

[htahir1]

@Mathis-Z You can use the pattern described here - its in our near future too improve this but in the meanwhile this should suffice for many use-cases: https://docs.zenml.io/how-to/pipeline-development/build-pipelines/fan-in-fan-out

[satya23web]

why don't you use zenml.io.Serializable to wrap custom types

[schustmi]

why don't you use zenml.io.Serializable to wrap custom types

Because it doesn't exist?