substrait icon indicating copy to clipboard operation
substrait copied to clipboard

feat(proto): Add Write relation to support multiple outputs

Open rtpsw opened this issue 3 years ago • 2 comments

Currently, a Substrait plan produces a single output corresponding to its root relation. In some cases, multiple outputs are needed, e.g., for multi-table computation as well as for check-point and debug output. The solution proposed here is to add a Write relation, which is a pass-through relation that does not modify the table yet writes it out to some external sink.

rtpsw avatar Jul 03 '22 13:07 rtpsw

I'm going to leave this open as a "generally good idea" but I'm adding the help wanted label as I don't think you're actively working on this?

westonpace avatar Mar 01 '23 21:03 westonpace

AFAIK, we ended up solving this using a NamedTapRel extension, i.e., outside the scope of Substrait itself. I think this can be closed. @icexelloss?

rtpsw avatar Mar 03 '23 07:03 rtpsw