substrait
substrait copied to clipboard
feat(proto): Add Write relation to support multiple outputs
Currently, a Substrait plan produces a single output corresponding to its root relation. In some cases, multiple outputs are needed, e.g., for multi-table computation as well as for check-point and debug output. The solution proposed here is to add a Write relation, which is a pass-through relation that does not modify the table yet writes it out to some external sink.
I'm going to leave this open as a "generally good idea" but I'm adding the help wanted label as I don't think you're actively working on this?
AFAIK, we ended up solving this using a NamedTapRel extension, i.e., outside the scope of Substrait itself. I think this can be closed. @icexelloss?