substrait icon indicating copy to clipboard operation
substrait copied to clipboard

feat: add explain TableFunction for substrait

Open drin opened this issue 1 year ago • 6 comments

This adds a new table function, explain_substrait, as well as a new execution kernel for the function and binding kernel. This reuses the FromSubstraitFunctionData structure since it contains all the pieces we would need.

The reason this is useful is because explain naively returns the table function itself instead of the plan it may execute. In the case of executing substrait (e.g. from_substrait(...)), this would be a single operator for the from_substrait function itself. If, instead one calls explain_substrait(...), the result would be an explain plan for what the substrait extension translates into duckdb (the result of SubstraitPlanToDuckDBRel)

drin avatar Oct 29 '24 19:10 drin

I realized this functionality would be generally useful. For now, this is mostly a placeholder until I can maybe test this at the end of the week and write some tests, etc.

drin avatar Oct 29 '24 19:10 drin

Hey Aldrin, thanks for the PR, I think it looks good!

Could you just add a few tests?

pdet avatar Oct 30 '24 09:10 pdet

@drin , can you rebase and add some tests?

Thanks

jacques-n avatar Nov 13 '24 20:11 jacques-n

99752e4 is the rebase, and 5707b53 fixes a few issues and adds a test.

Unfortunately, I am developing with protobuf v28.3 and modified my duckdb to link against mohair-substrait: so, it is difficult for me to actually test the changes.

at the moment, I'm getting the following error when trying to run against a mostly accurate version of this PR:

 symbol not found in flat namespace '__ZN6google8protobuf8internal26fixed_address_empty_stringE'

I can continue trying to tie this off in a few days once I reach a good checkpoint with other work.

drin avatar Nov 14 '24 01:11 drin

@pdet I was wondering if ExplainRelation::GetTableRef would return a TableRef that would eventually resolve to an equivalent QueryResult as Relation::Explain does?

In other words, I am currently returning a QueryResult via substrait_extension.cpp#L312 and using a normal TableFunction (uses a normal bind function). I'm wondering if I could construct an ExplainRelation similarly to how the root ProjectionRelation is constructed in TransformRootOp, and then I can call GetTableRef on that as you do for the other substrait functions (e.g. from_substrait).

Thanks!

drin avatar Nov 14 '24 01:11 drin

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

CLAassistant avatar Oct 07 '25 17:10 CLAassistant