superset
superset copied to clipboard
apache
'Explore' button on SQL Lab view disabled when connected to Apache Pinot as a database
I connected superset to pinot successfully and was able to build SQL lab queries only to find out that Superset does not support Exploring of SQL Lab virtual data as a chart if the connected database is Apache Pinot. (The “Explore” button is disabled)
I am using Superset version 1.0.1
Screenshot
Looks like it's related to the fact that pinot does not support nested queries. For other databases superset creates a virtual table dataset which is used as the FROM table in the explore query.
@yousoph I think we can probably close this issue since (a) it appears this button is disabled on purpose, so it's not quite a bug, and (b) The issue is a bit vintage in terms of superset version.
That said, if the button is disabled intentionally, do you think it's reasonable to have a tooltip explaning why?
Also, I'm trying to get my hands on a Pinot test DB to see if this works in the newer workflow where we DO NOT create a virtual dataset by default. If anyone can help test this, it'd be appreciated
Hi @rusackas , were you able to test this by any chance? With Pinot having V2 engine in beta (which supports joins), can this button be enabled?
It would be quite a game changer.
On Sat, 18 Nov 2023, 03:41 Kamal Chavda, @.***> wrote:
Hi @rusackas https://github.com/rusackas , were you able to test this by any chance? With Pinot having V2 engine in beta (which supports joins), can this button be enabled?
— Reply to this email directly, view it on GitHub https://github.com/apache/superset/issues/13623#issuecomment-1816986741, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFPNQ4QF3D3DA2IL5EYOFMTYE64YLAVCNFSM4ZGU3RQ2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBRGY4TQNRXGQYQ . You are receiving this because you commented.Message ID: @.***>
Sounds like @ege-st is experiencing this still, alas. @rusackas were you ever able to test this out? Is there a committer or community member who could tackle this - maybe we could reach out to the Pinot project since it is also under the Apache Software Foudnation?
Nope... I'm afraid I wasn't able to test, and don't have a Pinot DB currently. Not sure if @sadpandajoe has one handy. @john-bodley do we have any Pinot-sipping friends to reach out to?
I work in StarTree and volunteer to help resolve the issue. Found why the CREATE CHART is disabled for Pinot:
- Button is disabled HERE if allows_subquery is not defined or false
- For PinotEngineSpec , allows_subqueries is false
- Features table summarizes the capabilities of all db engines
We have several Superset dashboards using Pinot in our dev and QA environments. Why does CREATE CHART depend on subquery support? Pinot supports subquery, join, etc., using the multi-stage query engine. We can enable this mode for all features depending on the subquery support. We can enable the multi-stage engine based on an engine configuration, and then the features requiring subqueries will only work in this mode, but the current mode will also be there.
Thanks for looking into it! I think @betodealmeida might be best qualified to answer that.
Why does CREATE CHART depend on subquery support?
Because the chart builder wraps the SQL in a subquery — the SQL from SQL Lab becomes the inner query, and the controls in the chart builder define the outer query.
Pinot supports subquery, join, etc., using the multi-stage query engine. We can enable this mode for all features depending on the subquery support. We can enable the multi-stage engine based on an engine configuration, and then the features requiring subqueries will only work in this mode, but the current mode will also be there.
If that's the case we should change allows_subqueries to True in the Pinot DB engine spec, and have good error messaging when the user tries to build a chart in a Pinot deployment that's not configured with the multi-stage engine (a message that says something like "In order to explore data in Pinot you need an engine with support for subqueries", eg.
Thanks @betodealmeida for suggestion, let me create the PR and we can iterate on that to improve the user experience.
I merged #28364 to address this. But now I see the suggestion from Beto above that says:
Have good error messaging when the user tries to build a chart in a Pinot deployment that's not configured with the multi-stage engine (a message that says something like "In order to explore data in Pinot you need an engine with support for subqueries" It would be great to get that implemented now that this condition can be encountered. @soumitra-st you mention "we can iterate on that to improve the user experience," could you take a shot at this PR?
Also it would be great if folks who have commented above asking for this feature can test out the Superset master branch where this is now merged.