[PowerBI] Data Ingestion Report (KQL Version) `AzureDataExplorer` Data Source Path Does Not Support OAuth with PowerBI VNet Data Gateway

[AErmie]

🐛 Problem

After publishing the Data Ingestion report to PowerBI.com, we attempted to follow the process to configure the gateway and cloud connections. While we were able to configure the VNet connection for most sources in the semantic model (namely: Web, and AzureDataLakeStorage), when trying to connect the AzureDataExplorer to the VNet, this process does not actually complete properly (and never appears in the data source options).

Even when attempting to create the new connection, the create itself works, but this is not displayed in the data source post-connection creation.

[!NOTE] It is interesting that the Connection Type is required, though it is disabled. The value should be Azure Data Explorer (Kusto).

👣 Repro steps

1. Deploy the FTK in a "private" configuration
2. Deploy a Power BI VNet Data Gateway integration
3. Publish the latest Data Ingestion report
4. Attempt to configure the Extension{"extensionDataSourceKind":"AzureDataExplorer","extensionDataSourcePath":"ADX_CLUSTER_NAME"} data source to the VNet

If you check the Data Source Credentials section, it shows that connectivity has been established for all other data sources (all using OAuth). If you attempt to edit credentials for the AzureDataExplorer and configure it manually (still using OAuth), it returns the error: "The OAuth authentication method isn't supported for this data source."

🤔 Expected

All data sources and connection should support OAuth. If the Data Ingestion Report's Azure Data Explorer (Kusto) data source does not support this authentication method, then additional options should be available.

ℹ️ Additional context

The FTK deployment configuration we are using, is the "private" mode, with an ADX cluster. We have also configured a Virtual Network Data Gateway.

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.

[flanakin]

@ro100e @MSBrett Can y'all look into this one?

[AErmie]

ℹ FYI, this is also affecting the Cost Summary (KQL version) report as well.

[MSBrett]

@AErmie - I recommend the checkbox "Report viewers can only access..." so that the user's creds are presented to ADX, not whatever is cached in PBI.

Is PBI running in CMMC?

[AErmie]

Thanks @MSBrett, but that doesn't resolve the original authentication issue (though I agree with passing the User's creds).

The data source for Extension{"extensionDataSourceKind":"AzureDataExplorer","extensionDataSourcePath":"bcgov-live-finops-adx.canadacentral"} does not support OAuth, whereas that's the only option available to select.

I'm not familiar with "CMMC" stands for (in relation to PowerBI). We have the Virtual Network Data Gateway configured into the VNet (obviously), along with using Microsoft Fabric as it's capacity.

The gateway itself shows as "online":

Also, the Azure Data Explorer (Kusto) connection type (that is successfully created, but unable to be linked in the report data source), even shows as "online", but for some reason doesn't show as an option for the data source.

[!NOTE] The other non-gateway connections were auto-created when the VNet Data Gateway was originally created.

[AErmie]

Quick update (concerning the Cost Summary report). I removed it from PowerBI.com, and then re-published it, and now AzureDataExplorer data source connection (that was successfully created before), appears/is available!

I'll try with the other reports as well. Very odd.

[AErmie]

Discovery!!

Through additional testing, I discovered the following (potential) "root cause".

Although the PowerBI.com UI provides the interface to create the ADX connection for the data source (and creates it successfully), that Maps to option never appears. This is even though the connection itself is "online".

When you first open any of the PowerBI (KQL) report template files (ie CostSummary.kql.pbit), you are prompted to provide (among other things), the Cluster URL.

If you notice the example, and more importantly the tooltip, it says that you can use name.region as a shortform. However, when I used that (for every PBI report), I encountered the Add to VNet / data source issue. When I used the FULL URI instead (ie. https://ADX_CLUSTER_NAME.canadacentral.kusto.windows.net), only then did the successfully created AzureDataExplorer data source connection appear for selection!

[MSBrett]

@AErmie. Good find.
Looks like PBI has some requirements around naming of the datasource for the managed gateway?

To paraphrase for the docs - 'When using private endpoints in conjunction with a Power BI data gateway make sure to use the full cluster URL rather than the abbreviated version.'

[AErmie]

Thanks @MSBrett. I couldn't find that in the documentation. It might be helpful to highlight that (maybe in a call-out) somewhere in the FTK documentation.

[MSBrett]

We should update our docs to state that when using private endpoints and a power bi data gateway one needs to use the fully qualified domain name of the Azure Data Explorer cluster rather than the shortcut to ensure that name resolution for the private endpoint functions as expected.