azure-data-explorer-datasource icon indicating copy to clipboard operation
azure-data-explorer-datasource copied to clipboard

Support for empty or missing `database` key in dashboard JSON, fallback to default

Open kspeeckaert opened this issue 2 years ago • 1 comments

When using an ADX data source in a dashboard, the underlying dashboard JSON will look something like this:

{
  "panels": [
    {
      "datasource": {
        "type": "grafana-azure-data-explorer-datasource",
        "uid": "cfb97634-30e4-4c76-a9ef-ddbe79be15d7"
      },
      "targets": [
        {
          "OpenAI": false,
          "database": "demo_db",
          "datasource": {
            "type": "grafana-azure-data-explorer-datasource",
            "uid": "cfb97634-30e4-4c76-a9ef-ddbe79be15d7"
          },
...

Note that the database is referenced. This complicates provisioning dashboards, as we can't reference environment variables from JSON files (only from the provisioning config files), and we're stuck with a hardcoded database name.

To give some context: we spin up a separate Grafana instance per customer, and each instance is tied to the same storage for provisioning. Everything that's customer-specific is set through environment variables. Databases are also customer-specific, and this works great for provisioning config files (e.g. to create the data sources), but breaks in dashboards where ADX is used. In contrast, the MSSQL plugin has no issue.

As we already defined the database in the data source definition, there is no point in replicating it again here. If the database is the same as the default database (i.e. the one in the data source definition), the database entry should be blank or missing.

If that isn't possible, it should at least be possible to remove or blank the database entry in the dashboard JSON files for provisioning, and at runtime, a missing or blank database should be replaced by the default one. Alas, if we do this, this results in an error:

Azure HTTP "520 InternalServiceError": "name (Parameter 'Argument 'name' is empty: at .ctor in C:\\__w\\1\\s\\Src\\Engine\\DataNode\\Common\\Entities\\DatabaseIdentifier.cs: line 88')\r\nTimestamp=2023-09-01T06:52:26.9753150Z\r\nClientRequestId=KGC.raw;a4f21088ab4ff7d6;admin\r\nActivityId=REDACTED-GUID\r\nActivityType=GW.Http.CallContext\r\nServiceAlias=REDACTED\r\nMachineName=KEngine000000\r\nProcessName=Kusto.WinSvc.Svc\r\nProcessId=9212\r\nThreadId=8620\r\nActivityStack=(Activity stack: CRID=KGC.raw;a4f21088ab4ff7d6;admin ARID=REDACTED-GUID> GW.Http.CallContext/REDACTED-GUID)\r\nMonitoredActivityContext=(ActivityType=GW.Http.CallContext, Timestamp=2023-09-01T06:52:26.9701541Z, ParentActivityId=REDACTED-GUID, TimeSinceStarted=5.1904 [ms])"

Environment:

  • Grafana version: v10.1.0 (838218ba20)
  • Plugin version: 4.6.2 (also tested and confirmed with 4.5.0)
  • OS Grafana is installed on: Linux (Azure Container App)
  • User OS & Browser: Windows 10, Firefox 117.0

kspeeckaert avatar Sep 01 '23 07:09 kspeeckaert

Hi @kspeeckaert, thank you for raising this. We'll investigate how we can approach resolving this 😊

aangelisc avatar Sep 01 '23 11:09 aangelisc