Option to exclude some panel types from Simple layout report

[ebabeshko]

It would be nice to have an option to exclude some panels from report by panel type when Simple layout is chosen.

For example, I have a dashboard, report for which in Grid layout looks like this:

It contains panels with data (Table, Time series, Pie chart) and some auxiliary panels (Variable panel, Text panel etc.).

When I generate report using Simple layout, it includes many pages with auxiliary panels which are not really useful:

etc.

I would like to have an option to include in the report only 'useful' panel types like Table, Time series etc. and exclude 'auxiliary' ones:

etc.

Of course possible workaround is to create a separate dashboard without auxiliary panels just for report, but still it would be nice to have a possibility to configure list of panel types that should be excluded from report, preventing in this manner a need in additional dashboards.

[mahendrapaipuri]

Cheers @ebabeshko for opening this issue.

Yes, I thought about this and a "straight forward" solution for this is to have a query parameter that takes panel IDs that will be rendered in the report. You can configure the report URL link with the panel IDs that you would like to render in the report. Do you think it will work for your use case?

[ebabeshko]

Cheers @ebabeshko for opening this issue.

Yes, I thought about this and a "straight forward" solution for this is to have a query parameter that takes panel IDs that will be rendered in the report. You can configure the report URL link with the panel IDs that you would like to render in the report. Do you think it will work for your use case?

Yes, I think your approach will be even a more universal way to do it.

Also I noticed if dashboard has non-ASCII symbols in title, PDF-file gets a strange name:

For example, I get u0414u0430u0448u0431u043eu0430u0440u0434.pdf instead of Дашбоард.pdf.

Looks like Unicode encoding of non-ASCII symbols is used (\u0414\u0430\u0448\u0431\u043e\u0430\u0440\u0434), and then slashes are removed because they can't be used in a filename.

Not a big issue anyway.

[mahendrapaipuri]

@ebabeshko I added the support for the query parameters includePanelID and excludePanelID in #33. Check updated docs. This PR also adds support for repeated panels and/or rows. You can download the plugin artifiacts from CI.

The filename encoding has been fixed as well!

[ebabeshko]

@mahendrapaipuri, thank you very much, this is a valuable contribution.

[mahendrapaipuri]

@ebabeshko Cheers!! It would be super helpful if you can test it out and give some feedback!

[ebabeshko]

Filename encoding is fine.

But the report is blank now if I do not specify includePanelID and excludePanelID:

The log is the following:

logger=plugin.mahendrapaipuri-dashboardreporter-app t=2024-05-16T18:32:29.3386878+03:00 level=warn msg="failed to get plugin app secret" endpoint=callResource err="PluginAppClientSecret not set in config" pluginID=mahendrapaipuri-dashboardreporter-app logger=plugin.mahendrapaipuri-dashboardreporter-app t=2024-05-16T18:32:29.3386878+03:00 level=info msg="filtering panels" endpoint=callResource excluded=null included="[1 2 3 4 5 6]" user=admin dash_uid=bdjpetyaaw7i8e pluginID=mahendrapaipuri-dashboardreporter-app logger=plugin.mahendrapaipuri-dashboardreporter-app t=2024-05-16T18:32:29.8595544+03:00 level=info msg="report generated" endpoint=callResource pluginID=mahendrapaipuri-dashboardreporter-app user=admin dash_uid=bdjpetyaaw7i8e

May be if includePanelID and excludePanelID are not specified, the behaviour should be as before, i.e. including everything?

Anyway, even if I specify includePanelID, the report is blank...

[mahendrapaipuri]

@ebabeshko Cheers for the rapid testing. Appreciate it.

I guess it must be due to another major feature of this PR: supporting repeated panels and rows. If you do not specify includePanelID and excludePanelID query parameters, the plugin is meant to have the same behaviour as before.

Those are the only logs the plugin emitted? You can get result for any dashboard? Is it possible to share the dashboard JSON model that you are using for your test?

Cheers!!

[mahendrapaipuri]

Also, the repo ships test dashboards that we use for CI testing. Could you add these dashboards to your Grafana instance and attempt to generate reports?

[ebabeshko]

Result is the same for any dashboard, including test ones:

Log:

logger=plugin.mahendrapaipuri-dashboardreporter-app t=2024-05-16T18:52:30.2575939+03:00 level=warn msg="failed to get plugin app secret" endpoint=callResource err="PluginAppClientSecret not set in config" pluginID=mahendrapaipuri-dashboardreporter-app logger=plugin.mahendrapaipuri-dashboardreporter-app t=2024-05-16T18:52:30.2576472+03:00 level=info msg="filtering panels" dash_uid=e472bbf0-140c-4852-a74b-1a4c32202659 excluded=null included=null endpoint=callResource pluginID=mahendrapaipuri-dashboardreporter-app user=admin logger=plugin.mahendrapaipuri-dashboardreporter-app t=2024-05-16T18:52:30.9081999+03:00 level=info msg="report generated" dash_uid=e472bbf0-140c-4852-a74b-1a4c32202659 endpoint=callResource pluginID=mahendrapaipuri-dashboardreporter-app user=admin

[ebabeshko]

I am accessing Grafana via https if that matters.

Tried the same dashboard with Released version 1.3.0, result is the following:

[mahendrapaipuri]

Thanks again for quick testing.

I am accessing Grafana via https if that matters.

Yeah, this could be it. You configured TLS on Grafana server or you use a reverse proxy to access Grafana? What is the appUrl in your provisioned config?

[ebabeshko]

TLS is on Grafana server.

      appUrl: https://localhost:3000
      skipTlsCheck: true

[mahendrapaipuri]

@ebabeshko Cheers!! That was some valuable feedback. Yes, there was a bug with TLS handling in the new implementation and I could reproduce your results with TLS enabled. I have already a fix. I will let you know as soon as I push the fix. I would like to add a test with TLS enabled in CI to avoid these situations.

[mahendrapaipuri]

@ebabeshko I have made quick a big refactor and added more CI tests. Here are the new plugin artifacts. If you could test, it would be super helpful!

Cheers!!

[ebabeshko]

@mahendrapaipuri , thank you for your work.

I've tried, but didn't succeed this time. CPU resources are utilized at 100%, but report isn't being generated:

> logger=plugin.mahendrapaipuri-dashboardreporter-app t=2024-05-20T13:13:36.2489534+03:00 level=info msg="provisioned config" config="Grafana App URL: https://localhost:3000; Skip TLS Check: true; Grafana Data Path: C:\\grafana\\data; Orientation: portrait; Layout: simple; Dashboard Mode: default; Encoded Logo: ; Max Renderer Workers: 2; Persist Data: false; Included Panel IDs: all; Excluded Panel IDs: none" endpoint=callResource pluginID=mahendrapaipuri-dashboardreporter-app
> logger=plugin.mahendrapaipuri-dashboardreporter-app t=2024-05-20T13:13:36.2529528+03:00 level=warn msg="failed to get plugin app secret" endpoint=callResource err="PluginAppClientSecret not set in config" pluginID=mahendrapaipuri-dashboardreporter-app
> logger=plugin.mahendrapaipuri-dashboardreporter-app t=2024-05-20T13:13:36.2529528+03:00 level=info msg="updated config" endpoint=callResource pluginID=mahendrapaipuri-dashboardreporter-app user=admin config="Grafana App URL: https://localhost:3000; Skip TLS Check: true; Grafana Data Path: C:\\grafana\\data; Orientation: portrait; Layout: simple; Dashboard Mode: default; Encoded Logo: ; Max Renderer Workers: 2; Persist Data: false; Included Panel IDs: all; Excluded Panel IDs: none" dash_uid=f9771316-6014-4402-943b-eaea3437f5f1
> logger=context userId=1 orgId=1 uname=admin t=2024-05-20T13:13:37.2540104+03:00 level=info msg="Request Completed" method=GET path=/api/live/ws status=-1 remote_addr=[::1] time_ms=0 duration=998.8µs size=0 referer= handler=/api/live/ws status_source=server
> logger=rendering renderer=plugin t=2024-05-20T13:13:38.5510844+03:00 level=info msg=Rendering path="d-solo/f9771316-6014/_?from=now-1h&height=500&panelId=27&theme=light&to=now&width=1000"
> logger=rendering renderer=plugin t=2024-05-20T13:13:38.5610859+03:00 level=info msg=Rendering path="d-solo/f9771316-6014/_?from=now-1h&height=500&panelId=22&theme=light&to=now&width=1000"
> logger=plugin.grafana-image-renderer t=2024-05-20T13:13:38.878103+03:00 level=error msg="Browser request failed" url="https://localhost:3000/d-solo/f9771316-6014/_?from=now-1h&height=500&panelId=22&theme=light&to=now&width=1000&render=1" method=GET failure=net::ERR_CERT_AUTHORITY_INVALID
> logger=plugin.grafana-image-renderer t=2024-05-20T13:13:38.8791029+03:00 level=error msg="Error while trying to prepare page for screenshot" url="https://localhost:3000/d-solo/f9771316-6014/_?from=now-1h&height=500&panelId=22&theme=light&to=now&width=1000&render=1" err="Error: net::ERR_CERT_AUTHORITY_INVALID at https://localhost:3000/d-solo/f9771316-6014/_?from=now-1h&height=500&panelId=22&theme=light&to=now&width=1000&render=1\n    at navigate (C:\\snapshot\\src\\node_modules\\puppeteer-core\\lib\\cjs\\puppeteer\\common\\Frame.js:114:23)\n    at async Deferred.race (C:\\snapshot\\src\\node_modules\\puppeteer-core\\lib\\cjs\\puppeteer\\util\\Deferred.js:82:20)\n    at async Frame.goto (C:\\snapshot\\src\\node_modules\\puppeteer-core\\lib\\cjs\\puppeteer\\common\\Frame.js:80:21)\n    at async CDPPage.goto (C:\\snapshot\\src\\node_modules\\puppeteer-core\\lib\\cjs\\puppeteer\\common\\Page.js:651:16)\n    at async Browser.takeScreenshot (C:\\snapshot\\src\\build\\browser\\browser.js:262:13)\n    at async Browser.render (C:\\snapshot\\src\\build\\browser\\browser.js:236:20)\n    at async PluginGRPCServer.render (C:\\snapshot\\src\\build\\plugin\\v2\\grpc_plugin.js:116:13)"

[mahendrapaipuri]

@ebabeshko Cheers for the test. It is grafana-image-renderer that is taking up all the resources. By default, it launches a chrome instance to render each panel in the dashboard. To improve the performance and resource consumption of grafana-image-renderer, set these env vars.

Regarding the error, it is not coming from the reporter plugin app per se. It is from grafana-image-renderer as well about unknown TLS certificate. Could you verify if you still have the env vars setup for grafana-image-renderer to ignore TLS errors?

Also, at times these errors can be transient. So please couple of times to see if the error persists!

[ebabeshko]

Also, at times these errors can be transient. So please couple of times to see if the error persists!

Yes, looks like this is issue. I tried several times without success. But now it works as expected, and I can't reproduce that error, although no changes were made in settings.

I've tried includePanelID and excludePanelID options, and they seem to be working fine. Filename encoding is also fine.

Great job, thanks!

[mahendrapaipuri]

Awesome!! Thanks a lot for helping me in testing the plugin. I will try to cut a new release soon!

[mahendrapaipuri]

@ebabeshko Made a new release v1.4.0 with new features! Cheers again for testing it out!