Upgrade Grafana

[sebestenyb]

Hi there,

Thanks for the great work with this package. I wonder if it would be possible to upgrade it with a more recent Grafana version. The docker file uses version v6.5.1 which is a fairly old one, the current major version is 9.

Cheers, Sebi

[jasonacox]

Hi Sebi, thanks for opening this issue. The stack will work with the latest version of Grafana, but the rendering of some of the plugins were a bit off. It would require some debugging to get the configuration (likely dashboard.json updates) to look properly. I'll add that to my list to work on.

If you want to experiment or even contribute some help, here is where you would update the version in the powerwall.yml file:

    grafana:
        image: grafana/grafana:6.5.1-ubuntu
        container_name: grafana

If you edit that, you can stop and delete the 6.5.1 container and re-run the compose:

docker-compose -f powerwall.yml up -d

[techlover1]

I have started work on upgrading the dashboards. Hopefully I'll have it done in a day or two

[techlover1]

Just created #73

[jasonacox]

Thanks, @techlover1 ! This is great!

I'm traveling this week but will try to test the Powerwall+ graphs and merge when I can. If you don't mind, paste some screenshots of the main dashboard so we can confirm we see the same thing.

[jasonacox]

(from PR comments) First observations -

1. The 9.x version menus are different so it will require an update to the README, setup.sh and upgrade.sh instructions for users.

2. It seems the Power Meter panels have lost their color and font size - I'll need to figure out how to bring this back:

Old:

New:

[mcbirse]

Great progress guys at updating to 9.x - it is almost there I think!

I have tested this to help provide some feedback, and also note a couple of small issues (for me at least).

1. The Current State panel appears to be cut off, and needs minor adjustment to size/position?

2. The left side scale on the Wind, Pressure and Precipitation appears to be incorrect, some data is not showing on the graph Old: New:

3. As noted already, font sizes and colours are different on a number of elements

4. dashboard-weather411.json was not included in the updates. I use this occasionally - it appears to work okay still in Grafana 9, however not sure if there are some changes required regardless?

5. Definitely agree README etc. will need some updates too. The interface is quite different and might take a bit of getting used to.

[techlover1]

Thanks for the feedback! I have made a first pass at updating the documentation, and will take a look at the other items this evening

[techlover1]

I just pushed updated power meters and fixed the visibility scaling on the Wind, Pressure and Precipitation graph (for dashboard.json only so far). The current state panel still needs to be worked on, I may end up recreating it with grafana's built in table panel

[jasonacox]

It's getting very close @techlover1 - thanks for doing this!

[techlover1]

Of course! Getting this done prevents me from having to run a separate instance of grafana :)

After the bugs are worked out, how do you feel about moving from the old style graph to the new time series graph? https://grafana.com/blog/2021/02/10/how-the-new-time-series-panel-brings-major-performance-improvements-and-new-visualization-features-to-grafana-7.4/

Old

New

[jasonacox]

Yes! I love it! Having the 3rd axis (units for % and C) is something that has ben bugging me with the current version.

I keep looking for an alternative to the boom table. It has a bug that keeps causing "No data" (for all version of Grafana) and despite having a PR in place to fix it, it seems abandoned which is likely why it doesn't work well with Grafana 9. I looked at the CSS and see that the issue is caused by a computed max-height control that is likely not compatible with Grafana 9. Does anyone have a suggestion for an alternative? I like the panel / data but it is problematic.

[jasonacox]

I see the latest update fixes the boom table clipping! @mcbirse how does this look for your setup?

[mcbirse]

Mine is looking good too! Awesome work @techlover1 👍

Just one minor issue, perhaps? (depending on opinion)

I notice some of the units for kWh now show Wh when the value is small (old Grafana panel always showed kWh). This is showing up in a few places in the example screenshots, e.g. in my "From Grid" and Jason's "To Grid". Also the Current State Panel now shows W when it would always show as kW previously.

Just a thought - for consistency would it be possible to show kW and kWh even for small values everywhere? I had to have a double-take at my "From Grid" as I knew I hadn't used 820kWh today!!

[techlover1]

Glad we're getting closer! I'll plan on bringing the other dashboards up to date in the next couple days and transition over to the new time series graphs

Just a thought - for consistency would it be possible to show kW and kWh even for small values everywhere? I had to have a double-take at my "From Grid" as I knew I hadn't used 820kWh today!!

The way to do that would be using custom kWh units instead of grafana's built in, which is an easy change. Curious on @jasonacox 's thoughts

[youzer-name]

I originally installed Powerwall Dashboard alongside some existing docker containers, so I've been using a newer version of Grafana all along. I had updated many of the panels to newer components, including switching to Time Series for the energy usage chart. I'm currently on Grafana v9.0.7.

One thing I ran into was that you can't align the left and right Y axes in Time Series graph. In @techlover1 post above, you can see that when the left axis has a max of 12 KW and a min of -6 KW, the left zero and right zero are not aligned. I didn't like that this moved the zero point of the sun angle graph away from the zero point of the left axis. This can be solved by using hard min/max on the axes, but that just introduces new issues.

I solved this by adding hidden values to the graph that are the inverse of some of the displayed values. So I have a query for home usage, and one for 'hidden home usage' which is equal to "home usage query x (-1)" Even though the values are hidden, the axis adjusts for them. So if my home usage is 15 KW, my "hidden home usage" will be -15 KW. This means the left axis will always have the same max and min values, keeping the zero at the mid-point of the graph.

I posted about this a while back (not realizing it probably didn't apply to people using the default version of Grafana). Now that the default is moving to Time Series, you may want to consider applying the same solution to the main code.

Here is my earlier post: https://github.com/jasonacox/Powerwall-Dashboard/discussions/16

[jasonacox]

@techlover1 Awesome work here! Thank you so much. 🙏

• I agree with @mcbirse on consistency of using kWh to make it easier to read at a glance (comparing the values). Can we update that?
• @youzer-name has a great point on the zero-point. It really shows up on the sun graph. Can we switch to the new time series but add the hidden "home usage" to force zero point?

Looking forward to merging all of this when it is ready. 👍

[youzer-name]

@jasonacox Just to clarify, I had to add inverse values for "home", "solar", and "from_grid". There needs to be an inverse for whatever value might be the largest absolute value on the lieft-axis for any given view. For my house those three fields take care of it, but I don't know if someone else's situation might be different.

[jasonacox]

Ah, thanks @youzer-name - I didn't realize that. I think we should test more before implementing the new time series graph.

@techlover1 , I suggest we first release the Grafana 9 updates (just fix the kWh bit) and then work on the zero-point issue. I want to get your great efforts out there.

[jasonacox]

A minor adjustment: Under the Powerwall+ section, there are a few errors:

• String Voltage: The "A string" is missing due to a typo on Voltage in the query.
• String Current: Has the wrong unit, it should be ampere (A).
• String Power: Has the wrong unit, it should be wattage (W).

I am going to proceed with the PR merge and make these updates. I've archived the Grafana v6 for those that want to continue to use that version (in a v6.5.1 directory).

[jasonacox]

The Grafana v9 upgrade work from @techlover1 has been merged into main. 🚀

I corrected the few items above moved VERSION 2.6.0 for the upgrade.sh which I tested on a older Grafana v6 install. It worked correctly. However, users will need to note that the dashboard.json install requires selection of Data sources. I added that to the upgrade.sh script instructions.

Now that this update is live, please test and report any glitches or bugs. 🐞

Thanks for the great work @techlover1 !

Open TODOs:

• Fix Power Meters to use only kWh units
• Switch to new Time Series graph with zero-point correction

[jgleigh]

Setting Timezone back to America/Los_Angeles... sed: dashboard-animation.json: in-place editing only works for regular files sed: dashboard-animation.json: in-place editing only works for regular files Waiting for InfluxDB to start... up!

Add downsample continuous queries to InfluxDB... 2022/09/21 22:36:06 error: continuous query already exists 2022/09/21 22:36:06 error: continuous query already exists

Anything to worry about here?

[jgleigh]

Color schemes for the Power Meters should be changed to Single instead of From Thresholds. Currently, the numbers turn Red if you start changing the time range.

[jasonacox]

Thanks @jgleigh - I'll add some logic to fix the first error about "regular files". Nothing to worry about there or the InfluxDB "already exists" errors (just means you already have all the queries installed).

I'm not following on the color schemes. Can you paste a screen shot? I'm not able to replicate. Did you import the new dashboard.json file?

Example:

Thanks for these updates!

[jgleigh]

Default Time Range:

Last 7 Days:

The colors are defined as Thresholds which is set at 80. So once the value goes over 80 they turn red.

[jasonacox]

Thanks @jgleigh !! Great find here! I have removed the 80 threshold and have pushed the latest change to dashboard.json.