Feature request: Provide forecast precipitation amount as a numeric value

[drewkeller]

I could be wrong, but it looks to me like the precipitation amount is only provided as part of the textual description of forecasts. I implemented handling of the twice daily forecast in weather-chart-card (for Home Assistant), but the precipitation amount cannot be shown because it looks for a value. Below are some screenshots of the object I captured from Firefox debugging. So I think the field would need to be implemented in pynws and then consumed by the NWS integration in Home Assistant.

An inch of rain seems important to know about :)

Here are various weather cards in Home Assistant. Some show the change of rain, but none can show the amount :( Well, unless I use something other than NWS ( ...Accuweather shown... ) :(

[MatthewFlamm]

The integration in HA uses a particular end point that does not have this data. There is a "raw data" endpoint, but it will require a bunch of specific handling to augment the existing data. This is doable if anyone is interested to take it on.

[drewkeller]

Thanks for the lightning response :)

So I guess, we (I?) need to implement an adapter from the "forecastGridData" style end point (which I see does have a "quantitativePrecipitation" value) to the data output of the Home Assistant integration (https://developers.home-assistant.io/docs/core/entity/weather/#weather-forecasts). It sounds like you're suggesting something along the lines of adding some extra fields to what already exists in the integration.

Is that endpoint accessible through pynws?

Can you elaborate on the "specific handling" bit?

[lymanepp]

The DetailedForecast class was added to pynws several years ago in order to provide relative humidity to the forecasts. By the time the HA core devs made it possible to use this in HA, the NWS had added it to the "simple" forecast used by pynws.

So, the DetailedForecast is currently not used by the NWS integration. One thing to note is that 500 (service error) responses are fairly common from the gridpoints API and retries will eventually succeed.

[drewkeller]

So far it seems pretty easy. I added the following to the example.py to retrieve the detailed forecast, and then got the output below.

How do I edit the nws integration in Home Assistant? It's in a docker container, right?

Updates to example.py (excerpted)

from pynws.const import Detail
from datetime import datetime

async def example():
    async with aiohttp.ClientSession() as session:
        nws = pynws.SimpleNWS(*PHILLY, USERID, session)
        await nws.set_station()

        print("------ DETAILED ------------")
        await nws.update_detailed_forecast()
        detailedForecast = nws.detailed_forecast
        now = datetime.now()

        print(f"precipitation : {detailedForecast.get_detail_for_time(Detail.QUANTITATIVE_PRECIPITATION, now)}")

        detailsByHour = detailedForecast.get_details_by_hour(now, 1)
        for detail in detailsByHour:
            print(detail['startTime'])
            print(f"temperature   : {detail['temperature']}")
            print(f"precipitation : {detail['quantitativePrecipitation']}")

output:

------ DETAILED ------------
precipitation: 0
2024-03-25T23:00:00
temperature  : 6.111111111111111
precipitation: 0

[MatthewFlamm]

@lymanepp was way more specific here. Thanks! One issue is that this data is inherently time point based. It is easier to think about how to add into the hourly forecast in HA, but a bit more difficult to add to the twice daily forecast. I'd suggest starting with hourly.

@drewkeller the changes need to be made in the HA NWS integration. See https://github.com/home-assistant/core/tree/dev/homeassistant/components/nws.

[lymanepp]

How do I edit the nws integration in Home Assistant? It's in a docker container, right?

Start here https://developers.home-assistant.io/docs/development_environment/

Edit: I was originally hoping to use the detailed forecast as the only source of data, but besides the issue that @MatthewFlamm mentioned, the detailed forecast doesn't include the textual description of the forecast and it's not easily recreated from the detailed forecast.

[drewkeller]

My 6+ year old computer seems to not like that development environment very much. The docker service keeps crashing and uses a lot of my memory. Anyway, I managed to work through that and get something working. It looks like the unit conversion is not quite right, so the numbers are way off but the proportions are roughly OK in comparison.

Unfortunately, when I went to create a branch to commit my changes, git or docker or something got confused and deleted my files instead of stashing them (I do this all the time without any issue so I know how git is supposed to work). So tomorrow I can start over. At least the changes aren't very complicated.

[drewkeller]

I got my changes written again (took 2 BSODs and several docker crashes). Perhaps it is ready for some reviewing. https://github.com/drewkeller/home-assistant-core/pull/1/files

I don't know if it is better to use a separate coordinator to update the detailed forecast or to do it along with hourly and the regular forecast. With the separate coordinator, I set up a local callback and it never seems to get hit, so maybe there is a step I missed for that.

It looks like NWS calculates precipitation in groups of 6 hours. So to get the precipitation for a 12 hour period, I grabbed a reading from each set of 6 and summed them.

I added some tests but when I run testing, it gives me errors about MagicMock not working with await. I don't know what can be done about that or if I'm running the test wrong.

I copied the nws folder to /config/custom_components/nws to run it on my production HA. I was getting errors in the log that it couldn't import ConfigFlowResult for some reason (from homeassistant.config_entries import ConfigFlow, ConfigFlowResult ). When I changed to asterisk, it worked (from homeassistant.config_entries import *).

This is from my home HA

[lymanepp]

I previously added this comment to a PR, but also adding here for context

I see what's happening. There isn't currently any correct implementation possible due to the nature of the one-and-only "quantitative" value. The root problem is that the NWS doesn't provide a precipitation quantity forecast by hour. Based on the example below, it looks like they currently forecast the quantity by 6-hour intervals aligned to 6am and 6pm. Unfortunately, they are aligned to UTC instead of the local time zone. This wouldn't align to the day/night interval or the daily interval for most time zones.

The DetailedForecast class implementation for "quantitative" values is wrong as it returns the total quantity for each hour. Within the current API, the only mitigation that comes to mind would be for the DetailedForecast class to divide "quantitative" values by the number of hours in the time interval. Not great, but better than the current results.

        "quantitativePrecipitation": {
            "uom": "wmoUnit:mm",
            "values": [
                {
                    "validTime": "2024-03-31T04:00:00+00:00/PT2H",
                    "value": 0.50800000000000001
                },
                {
                    "validTime": "2024-03-31T06:00:00+00:00/PT6H",
                    "value": 0
                },
                {
                    "validTime": "2024-03-31T12:00:00+00:00/PT6H",
                    "value": 0
                },
                {
                    "validTime": "2024-03-31T18:00:00+00:00/PT6H",
                    "value": 0
                },
                {
                    "validTime": "2024-04-01T00:00:00+00:00/PT6H",
                    "value": 0
                },
                {
                    "validTime": "2024-04-01T06:00:00+00:00/PT6H",
                    "value": 10.16
                },
                {
                    "validTime": "2024-04-01T12:00:00+00:00/PT6H",
                    "value": 4.0640000000000001
                },
                {
                    "validTime": "2024-04-01T18:00:00+00:00/PT6H",
                    "value": 0.254
                },
                {
                    "validTime": "2024-04-02T00:00:00+00:00/PT6H",
                    "value": 1.524
                },
                {
                    "validTime": "2024-04-02T06:00:00+00:00/PT6H",
                    "value": 10.414
                },
                {
                    "validTime": "2024-04-02T12:00:00+00:00/PT6H",
                    "value": 12.446
                },
                {
                    "validTime": "2024-04-02T18:00:00+00:00/PT6H",
                    "value": 5.5880000000000001
                },
                {
                    "validTime": "2024-04-03T00:00:00+00:00/PT6H",
                    "value": 4.8259999999999996
                },
                {
                    "validTime": "2024-04-03T06:00:00+00:00/PT6H",
                    "value": 3.048
                }
            ]
        },

[MatthewFlamm]

Sorry I haven't had time to devote to this, and it looks like a lot of work has gone into it!

I agree with @lymanepp that it is reasonable to assume that the "Rate of precipitation" per hour is even across the interval provided. Also note in the "validTime" key, it already provides the delta time for each measurement. For example, "validTime": "2024-03-31T04:00:00+00:00/PT2H" is valid from 4AM UTC to 6AM UTC. The next entry is "validTime": "2024-03-31T06:00:00+00:00/PT6H", which confirms this and is valid for another 6 hours. The entry

{
 "validTime": "2024-03-31T04:00:00+00:00/PT2H",
 "value": 0.50800000000000001
}

could reasonably be interpreted as having 0.25 mm/hr precipitation.

For day/night forecasts, the day/night "simple" forecast supplies the start and end times of the forecast, so you could use this to sum up the precipitation reports within that time period. You would have to handle the cases where the precipitation block contributes to both a day and night block.