TOD incorrect after midnight restart

[toogooda]

The problem

binary sensor TOD set to off when should be on after 2am restart

- platform: tod
   name: lateNightTime
   after: '21:30'
   before: sunrise

What version of Home Assistant Core has the issue?

2022.2.0

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Supervised

Integration causing the issue

TOD

Link to integration documentation on our website

here

Diagnostics information

No response

Example YAML snippet

- platform: tod
    name: lateNightTime
    after: '21:30'
    before: sunrise

Anything in the logs that might be useful for us?

No response

Additional information

No response

[probot-home-assistant[bot]]

tod documentation tod source _{^{(message by IssueLinks)}}

[bleem313]

I am having the exact same problem, also with a daily reset at 2 am. Once the system comes back online TOD changes the sensor to Off using the Night sensor example found in the documentation.

binary_sensor:
  - platform: tod
    name: Night
    after: sunset
    before: sunrise

As you can see in the sensor history, it turns off just after 2 am which is when HA restarts.

I am currently on Home Assistant OS 7.5 with core-2022.3.7 and supervisor-2022.03.5

[DarkWarden85]

Hi, I also have the same problem. The HA restart was caused by a power outage at around 1h42. Normally, the tod sensor should have stayed on till around 4h48.

[DmytroKorniienko]

All tod sensors going to off after core restart.

[ahaverty]

I had similar issues last night/today after some restarts beyond midnight.

binary_sensor:
  - platform: tod
    name: Night
    after: sunset
    before: sunrise

[ahaverty]

I hit this again last night. After a restart of HA at around midnight, my

    after: sunset
    before: sunrise

sensor returned to off incorrectly since.

It's fairly impactful, as I have a good few of my lights automations working off TOD conditions, and seemingly have no way to manually fix the values on these sensors, since they're defined in yaml.

I can see older, closed, but related issues at #50315 and #24577

[ahaverty]

Does anyone have a workaround or alternative integration that follows the sun? I'm hitting this regularly, as I'm doing a good bit of late night tweaking recently.

[DmytroKorniienko]

@ahaverty

Does anyone have a workaround or alternative integration that follows the sun? I'm hitting this regularly, as I'm doing a good bit of late night tweaking recently.

  - binary_sensor:
    # tod workaround
    - name: "Night"
      unique_id: tod_night
      state: >
        {% set cur_sunset = iif(state_attr("sun.sun", "next_setting")>state_attr("sun.sun", "next_rising")
        ,as_datetime(state_attr("sun.sun", "next_setting"))-timedelta(days=1)
        ,as_datetime(state_attr("sun.sun", "next_setting")))%}

        {% if 
        now()>cur_sunset
        and now()<as_datetime(state_attr("sun.sun", "next_rising"))
        -%}
          on
        {%- else -%}
          off
        {%- endif %}

[DarkWarden85]

I had an unexpected restart at 00:38 AM in the morning. All my tod sensors lost their state. Here's a picture of a tod sensor controlling my blinds. Notice how it turns of at 00:38 in the morning without any corresponding logbook entry. I have another tod sensor which does not depend on the sun but uses fixed hours which does not have that problem. Can anybody else confirm that the problem only occurs when a tod sensor depends on the sun?

[ahaverty]

@DmytroKorniienko thanks for sharing! I'm stuck though with some more complex TOD's that are important to me for outdoor light automations. I had a go at modifying your code for evening for example, but couldn't find a way to get the after_offset working properly.

binary_sensor:
  - platform: tod
    name: Night
    after: sunset
    before: sunrise

  - platform: tod
    name: Day
    after: sunrise
    before: sunset

  - platform: tod
    name: Evening
    after: sunset
    after_offset: "-00:30"
    before: "00:30"

  - platform: tod
    name: Outdoor Evening
    after: sunset
    before: "23:30"

  - platform: tod
    name: Early Morning
    after: sunrise
    before: sunrise
    before_offset: "02:00"

What I tried:

{% set cur_sunset = iif(state_attr("sun.sun", "next_setting")>state_attr("sun.sun", "next_rising")
        ,as_datetime(state_attr("sun.sun", "next_setting"))-timedelta(days=1)
        ,as_datetime(state_attr("sun.sun", "next_setting")))%}


        {% if now()>(cur_sunset - timedelta(minutes = 30)) and now()<today_at("00:30") -%} on {%- else -%} off {%- endif %}

Side note, is this a complex bug for HA to solve properly, or is it just low priority/forgotten? Maybe someone knows who to reach out to for a fix?

[DarkWarden85]

The problem is still present with version 2022.12.8. My home assistant instance suffers from regular unplanned restarts during the night, the cause of which I haven't found yet. This would not really be an issue if it weren't for all my time of day sensors having a wrong state after the restart. Is there really no fix for this?

[DmytroKorniienko]

@DmytroKorniienko thanks for sharing! I'm stuck though with some more complex TOD's that are important to me for outdoor light automations. I had a go at modifying your code for evening for example, but couldn't find a way to get the after_offset working properly.

after_offset requires more complex logic and I'm not implemented it for my automations, but can share one more example:

  - binary_sensor:
    # from 21:00 to 5 minutes before next_rising
    - name: "Light Control"
      unique_id: tod_light_control
      state: >
        {% set cur_sunset = iif(state_attr("sun.sun", "next_setting")>state_attr("sun.sun", "next_rising")
        ,as_datetime(state_attr("sun.sun", "next_setting"))-timedelta(days=1)
        ,as_datetime(state_attr("sun.sun", "next_setting")))%}
        {% set cur_start = strptime(as_timestamp(cur_sunset) | timestamp_custom('%Y-%m-%d 00:00:00%z'),'%Y-%m-%d %H:%M:%S%z') %}

        {% if 
        now()>cur_start+timedelta(hours=21, minutes=00)
        and now()<as_datetime(state_attr("sun.sun", "next_rising"))-timedelta(hours=0, minutes=5)
        -%}
          on
        {%- else -%}
          off
        {%- endif %}

[DarkWarden85]

Issue is still present on Home Assistant 2023.1.2. I had an unforeseen restart in the middle of the night and all my sunbased tod-sensors simply turned off. I included a pic of the tod-sensor handling my covers. All my covers stayed closed this morning instead of reacting to sunrise.

[ahaverty]

@frenck sorry for the direct @, but perhaps nobody at HA is assigned/tracking TOD issues. I'd appreciate hearing if you think this issue will be eventually triaged by anyone at HA? Thanks!

[frenck]

but perhaps nobody at HA is assigned/tracking TOD issues

@ahaverty There is no code owner for this integration, so that is correct.

I'd appreciate hearing if you think this issue will be eventually triaged by anyone at HA?

Home Assistant is an open source project and relies on contributions. Feel free to jump in an contribute a fix.

../Frenck

[ahaverty]

Thanks @frenck 👍

---

For anyone else trying to fix this: I took a stab at it, but unfortunately, I can't even reproduce the issue in unit tests.

As far as I understood, my test case here should fail after the HA restart, based on what I've experienced with my actual HA instance restarting just after midnight: https://github.com/ahaverty/core/commit/ecb5353025a9bc14cac41594796406801db9b362

I did see there was a recent cleanup of TOD in #79412, so I tested against an older version of HA (an Oct 2022 release) just in case that PR has already fixed this issue and we haven't noticed since, but I still couldn't get my test to fail on an older Oct release.

My theory on when the issue occurs (and what I was trying to fail in unit tests):

• When a sun-based TOD sensor starts on the day previous to a restart, but ends after the restart I've found that sun-based TOD sensors work fine for restarts during the day, for configurations like 'daytime', so my guess is there's something not being picked up during initialisation of the sensors after a restart. I'm just unfortunately not knowledgeable enough on HA to pinpoint/reproduce it here in code.

My theories on why I can't get the test to fail:

• I'm misunderstanding what hass.async_stop & hass.async_start actually does
  • Perhaps the async stop/start isn't a realistic replica of a real HA reboot.
• This restart issue is more complex to reproduce than I expected (I did also include offsets in my test, just in case that's required for this issue, but the test still passes with them)

EDIT: No, I have an obvious issue with a couple of things with my test case above. It looks like the suite is setup for sunsets happening before midnight for some reason. ~I'm now able to fail the test when I change the timezone and location to Ireland, against the Oct release. I'll look at testing against dev branch next, and see if I can add a fix also (i.e the hard part 😬)~

EDIT 2: yes, there's something odd going on with this test suite if I'm reading this correctly. The timezone is set to the US, but the lat/lng is in Poland. So the sunrise/sunsets aren't going to be realistic at all 🤔

EDIT 3: I'm still unable to reproduce this in unit tests unfortunately. As far as I can tell, I'm calling the correct restart methods. I've also fixed the suite's lat/lng to match the US Pacific timezone, so that sunrise/sunset happen on different days (as typically expected for most of the world). If anyone else wants to try fix, perhaps have a look at my attempt first: https://github.com/home-assistant/core/compare/dev...ahaverty:core:fix/TOD-incorrect-after-midnight-restart

EDIT 4: Nope, I've gone back to an Oct '22 release again with my latest test fixes, and can't reproduce the issue in unit tests. I've also tried:

• Testing with my exact TOD configuration, with multiple TOD sensors (below).
• Starting the hass async_stop before midnight, jumping to after midnight, and starting hass again

I'm stumped..

My TOD config for reference:

config = {
        "binary_sensor": [
            {
                "platform": "tod",
                "name": "Night",
                "after": "sunset",
                "before": "sunrise",
            },
            {
                "platform": "tod",
                "name": "Day",
                "after": "sunrise",
                "before": "sunset",
            },
            {
                "platform": "tod",
                "name": "Evening",
                "after": "sunset",
                "after_offset": "-00:30",
                "before": "00:30",
            },
            {
                "platform": "tod",
                "name": "Outdoor Evening",
                "after": "sunset",
                "before": "23:30",
            },
            {
                "platform": "tod",
                "name": "Day",
                "after": "sunrise",
                "before": "sunset",
            },
            {
                "platform": "tod",
                "name": "Early Morning",
                "after": "sunrise",
                "before": "sunrise",
                "before_offset": "02:00"
            }
        ]
    }

[DarkWarden85]

Hi @ahaverty,

Thank you for going through so much trouble to help get this fixed. It is odd that the problem doesn't occur during unit tests. I have currently three sun-based tod-sensors, which fail reliably after a midnight restart (last one was at 00:38). Maybe you could try with that config?

binary_sensor:
  - platform: tod
    name: Dachfenster Nachtmodus
    after: sunset
    before: sunrise
  - platform: tod
    name: Jalousien Nachtmodus
    after: sunset
    after_offset: "00:35:00"
    before: sunrise
    before_offset: "-00:35:00"
  - platform: tod
    name: Jalousien Südseite RDC Nachtmodus
    after: sunset
    before: sunrise
    before_offset: "-00:10:00"

Time Zone: (GMT+01:00) Luxembourg

Elevation: 420 meters

Country: Luxembourg

Coordinates: Somewhere in Luxembourg (I'm hesitant to post the coordinates of my home here in public)

[ahaverty]

Thanks @DarkWarden85, we practically have the same tod sensor there, which I've covered already in testing. I'm also having the restart issue myself in production consistently, so I don't think it's my config that's the problem for this test 🤔 Thanks though!

[ahaverty]

Although @DarkWarden85 just re-reading back through people's comments here. I'm assuming based on languages a bit, but the majority of us seem to be located around Europe. I have found it odd that this issue isn't more popular/fixed, maybe it is something unusual like:

• TOD component initialising with a default US timezone on boot, before HA had set the correct timezone (hence why it thinks the sensor is off) 🤷‍♂️
  • I'm not sure that's easy to prove in unit testing either, since the tests are overriding HA's timezone settings by default

Maybe if anyone here is based in the US, and also has this post-midnight issue, they could comment to let us know👍

[DarkWarden85]

Looking at your unit-tests everything seems to be in-line with the other unit-tests. Maybe there is something odd with the force stop and restart where it initializes itself with a different time when restarting. Maybe you should set the time again after the restart. This is just a wild guess and I'm not that great at programming in the first place

Edit: Stupid suggestion from me with setting the time again after a restart. This should have no effect because the init during restart is what causes the problem. But still... is it using the correct testing time after the restart?

[wafliron]

@ahaverty FWIW I experience this issue in Australia too - reliably reproducible with an after-midnight HA reboot. Another data point towards it potentially being something to do with non-US timezones - or perhaps non-US locales/languages?

Random thought, but possible it's got something to do with the environment HA is running in too? I'm using Home Assistant Operating System running as an ESXi virtual machine (think I downloaded an OVF from somewhere and used that, but so long ago I don't remember properly).

[DarkWarden85]

@wafliron to add to this, I'm running Home Assistant OS on a raspberry pi 4 while also experiencing the issue.

[ahaverty]

I may as well add to the list too in case anyone else beats me to fixing:

• Ireland
• Home assistant Blue (supervised, usually running close to the latest minor patches)

[ahaverty]

I tried the unit tests with my timezone and lat/lng in Ireland, but unfortunately couldn't get the restart to fail the test.

[ahaverty]

Hmm, I have a (possibly messy) workaround idea that might be worth spitballing with ye (while we wait for a saviour to fix the issue).

Since this seems to only affect sun-based TOD's that end one day later than the start (right?)

I wonder if I split them into two parts, for a night TOD for example:

- platform: tod
  name: night_a
  after: sunset
  before: "23:59:59"

- platform: tod
  name: night_b
  after: "00:00"
  before: sunrise

And then created a group for these two called "Night" via a group UI in helper settings.

Edit: Hmm, it'll probably cause havoc for that 1 second gap around midnight, for anyone with automations listening to state changes for light automations etc.

[bleem313]

I have more information listed in the 3rd comment on this but I will include my config again here. I'm US-based with a US timezone currently running on a raspberry pi 4 and do indeed have this issue, so I don't think being non-US is a cause. I had this issue on a pi 3 as well.

binary_sensor:
  - platform: tod
    name: Night
    after: sunset
    before: sunrise

[DarkWarden85]

I just wanted to confirm that the bug is still present in the newest version 2023.2.0, so no need for @ahaverty to downgrade your testing version.

Here's the result from a scheduled restart at 00h30 AM on the prior version 2023.1.7:

The same happens with a scheduled restart at 00h30 AM on version 2023.2.0 :

In both instances, the sensor was only turned on until 00h30 AM in the morning and it turned off after the restart

[zzachattack2]

but perhaps nobody at HA is assigned/tracking TOD issues

@ahaverty There is no code owner for this integration, so that is correct.

I'd appreciate hearing if you think this issue will be eventually triaged by anyone at HA?

Home Assistant is an open source project and relies on contributions. Feel free to jump in an contribute a fix.

../Frenck

As this integration has been “broken” for over a year, and apparently with no code owner to remedy, why does it even remain part of the core? Seems like the integration should either function, or progress to functioning, or just not exist at all…

[ahaverty]

Very untested, but I think I've come up with a decent workaround for the likes of a "night" sensor that relies on sunrise and sunset:

binary_sensor:
  - platform: tod
    name: left_night
    after: sunset
    before: "23:59"

  - platform: tod
    name: right_night
    after: "00:00"
    before: sunrise
    before_offset: "-00:02"

template:
    - binary_sensor:
      - name: "Night"
        unique_id: night
        delay_off:
          minutes: 2
        state: >
          {{ (is_state('binary_sensor.left_night', 'on') or is_state('binary_sensor.right_night', 'on')) }}

i.e: The left_night and right_night I don't intend to use anywhere other than in the template's state for binary_sensor.night above.

I've tested restarting just after midnight, and see binary_sensor.right_night is set to 'on' as expected. I've gone with 23:59 to be safe, but added a delay_off of 2 minutes so that the night sensor doesn't actually change to 'off' too early or trigger any automations just before midnight.

And the -2min before_offset is to counteract that delay_off. So it should function exactly like the standard night sensor, but without the nightime restart issues.

Edit July 2024: I've been using this without any issues since 2023 in case anyone is wondering if they should use it 👍

[DarkWarden85]

@zzachattack2 I agree that the non-functioning part of tod should be removed, but how to do that with no code maintainer for it? Also, there is the problem that tod might break completely in the future if there are any changes to core functionality it relies on.

@ahaverty Thanks for that workaround, it's the easiest to setup I have seen so far. Will try it out too.