Heartrate measurements in background

[patricgruber]

This implements heart rate measurements when the screen is turned off. The ticket I found for this is: https://github.com/InfiniTimeOrg/InfiniTime/issues/183

When starting the heart rate measurements through the HeartRate screen and turning off the screen, the heart rate task doesn't stop but keeps running in the background until the heart rate measurement is stopped through the screen again. The task wait delay is set to ~10 seconds (10k ticks) so the task doesn't run all the time and drains the battery too much.

Already tested it on my PineTime and works great. Right now it was more a proof of concept, therefore the interval between background measurements is hard-coded to ~5 minutes. But I'd be happy to implement a settings screen to configure the interval or other features that are wanted.

[github-actions[bot]]

Build checks have not completed. Possible reasons for this are:

1. The checks need to be approved by a maintainer
2. The branch has conflicts
3. The firmware build has failed

[eliedrian]

Was about to implement the same thing!

Another idea that I was thinking of was to measure heart rate after a number of steps taken. Perhaps taking measurements after a certain number of steps in a time window.

[lman0]

Another idea that I was thinking of was to measure heart rate after a number of steps taken.

This idea should be a setting and not a default, Because there is people with leg disability that could use the pine time with the heart rate. And this people wouldn't be have the heart rate function. But for fitness related event, without leg disability it could be .

[patricgruber]

Another idea that I was thinking of was to measure heart rate after a number of steps taken.

This idea should be a setting and not a default, Because there is people with leg disability that could use the pine time with the heart rate. And this people wouldn't be have the heart rate function. But for fitness related event, without leg disability it could be .

It could maybe be an additional trigger for measuring, but not the only one.

[btdogan]

I use hr monitor for my sleep. this shouldn't prevent that.

[patricgruber]

I use hr monitor for my sleep. this shouldn't prevent that.

The code just runs measurements every 5 minutes when the heart rate task is started (through the heart rate screen) and the watch is locked/the screen is off.

All the other functionality is exactly as before. The new changes don't interfere with monitoring the heart rate while staying in the heart rate screen.

[patricgruber]

I noticed during testing that notifications interrupt the background measurement delay and make it reset, so I removed the reset for the GoToSleep message. Now the behavior is slightly different as the background measurement will every 5 minutes from the last background measurement or as soon as the watch is going to sleep. Instead of measuring every 5 minutes starting from the time it goes to sleep.

Notifications and "just checking the time quickly" won't reset the delay now and the measurements will run more consistently over time.

[LinuxinaBit]

This idea should be a setting and not a default, Because there is people with leg disability that could use the pine time with the heart rate. And this people wouldn't be have the heart rate function. But for fitness related event, without leg disability it could be .

And the heart rate sensor likes to use a lot of battery power, so for those feeling battery conscious it would need to be a setting somewhere. Maybe in the HR app?

[patricgruber]

Maybe in the HR app?

The background measurements are only running, if the user activates the heart rate measurement in the HR app. If the user doesn't activate the HR measurements in the HR app or if they are disabled again, no background measurement will be taken.

So there is no new default that HR measurements are taken in the background all the time, just when activating them through the HR app.

The behavior when activating the measurement and leaving the HR app is that when the screen is on the HR is measured all the time. If the screen is off, HR is not measured. If the HR measurement is not activated in the app no measurement is taken ever.

My code does not change the case when measurements are activated and the screen is on (stays at "measure as often as possible"). Also when the measurement is not activated in the HR app nothing changes (stays at "never measure"). Only for the case when the measurement is activated in the app AND the screen is off, then the new mode "measure every 5 minutes" is activated.

So the the user still always has the option to just don't activate HR measurements at all, but if they are activated, then additionally to measuring when the screen is on there are also occasional measurements when the screen is off.

I'm not sure if it makes sense to have another setting that switches between the current mode and the new "measure additionally in the background" mode. I think that would be a bit confusing if you have to activate two things in separate places to activate the background measurements. But if that is needed and wanted I'll of course add it.

[LinuxinaBit]

That sounds pretty reasonable.

One other concern is just that it takes so long to take a HR measurement, and the first reading is often wildly inaccurate especially during movement. Maybe wait for 3 measurements and average the last two, as well as checking for excess movement from the accelerometer before measurement.

Another idea is to pause readings when absolutely zero accelerometer movement is detected for the past few readings, i.e. when the watch has not been on one's wrist for a while (of course the accelerometer would still be checked every 5-ish minutes even when off one's wrist and would also resume periodic readings when the watch is woken up).

These would both help the battery life and improve accuracy immensely.

[Itai-Nelken]

One other concern is just that it takes so long to take a HR measurement, and the first reading is often wildly inaccurate especially during movement.

After #1486 is merged, measurements will be almost instantaneous and much more accurate.

[LinuxinaBit]

Alright, though I still think excessive and no movement detection should be added to preserve some amount of battery life...

[pankk]

I've been testing https://github.com/InfiniTimeOrg/InfiniTime/pull/1486 for a couple of days now and while the updates are almost instantaneous the first reading usually takes up to 10 seconds for me. I can't say much about the accuracy, but it's similar to the miband 3. I found that for best results, wearing the watch higher up the forearm (about 1/3 of the way) and having it face the inside of the arm works. I was not able to get a measurement restart when I wore it like that (i.e. the reading did not reset to 000, like it does when moving around while having it on the wrist). PSA: It also might be worth checking if the protective film has been removed from the sensor "window". (I assume mentioning the PPG PR here is enough and that I don't have to add this feedback to that PR as well.)

[khimaros]

i'm also testing the new PPG algorithm and sometimes it takes upward of 15-20 seconds to get a fix on the heart rate. i've checked out @patricgruber PR and built it. i'll take it for a spin.

also, @pankk thank you for the tip with the film, i'd totally forgotten to remove mine! that might explain the 15-20s delay :)

[khimaros]

i'm testing this PR on my device and i'm happy to say it is working quite well. (removing the sensor film reduced the fix time considerably).

to start, i tested heart rate characteristic notifications in bluetoothctl:

$ bluetoothctl
[bluetoothctl]# devices
Device C9:2D:E5:XX:XX:XX InfiniTime
[bluetoothctl]# pair C9:2D:E5:XX:XX:XX
[bluetoothctl]# connect C9:2D:E5:XX:XX:XX
[InfiniTime]# menu gatt
[InfiniTime]# select-attribute 00002a37-0000-1000-8000-00805f9b34fb
[InfiniTime:/service005b/char005c]# notify on

i received notifications almost exactly every five minutes with the screen off:

[CHG] Attribute /org/bluez/hci0/dev_C9_2D_E5_XX_XX_XX/service005b/char005c Value:
  00 53                                            .N              
  00 53                                            .N    

after that, i installed nrfConnect (unfortunately, a proprietary app and there doesn't seem to be an open source equivalent) and logged all of the notifications overnight.

inspecting the logs, the heart rate accuracy and success rate of notifications seemed very reliable and consistent. awesome!

i estimate that total battery drain overnight was less than 10%, but i did not look at the exact percentage before sleep. i will take more accurate battery measurement tonight.

[lman0]

@khimaros could you share the firmware ? I would like to test it too

[khimaros]

@lman0 -- i'm open to sending this privately if you provide an email address. unfortunately, the build artifact contains private information (usernames and unabridged filesystem paths, possibly more). alternatively, i've written up detailed reproducible build instructions in the comments on this PR: https://github.com/InfiniTimeOrg/InfiniTime/pull/1761

[mark9064]

So I've been testing this PR along with the related gadgetbridge PR. Super cool to have the HR measurements showing up. A few thoughts on it

• Previous HR values are changed to no longer be cleared upon screen wake with this PR, or upon the PPG algorithm losing fix on the HR
  • I think this is confusing as it is difficult to tell when the HR value on screen is fresh and when it is the last reading.
  • It's also difficult to tell when it goes from a live reading to holding the last value when the PPG algorithm can't find a value
• The running time is unlimited when doing a background measurement
  • In the extreme case, if there is no sensor contact, the PPG will run continuously

Suggestions

• I think the UI needs to convey that a HR measurement is stale, or it should not show it at all
  • Not sure how this applies to all watchfaces
• Limit runtime to 15-20s
  • Can be done by checking the running duration using the tick count inside HandleSensorData

Regarding the measurement interval: I was thinking about what it works best as. There are a few options I see regarding when to trigger a measurement

• Measurement begins every 5 minutes, ignoring whether the device has been woken or not (close to what is currently implemented)
  • In this case it needs to be wary of a possible BackgroundMeasuring -> (screen on) Measuring (total time < limit) -> (screen off) BackgroundWaiting
    • A measurement is then skipped
  • Also possible to have Measuring -> background timer expires -> (screen off) BackgroundWaiting -> BackgroundMeasuring
    • i.e the PPG stops and then immediately gets started after, discarding all state and forcing a refix
• Measurement runs at least every 5 minutes, taking into account measurements from being awake
  • In other words, every time a HR fix is achieved, the measurement timer is set to expire in 5 minutes
  • The other condition is that the expiry timer must be reset when a background measurement is attempted but fails due to the time limit
  • This way solves the second problem above by design and also seems more sensible to me intuitively
    • I think solving the first problem might require another state?

Not sure if I made any sense so interested to hear any thoughts :)

[khimaros]

@mark9064 i think these are great suggestions. i'll just mention that, even though the data is stale, it is sometimes useful to be able to quickly turn the watch on to see what the most recent HRM measurement was. maybe it can be displayed in another color to indicate staleness?

[khimaros]

FYI, slightly off topic here, but possibly useful for testers of this PR. phyphox also works as a tool for graphing and exporting heart rate data from InfiniTime on android: https://codeberg.org/Freeyourgadget/Gadgetbridge/issues/2383#issuecomment-915776

[patricgruber]

Thanks for all the suggestions.

Regarding the timeout reset for background measurements: Before I implemented the checks so that whenever a measurement started, the timer reset. Which was a problem since just checking the time reset the timer. I then changed it to the current implementation that only resets the timer when specifically a background measurement is done. This way having the screen up doesn't effect the background measurement at all and so if the watch goes to sleep, the code pretends as if nothing happened and will immediately measure the heart rate if the screen was on for longer than the interval. A nice change would be to change it so whenever a measurement is done, either in the foreground or background, the interval is reset.

Regarding the reset to 0 when the screen wakes up when ambient light is detected: I personally rather have a slightly older/stale value then no value at all. If I want a fresh value I can just wait, but at least I can just check the screen and see in which region I am, if I had some more or less constant state for at least 10 minutes or so. But if the heart rate is reset to 0, then I don't have that. I have the background measurements that are either sent to companion app or go no where, but I won't ever see them on the watch. I think a good compromise is having a different color for stale values so I know that the values are old, but I also see the last measurement.

Regarding the ambient light sensor and running forever: I think implementing a timer to stop the sensor if there is ambient light for more than 10-20 seconds and then just try again for the next "scheduled" measurement seems reasonable.

[mark9064]

Sounds sensible overall :+1: One thing to note re running time limits: if it is dark but the PPG has no contact it will still run forever if it just checks ambient light. If the PPG has been running for say 30s with no success the chance it's going to succeed is probably pretty low so I think giving up makes more sense? I guess it depends on how power hungry running the PPG is relatively. I'm planning on making a continuous measurement patch for testing so I should have some ideas on that soonish

[patricgruber]

@mark9064 I would just implement it to always stop trying to measure after 30s, no matter if it is because of ambient light or other reasons.

Also as side note: my first test was to just let it run continuously in the background and I think it drained around 50% battery or more over night. Also the watch got warm. I think letting it run for that long without breaks is probably not the best idea. But maybe with the updated PPG code made it possible. I have only tested with the old code.

[patricgruber]

I implemented a timeout of 30s for the background measurement. If there is no data within these 30s, then the measurement is stopped and retried after 10 mins.

[mark9064]

So I tried implementing functionality (https://github.com/mark9064/InfiniTime/commit/225be818b14637092e09e2ce3587e25c71a91bce and https://github.com/mark9064/InfiniTime/commit/97d894edc4a2787ada38f6c3057d49aa4326e934) that allows the user to choose between no background measurements, periodic background measurements and continuous measurement. It seems to work as expected but as you noted battery life with continuous measuring is poor (~24h). I don't know where the majority of the power is consumed but it makes sense that it would be the PPG LED as it's on about 50% of the time and has a high drive current. The data is interesting though, sleep zones are clearly visible, but the UI for switching between modes sucks. Not sure if this is a feature users would want or if we should just keep it simple

[khimaros]

@mark9064 i'm using your testing branch now on my Pinetime device and it is working quite well. thank you for putting this together and sharing it! i especially like the new raise to wake/lower to sleep functionality. it's very reliable and accurate and i haven't seen any unintended wakes yet.

battery life is, expectedly, worse with the continuous mode on. apart from the battery used to power the LED/hardware, i suspect continuous mode is also preventing the main CPU from going to sleep, since it is always busy with the measurement task? this is just a guess, i don't know any of the inside details of InfiniTime/FreeRTOS power management.

i agree with you the UI for choosing the HRM mode could be a bit clearer, but as a power user it was intuitive enough. maybe it's just a matter of choosing more descriptive names for the toggle? alternative descriptions: "Always", "Periodic", "Foreground". another option is to make the "background delay" configurable, but then we'd likely need two controls; one for the delay and one to toggle background mode on/off.

overall, however, i think this is a great feature and would love to see it land in a release version! it's really incredible to be able to raise my wrist, look at the screen, and see instantaneous heart rate information! i suspect i'm not the only person who would think so.

[mark9064]

Finally got time to do some proper power testing using the patchset in my last message With low brightness, raise wake and lower to sleep (latest PR versions), sleep mode ~8h/day, wearing watch ~97% of the time: 8 days battery from a full charge. So it would probably be feasible to enable unconditionally if we wanted to, though I think having controls is still nicer

[khimaros]

@mark9064 is this using the testing branch at https://github.com/mark9064/InfiniTime/tree/testing ? which measurement mode were you using? periodic? continuous?

[mark9064]

@mark9064 is this using the testing branch at https://github.com/mark9064/InfiniTime/tree/testing ? which measurement mode were you using? periodic? continuous?

Using periodic mode, though I've rebased the branch on 1.13 locally

[mark9064]

@patricgruber What's your opinion on the proposal that the user can choose whether they want background measurements or not? I think we should choose what we want to do here so we can get closer to having this merged.

Personally I think it's useful as it allows users to have better idle battery life if they don't care for periodic measurements, and it also allows users to enable continuous monitoring which is useful for exercise or detailed sleep tracking where constant updates without having the screen on are desirable