[Bug]: nrf52 boards are not entering deep sleep when battery is critically low (make system_off mode work on nrf52)

[geeksville]

Category

Hardware Compatibility

Hardware

All nrf52 boards

Firmware Version

master (private build)

Description

I just happened upon this code in Power.cpp:

#ifdef ARCH_NRF52
                // We can't trigger deep sleep on NRF52, it's freezing the board
                LOG_DEBUG("Low voltage detected, but not triggering deep sleep\n");
#else
                LOG_INFO("Low voltage detected, triggering deep sleep\n");
                powerFSM.trigger(EVENT_LOW_BATTERY);
#endif

Freezing boards is bad, but IMO it is super important to enter deep sleep anytime we are able to detect that the battery is critically low. Because to not do this will kill lipos.

I'm happy to look into this as I continue my adventure of improving nrf52 power consumtion. But opening an issue in case any of ya'll have comments on this theory?

Relevant log output

No response

[todd-herbert]

I built one node with a small solar enclosure from RAK. (Unify Enclosure Solar IP65 100x75x38)

• RAK4631
• RAK19007
• 1000mAh LiPo cell (to fit claimed 55x34x6mm space below mounting plate)

I was concerned that this battery could run flat. I made a quick-and-dirty modification to the firmware to add some level of software protection.

In essence:

• Early in boot, check voltage
  • If below 3.7V, enter pseudo deep-sleep: delay() 30 minutes, then reboot
  • If above 3.7V, continue boot
• If battery consistently reports below 3.45V, reboot the node. This triggers the low voltage check.

This current draw in this pseudo deep-sleep state is very good; a few µA from memory? However, below a certain voltage, I believe the buck converter on the RAK19007 baseboard switches to a linear mode. Between a certain voltage range (3.5V down to 3.3V?) the current is higher; almost 1mA at worst.

Even in this 1mA region, the current is low enough to protect the battery from many days of poor weather. Despite the small battery, I haven't actually had this happen yet since deployment, although it worked very well in testing.

The pseudo deep-sleep state is the same one used with NRF52 tracker and sensor roles, however the current draw in those cases (after normal use, instead of early in boot) is higher (~1mA, even with good voltage). I suspect some piece of hardware isn't shutting down perfectly, but I couldn't identify exactly what.

---

This doesn't entirely fix the situation, but I found it to help in my specific use case. I'm not sure whether something like this would be suitable for general use though.

[geeksville]

see related comments here: https://github.com/meshtastic/firmware/pull/2040 (btw: I think some of the comments in this issue misunderstood how nrf52840 treats 'shutdown mode'.

[geeksville]

Progress notes (mostly for future searchers/reference on state of power as of today):

• I've found why waking system-off wasn't working - because it could only work for 'wake due to USB connect', I'm working on expanding that to also use wake based on ADC comparision of the battery voltage. Will include in the fix for this bug. For boards that can't do either of those things I'll have system-off mode failback to just super long sleeps with the existing old behavior.

Test configs

RAK4630 with OLED display and GPS modules:

Testing battery at 3.7V

Power consumption in system off (with the fixes I'll be submitting) is excellent when the battery voltage is at 3.7V. It is drawing only 5uA which is almost the optimal possible per datasheet.

Testing with battery at 3.5V

Fine - same as above.

Testing with battery at 3.4V

Beginning to get bad but not absolutely horrible: 50uA average. Therefore if we want to shut down because of low battery (on the rak4630) we should do so at 3.5V (to stay in the realm of low current draw - the rak LDO seems to have issues at low V)

Testing with battery at 3.2V (min safe lipo voltage)

awful - the power draw is is 13mA (enormous) while in system-off-mode. Also saving our state to the filesystem when at this voltage seems to cause #4184 100% of the time. Also it is worth noting that the rak4630 datasheet specifies a minimum battery voltage of 3.3V. Which makes sense because this is their vcc regulator circuit:

Which is unfortunate because lipos will slowly fall in voltage below 3.3V. So to robustly support the rak4630 we need to make sure to shut the board down before the battery reaches that level.

Wio Tracker 1100 board (red pcb)

Testing with USB power or battery power

system-off-mode has current draw that is way to high with the current codebase. About 2.5mA. Will eventually investigate and instrume with the PPK2 power profiler.

[geeksville]

Notes on RAK4630 issues (only):

ooh - the rak folks populated a regulator for their 3v3 bus but the unfortunate thing is they use that regulator for all of the sources on the board. they could have instead passed the battery v straight to the cpu VDDH input (bypassing that regulator). This would have been better because a) it would the brown out detector work correctly 😉 and b) it would have allowed the cpu to keep working properly all the way down to 2.7V (well below 'min' lipo voltage). nordic designed that input to be for lipo inputs and works fine up to 4.2v (or higher - don't remember) right now I kinda bet the brownout detector is just seeing 3.3v in until suddenly it starts bouncing allover the place once we get below that. https://docs.rakwireless.com/assets/images/wisduo/rak4630-module/datasheet/schematic-2.png

(also it would have solved the problem that someone else mentioned (@markbirss ?) that when the battery is lower than about 3.4V that regulator burns an enormous amount of power)

[markbirss]

@geeksville it was someone else but i cannot find mention, on discord group ?

[geeksville]

added this note from a discord discussion:

(better to shut down while we still have battery than to start browning out when we can't control the shutdown process. And all bets get wonky in those super low v cases, possibly solar panel charges battery up some and the old write to meshtastic.txt (which I just removed in #4395) was highly likely to be writing when the board is coming-up then dying repeatedly when battery is below v3.3)

[geeksville]

note to self: turn EVENT_LOW_BATTERY back on! - WIP branch is pr-fixe1000

[Joseph-DiGiovanni]

I'm confused. This thread seems to imply we should deep sleep sooner than normal for nrf52 platforms to minimize current draw, but nrf52 is specifically prevented from entering deep sleep with no explanation beyond the comment that its freezing the board" from 2022. Could you please clarify the status of the issue?

[Joseph-DiGiovanni]

Also are the stability concerns addressed in newer hardware/firmware? I had my RAK19003 run down to 2.74v (according to the Meshtastic app) and it seemed incredibly stable all the way up until somewhere past 2.61v when it eventually stopped responding.

[Joseph-DiGiovanni]

they could have instead passed the battery v straight to the cpu VDDH input (bypassing that regulator)

I'm honestly a little over my head here, so forgive my ignorance if I'm mistaken, but aren't the current schematics for the RAK4630 and the RAK4631 module doing exactly this?

Schematics

[viric]

I was going after this problem. I had a RAK node which was NOT solar, where the battery drained, and then I plugged in USB to charge it, it would not boot. It required a physical press of reset button. So I went on testing.

Using a lab power supply, I can reproduce the hang if power goes down to 1.68V, no matter how high the voltage comes later back. The brownout reset (happening under 1.7V specs say) apparently doesn't have in NRF any way of recovering, unless the power goes to 0V first. Even the reset button doesn't work.

I cite from https://devzone.nordicsemi.com/f/nordic-q-a/37058/brownout-watchdog-reset-on-voltage-drop :

• So if you ever do get a brown-out event, how do you recover? How is the power cycling going to happen without user intervention?
• It's designed to require user intervention, like the change of a coin cell battery.

As far as I understood, a reliable solution would require external electronics that can ensure the power goes off to get then a normal power-on-reset.

If we cannot add those external electronics (what can be done with a RAK?) then the only hope is to go to deep sleep BEFORE the brownout reset occurs (on some low voltage) to avoid draining the battery, hoping that power will come back sooner than brownout 1.7V condition.