ExpressLRS icon indicating copy to clipboard operation
ExpressLRS copied to clipboard

[PWM receiver Firmware] frequently lost control.

Open chofchop opened this issue 1 year ago • 14 comments

Current Behavior

PWM receiver lost control irregularly. Once the receiver lost control, it will hold the last state for about 10 seconds and then regain control. Since it does not enter failsafe, it seems that the problem is not the wireless connection being interrupted, but a malfunction of the receiver.

Steps to Reproduce

Radiomaster MT12 EdgeTX2.10.0(dfaaad730) BetaFPV nanoTXmodule ELRS3.3.2 (2d9891) Happymodel EPW6 ELRS3.3.2(2d9891)

  1. I was using the ELRS configurator, but after learning about #2156, I reset the PIO and decided to use only the WEB Flasher.
  2. I flashed ELRS3.3.1 to EPW6 using a WEB Flasher, but the problem was not resolved.
  3. The figure below shows the log data from the bench test. The second image is an enlarged view of 10 seconds when control is lost. nanoTX332EPW6332 lost10sec

Possible Solution (Not obligatory)

I flashed ELRS3.3.0 to EPW6 using a WEB flasher, and no problems have occurred in a 10-hour bench test. I don't think 10 hours is enough, so I'm continuing the test.

Details

  • TX hardware:BetaFPV nanoTXmodule ELRS3.3.2 (2d9891)
  • RX hardware:Happymodel EPW6 ELRS3.3.2(2d9891)
  • Handset model:Radiomaster MT12
  • OpenTX version (including nightly number):EdgeTX2.10.0(dfaaad730)
  • ExpressLRS version (TX & RX MUST MATCH):ELRS3.3.2(2d9891)
  • Packet Rate:F1000
  • Telemetry Ratio:1:32
  • user_defines:

chofchop avatar Jan 29 '24 07:01 chofchop

I have bench tested EPW6 (ELRS3.3.0) for over 44 hours, and so far I have not had any problems with lost control. When I tried EPW6 (ELRS3.3.2) several times along the way, I still lost control, so I think the problem is with the receiver firmware.

I think this is a serious issue, but why isn't the development team responding?

chofchop avatar Feb 01 '24 02:02 chofchop

This is not something that we have been able to reproduce. We have seen similar behaviour before where the firmware was built on windows, and also can occur from other builds. It has been seen before that erasing the flash can also fix this kind of issue. It sounds the issue is that the receiver is rebooting which can also be a power issue.

pkendall64 avatar Feb 01 '24 05:02 pkendall64

This is not something that we have been able to reproduce. We have seen similar behaviour before where the firmware was built on windows, and also can occur from other builds.

Is this referring to #2156? I don't think this applies because I am using pre-built firmware with a Web Flasher.

It sounds the issue is that the receiver is rebooting which can also be a power issue.

I am getting power for the receiver from the ESC's BEC. I'll try testing with a different power supply. However, I don't think that explains why there is no problem with 3.3.0 and the problem occurs with 3.3.1 and later.

chofchop avatar Feb 01 '24 06:02 chofchop

This is not something that we have been able to reproduce.

Under me, this problem occurs with MT12, X9Lite, and all three EPW6. It is highly unlikely that this only occurs in my environment.

chofchop avatar Feb 01 '24 06:02 chofchop

In the previous tests, the ESC (BL-SP4) was connected to ch1 of EPW6, the servo (JR DS3405) was connected to ch2, and the receiver power was supplied from the ESC. To confirm the power problem, I tested EPW6 with 1S battery powered instead of ESC (servo connected to ch2). As before, EPW6 (3.3.2) lost control. ss 2024-02-01 164151

I'm worried that TRSS is down between the second lost and the third lost. Neither the radio nor the model has moved it.

chofchop avatar Feb 01 '24 09:02 chofchop

I ran the 3.3.2 release on a Radiomaster ER5A for well over 4 hours and had no issues. Can you try erasing before flashing just to make sure that the flash is totally clean. I cannot test with the EPW6 as I have not received any from Happymodel to test with.

pkendall64 avatar Feb 02 '24 20:02 pkendall64

Thank you for testing. I am also continuing to test, but the problem does not occur with EPW6 (3.3.0), but the problem occurs when I change to 3.3.2. After this, I will flash the ER5A firmware (3.3.2) to EPW6 and test it.

I check "Erase flash first?" every time I flash. スクリーンショット 2024-02-03 091157

chofchop avatar Feb 03 '24 00:02 chofchop

Hmm, well the firmware is exactly the same on all devices. We now just have a file attached that describes which pins do which functions (since v3.0.0). So if it's failing on the EPW6 but not the ER5A then it suggests some kind of systemic fault with the EPW6 receiver hardware.

pkendall64 avatar Feb 03 '24 01:02 pkendall64

To confirm that it was a hardware issue specific to EPW6, I changed the receiver to EP1tcxo (I don't have any other PWM receivers other than EPW6). As a result, control lost will occur even with EP1tcxo.

EP1tcxo : ER5A/C 3.3.2 firmware > control lost occur EP1tcxo : EPW6 3.3.2 firmware > control lost occur

There was a new discovery here. In previous tests, I had set the PWM Output mode for ch1 to ch4 to 333Hz, but when I changed it to the default 50Hz, control lost no longer occurred.

EP1tcxo : EPW6 3.3.2 firmware(50Hz) > no issues

I have only tested it for about 2 hours so far, so I will continue testing including other Output modes.

P.S. MT12(EdgeTX2.10.0), BetaFPV nanoTX module(Packet Rate:F1000, Telemetry Ratio:1:32).

chofchop avatar Feb 03 '24 09:02 chofchop

There was a new discovery here. In previous tests, I had set the PWM Output mode for ch1 to ch4 to 333Hz, but when I changed it to the default 50Hz, control lost no longer occurred.

EP1tcxo : EPW6 3.3.2 firmware(50Hz) > no issues

I have only tested it for about 2 hours so far, so I will continue testing including other Output modes.

Any update on further testing? I'm eager to read the findings on this as 333hz output is one of the key features of interest to me.

kmf123kmf avatar Feb 06 '24 17:02 kmf123kmf

Testing continues, but only a portion of it has been completed as there are a huge number of combinations. 333Hz is also essential for me, so I'm thinking of narrowing down the test target to 333Hz. In any case, it is a very time-consuming task, so I would like everyone to help.

chofchop avatar Feb 07 '24 00:02 chofchop

mine also would loose connection with 3.3.1. i flashed to firmware 3.3.0 and it solved the problem. i am using the mt12 and er5c-i receiver. i initially thought it was my servo pulling to much current from the bec but after i flashed to 3.3.0 it has not done it since.

jhue1973 avatar Feb 08 '24 12:02 jhue1973

I had set the PWM Output mode for ch1 to ch4 to 333Hz, but when I changed it to the default 50Hz, control lost no longer occurred.

Based on this information, I am now able to reproduce the problem. I have captured a trace when this is happening and it's a "watchdog timer reset", which means the code is stuck somewhere and not returning to the main loop to feed the watchdog, so the watchdog resets the CPU.

I'll continue working on this and see if I can get some more debug information out of the chip.

pkendall64 avatar Feb 09 '24 03:02 pkendall64

Thank you for finding the cause.

This information may not be necessary anymore, but in my testing, the higher the servo frequency, the more frequently the problem occurred. At 50Hz, the occurrence rate is extremely low, but it still occurred after a long test.

chofchop avatar Feb 09 '24 04:02 chofchop

Not trying to be annoying, but is there anything new to report on this?

kmf123kmf avatar Feb 23 '24 22:02 kmf123kmf

Yes, I have been testing a "fix" and it's looking promising. I've run a PWM receiver for 48hours now without any problems. I will have a PR ready soon, it's a workaround, not a proper fix as I'm still not 100% sure why it's happening.

pkendall64 avatar Feb 23 '24 23:02 pkendall64