raspberrypi
Raspberry Pi 3B+ and Wifi issues
Hello,
Looking at the threads on here, I might not be the only one having issues with the onboard wifi on my raspberry pi 3. But my symptoms a little different, so I figured I should start a new issue.
System
Raspbian 9.4 stretch, Kernel 4.14.71+ originally, now
Linux hame 4.19.0-v7+ #1157 SMP Tue Oct 23 18:10:04 BST 2018 armv7l GNU/Linux
/proc/cpuinfo says my chip is
Hardware : BCM2835
Revision : a020d3
Serial : 000000006398dxxx
and dmesg shows the following boot messages
[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 4.19.0-v7+ (dc4@dc4-XPS13-9333) (gcc version 4.9.3 (crosstool-NG crosstool-ng-1.22.0-88-g8460611)) #1157 SMP Tue Oct 23 18:10:04 BST 2018
[ 0.000000] CPU: ARMv7 Processor [410fd034] revision 4 (ARMv7), cr=10c5383d
[ 0.000000] CPU: div instructions available: patching division code
[ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[ 0.000000] OF: fdt: Machine model: Raspberry Pi 3 Model B Plus Rev 1.3
$ dmesg | grep brcm
[ 5.605576] brcmfmac: F1 signature read @0x18000000=0x15264345
[ 5.615125] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[ 5.615530] usbcore: registered new interface driver brcmfmac
[ 5.877269] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[ 5.895111] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Feb 27 2018 03:15:32 version 7.45.154 (r684107 CY) FWID 01-4fbe0b04
[ 7.295508] brcmfmac: power management disabled
(no more messages from brcm during/after boot)
and
# /opt/vc/bin/vcgencmd version
Oct 23 2018 18:14:53
Copyright (c) 2012 Broadcom
version 724d81b3f87eb71045e938e572eca140fb8828ab (clean) (release)
Scenario
I am trying to set up Home Assistant. This means I have an MQTT (+TLS) server on the rpi listening on port 8883 and I want random esp8266 to connect to it, over the rpi wifi interface, with TLS everywhere. But each one fails to connect. (as reported on the arduino serial monitor)
Running tcpdump on a nearby machine with a wifi card shows the esp8266 generate ARP requests and get no replies.
if I SSH into the rpi (over eth0) and from there I ping the esp8266's IP address, the esp8266 replies to the ping, and magically the esp's own TCP connection to the rpi now succeeds.
Sometimes, if I ping from the rpi to a nearby / different esp8266, the esp having issues will start working.
When it fails, and when it works I have nothing logged in dmesg. no mailbox errors, nothing.
Once any given esp8266 is working, it will continue to work for some time. If I unplug it and move to a second board (new IP address) the new board will immediately fail, until it is pinged. And after a short time, the original board will also fail if reattached (i.e. esp8266 working, unpower board, count to five, reconnect -> still works. unpower board, count 60s, repower -> probably not working).
When the esp8266 is removed, tcpdump shows a few ARP from the rpi to the missing IP address and (of course) no replies.
Steps taken to try to debug So far, I have
sudo BRANCH=next rpi-updatewhich took me from kernel 4.14.71 to 4.19.0; but did not change the brcm from the February 2018 f/w.- I have disabled bluetooth using
/boot/config.txtentrydtoverlay=pi3-disable-bt. - I have disabled ipv6 support entirely with
ipv6.disable=1in/boot/cmdline.txt. - I have put the wlan0 interface into promiscuous mode.
- I have "disabled" power saving using
iw/iwconfig(but of course, most 4.xx kernels disable power management already, anddmesgsays that too)
Steps next I am tempted to buy a usb-to-eth adapter, so I can create a working system to go forwards with my project. I could try a separate piece of hardware with the same SDCard to see if the behaviour moves or changes.
sundry The Wifi AP is a Ubiquiti UniFi UAP-LR 2.4GHz N300. Everything is getting DHCP-assigned addresses from a nearby Linux machine, which has wireline to the Ubiquiti. The rpi has a dhcp-static assignment (i.e. the server has the MAC address named and an IP assigned). The esp8266 are fully dynamic: just getting whatever the server gives them. To get a static MAC address, I have disabled MAC-randomisation on the rpi, using some nmcli command I forget the details of.
I have a new datapoint. Screwing up the debugging and accidentally wiping the UniFi access point SSID and settings, forcing me to reconfigure it from scratch has made the raspberry pi start behaving. Uptime on the AP is only 30min (and 11hr on the raspberry pi), and some of the other forum posts suggest 60-90min to fail, so I won't call it solved yet. Just that the behaviour changed by wiping the AP.
Story Time In the interests of gathering data, I started afresh this morning: new model3b, fresh SDcard, fresh Rasbian stretch, mosquitto server without home assistant, without docker, without encryption.
And it just worked. Against my workstation and one esp8266.
And then I tried to flash a second esp8266 to be unencrypted and broke everything. So I rebooted the UniFI UAP-LR. And updated the controller software and managed to do that wrong and wiped my SSID and settings. So I fixed all that.
And the Raspberry Pi with unencrypted mqtt still worked. But my esp8266 still didn't.
So I abandoned testpi and went back to my original raspberry pi, reflashed all the esp8266 to be encrypted mqtt and now they are working too.
Specifically, the raspberry pi has not been touched since I filed the report (11hr uptime). The Ubiquiti access point has been wiped and reinstated (0.5hr uptime), and each esp8266 has also been reflashed and reconfigured (zero uptime, I test by powering them on). The raspberry pi has no wireless sessions open - I have ssh'd in via wireline, so the wifi will be idle at the start of each test.
If I power on a new esp8266 on some IP address, it connects to the (wireless) rpi without issue. If I power that esp8266 off and power on a second one on some different IP address, it connects to the (wireless) rpi without issue.
For amusement (and so I can look at it later if it does break again), tcpdump on the rpi shows one esp8266 waking up:
21:20:50.520253 espmac > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.42.esp tell 192.168.42.esp, length 28
21:20:50.520470 espmac > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.42.dns tell 192.168.42.esp, length 28
21:20:52.465029 espmac > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.42.rpi tell 192.168.42.esp, length 28
21:20:52.465094 rpimac > espmac, ethertype ARP (0x0806), length 42: Reply 192.168.42.rpi is-at rpimac, length 28
21:20:52.468092 espmac > rpimac, ethertype IPv4 (0x0800), length 58: 192.168.42.esp.49153 > 192.168.42.rpi.8883: Flags [S], seq 6509, win 5840, options [mss 1460], length 0
21:20:52.468482 rpimac > espmac, ethertype IPv4 (0x0800), length 58: 192.168.42.rpi.8883 > 192.168.42.esp.49153: Flags [S.], seq 2612822460, ack 6510, win 29200, options [mss 1460], length 0
- "where is my IP?" (am I a clone?),
- "where is my DNS?" (reply not seen, because we are a switched network)
- "where is the raspberry pi?"
- raspberry pi replies "here is me". This packet is the one that was "missing" in the failing state
- "can we talk TCP?"
- "we can talk TCP" and we are away running.
new fault. uptime is now 21hr, so 10hr since it was spontaneously working.
the pi loadavg is 6.6 and /var/log/syslog is 7.9GB and growing
rebooting the PI in self-defence (my poor sdcard).
Notable moments from /var/log/syslog
Oct 27 09:20:05 hame kernel: [ 4.158237] brcmfmac: F1 signature read @0x18000000=0x15264345
Oct 27 09:20:05 hame kernel: [ 4.167206] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
Oct 27 09:20:05 hame kernel: [ 4.168107] usbcore: registered new interface driver brcmfmac
Oct 27 09:20:05 hame kernel: [ 4.401092] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
Oct 27 09:20:05 hame kernel: [ 4.418070] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Feb 27 2018 03:15:32 versi
on 7.45.154 (r684107 CY) FWID 01-4fbe0b04
...
Oct 27 09:20:06 hame kernel: [ 6.149227] brcmfmac: power management disabled
...
Oct 27 09:20:06 hame NetworkManager[339]: <info> [1540632006.9504] rfkill0: found WiFi radio killswitch (at /sys/devices/platform/soc/3f300000.mmc/mmc_host/mmc1/mmc1:0001/mmc1:0001:1/ieee80211/phy0/rfkill0) (driver brcmfmac)
...
Oct 28 06:02:14 hame dhclient[483]: DHCPREQUEST of <wireline_ip> on eth0 to <lan server> port 67
Oct 28 06:02:14 hame dhclient[483]: DHCPACK of <wireline_ip> from <lan server>
Oct 28 06:02:14 hame dhclient[483]: bound to <wireline_ip> -- renewal in 2817 seconds.
...
Oct 28 06:20:13 hame rngd[367]: stats: Entropy starvations: 0
Oct 28 06:20:13 hame rngd[367]: stats: Time spent starving for entropy: (min=0; avg=0.000; max=0)us
Oct 28 06:24:41 hame kernel: [75873.902928] brcmfmac: brcmf_sdio_htclk: HT Avail request error: -110
Oct 28 06:24:41 hame kernel: [75873.902992] brcmfmac: brcmf_sdio_htclk: HT Avail request error: -110
and that's it game over. 27873s of uptime is 21h04m, or less than an hour ago.
I have no other Pi 3B+ handy, only a Pi 3B. I suspect I will move to usb-ethernet for my project, but I am happy to help debug further if I can.
I only want to mention that AP mode seems to be broken in 4.19: https://www.raspberrypi.org/forums/viewtopic.php?f=29&t=224931&sid=38fd7ffea22221224fb3f52b828c3bed&start=25#p1396409
Probably related to this forum post https://www.raspberrypi.org/forums/viewtopic.php?t=32398
"Wifi ARP requests lost" - lists a possible workaround from
davidasailor26
For anyone that comes across this problem in the future: I still have no effective fix. For now, I'm working around it by adding a cron job to another host I have on the same subnet. Basically this cron job sends a broadcast ARP request to the Pi each minute. If no response is received, the job sends a deauthenticate packet to the Pi. The deauthenticate forces the Pi to reassociate with the AP, which takes about 2 seconds and fixes the issue. This does require that the other host have a wireless card capable of injecting raw packets. The script for the cron job is: Code: Select all
I will reply with any further breakthroughs I have.
if ! /sbin/arping -b -f -c 5 -I em1 [IP Addr of Pi] > /dev/null ; then /sbin/aireplay-ng -0 1 -a [AP BSSID] -c [MAC Addr of Pi] wlan0 > /dev/null ; /bin/date >> /var/log/fixPi ; fi
and to these bugs at least
https://github.com/raspberrypi/linux/issues/2677
https://github.com/raspberrypi/linux/issues/2522
@pmuntyanu that didn't fix it for me (i have tied APSD and GTK rekyeing etc) @SittingDuc how did you get dhclient to log to syslog - mine doesn't at all?
My issue https://github.com/raspberrypi/linux/issues/2522#issuecomment-556605204
I solved the issue by switching to static IP on the pi. The symptoms i had, i realized last night, were happening at DHCP lease expiration on my short lease of 34 minutes from a windows 2019 DHCP server (this would also explain why the pi stops responding to ARPs, it had no IP), Rather than troubleshoot why the pi was ignoring the server DHCP acks i switches to static IP address and all issues went away,
오래 사용하지 않은 라즈베리파이 3B+ 와이파이를 살리려고 온라인에 알려져 있는 모든 것을 시도해 보았습니다. 그러나 아무 소용이 없었어요. 그런데, 와이파이, 블루투스가 같은 안테나에서 기능을 하죠! 저는 블루투스로 라즈베리파이와 안드로이드폰 연결을 위해 잠깐 작업을 했습니다. 단순한 작업이죠. 그런데, 이 과정에서 update, upgrade를 했어요. 아주 오랜 시간이 걸렸어요. 그리고 다음 날 아침 일찍 파워를 넣어 보니 와이파이가 작동하는 것입니다! 와우!
Google translation:
I tried everything I could find online to try to revive my old Raspberry Pi 3B+'s Wi-Fi, but nothing worked. But, Wi-Fi and Bluetooth work on the same antenna! I briefly tried connecting my Android phone to my Raspberry Pi via Bluetooth. It was simple. However, during this process, I had to update and upgrade the device. It took forever. Then, early the next morning, I powered it on and Wi-Fi was working! Wow.
