nexmon icon indicating copy to clipboard operation
nexmon copied to clipboard

bcm43455c0 crash when injection frames,

Open cyberbalsa opened this issue 6 years ago • 28 comments

While using BetterCap to do frame injection for KARMA Attacks and otherwise general packet injection, The bcm43455c0 stops responding to all commands and just dies..

Please provide me with some details on how to start debugging this.

Thanks

Kernel: Linux Balsachan 4.19.75-v7+ #1270 SMP Tue Sep 24 18:45:11 BST 2019 armv7l GNU/Linux

root@Balsachan:/opt/nexmon-master# dmesg | grep brcmfmac
[    6.097852] brcmfmac: loading out-of-tree module taints kernel.
[    6.131816] brcmfmac: F1 signature read @0x18000000=0x15264345
[    6.144964] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[    6.145744] usbcore: registered new interface driver brcmfmac
[    6.509848] brcmfmac: brcmf_sdio_bus_preinit: before brcmf_sdio_debugfs_create
[    6.513661] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[    6.527989] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Oct 15 2019 20:30:25 version 7.45.189 (nexmon.org: -4) FWID 01-e1db26e2
[    6.581486] brcmfmac: brcmf_bus_started: before brcmf_debugfs_add_entry
[    9.181466] brcmfmac: brcmf_vif_add_validate: Attempt to add a MONITOR interface...
[    9.181485] brcmfmac: brcmf_mon_add_vif: brcmf_mon_add_vif called
[    9.181488] brcmfmac: brcmf_mon_add_vif: Adding vif "mon0"
[ 5387.511962] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4097, -110
[ 5390.551934] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4098, -110
[ 5393.601927] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110
[ 5396.631949] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4101, -110
[ 5399.671984] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4102, -110
[ 5402.711975] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[ 5405.752020] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4104, -110
[ 5408.791991] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4105, -110
[ 5411.831985] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4106, -110
[ 5414.872013] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53284, -110
[ 5417.911966] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53288, -110
[ 5420.951985] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53296, -110
[ 5423.511967] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110
[ 5426.312006] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4100, -110
[ 5429.111938] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[ 5431.911970] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4105, -110
[ 5434.711974] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53308, -110
[ 5437.511963] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53348, -110
[ 5440.311927] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53356, -110
[ 5443.111922] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53360, -110
[ 5445.911981] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53368, -110
[ 5448.711973] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53372, -110
[ 5451.511925] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53380, -110
[ 5454.311928] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110
[ 5457.111929] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4100, -110
[ 5459.911983] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[ 5462.711974] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4105, -110
[ 5465.511924] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53308, -110
[ 5468.311926] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53348, -110
[ 5471.111925] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53356, -110
[ 5473.912031] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53360, -110
[ 5476.471932] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4102, -110
[ 5479.271932] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[ 5482.081987] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4104, -110
[ 5484.871994] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4105, -110
[ 5487.671927] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53292, -110
[ 5490.471930] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53300, -110
[ 5493.271977] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53304, -110
[ 5496.071923] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53348, -110
[ 5498.872004] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53364, -110
[ 5501.671976] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53372, -110
[ 5504.471928] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53376, -110
[ 5507.031989] brcmfmac: brcmf_proto_bcdc_query_dcmd: brcmf_proto_bcdc_msg failed w/status -110
[ 5507.032002] brcmfmac: brcmf_cfg80211_get_channel: chanspec failed (-110)
[ 5509.591930] brcmfmac: brcmf_proto_bcdc_query_dcmd: brcmf_proto_bcdc_msg failed w/status -110
[ 5509.591939] brcmfmac: brcmf_cfg80211_get_tx_power: error (-110)
[ 5512.151989] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4102, -110
[ 5514.952027] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[ 5517.751924] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4104, -110
[ 5520.552020] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4105, -110
[ 5523.351981] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53292, -110
[ 5526.151976] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53300, -110
[ 5528.951979] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53304, -110
[ 5531.751931] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53348, -110
[ 5534.551972] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53364, -110
[ 5537.351926] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53372, -110
[ 5540.151929] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53376, -110
[ 5542.951963] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4102, -110
[ 5545.511930] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110
[ 5548.311979] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4101, -110
[ 5551.112097] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[ 5553.912023] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4105, -110
[ 5556.711979] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53292, -110
[ 5559.511930] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53300, -110
[ 5562.311920] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53312, -110
[ 5565.111950] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53348, -110
[ 5567.911974] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53356, -110
[ 5570.711932] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53364, -110
[ 5573.511929] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53372, -110
[ 5576.311972] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53384, -110
[ 5579.121924] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53388, -110
[ 5581.911940] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110
[ 5584.711931] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4101, -110
[ 5587.511929] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[ 5590.072026] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4098, -110
[ 5593.111980] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110
[ 5596.152050] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4100, -110
[ 5599.191867] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[ 5601.831814] brcmfmac: _brcmf_set_multicast_list: Setting mcast_list failed, -110
[ 5604.391691] brcmfmac: _brcmf_set_multicast_list: Setting allmulti failed, -110
[ 5612.071578] brcmfmac: _brcmf_set_multicast_list: Setting BRCMF_C_SET_PROMISC failed, -110
[ 5632.791347] brcmfmac: brcmf_cfg80211_del_ap_iface: interface_remove failed -110

cyberbalsa avatar Oct 15 '19 23:10 cyberbalsa

I have a full tracelog attached to this comment that should help understand the commands being sent syslog.gz

cyberbalsa avatar Oct 16 '19 00:10 cyberbalsa

Does it also happen with 7.45.154?

DrSchottky avatar Oct 18 '19 10:10 DrSchottky

Yes, This was tested on a stock Re4son Kernel, and A Stock Raspian kernel compiled for 154. same issues. Some of our users of our project suggest it might be heat related. Also We found that a Full power cycle is required or the driver never fully comes back (So a simple warm reboot does not fix the issue)

cyberbalsa avatar Oct 18 '19 10:10 cyberbalsa

Never had heat problems, but try to repeat the tests with a heatsink and a proper PSU. What do you mean with warm reboot? A system reboot shoud power-cycle the wifi chipset during mmc driver rebinding (and you can even do it w/o rebooting)

DrSchottky avatar Oct 18 '19 12:10 DrSchottky

So, when I mean heat issues, I'm talking the wifi IC under the RF Can overheating, you can't put a heatsink on that. So, We have found if you just issue a "reboot" from the shell, when the rPi 3B+ boots back up the wifi chip will crash within 2-3 minutes, If you pull power, and then plug it back in (A cold boot) the Wifi chip is fine for about 10-30 until it randomly locks up and stops responding. We have found this issue on dozens and dozens of Pis, even the rPi 4 has this issue.

https://github.com/evilsocket/pwnagotchi/issues/267 is the current issue that we are tracking over at our project.

cyberbalsa avatar Oct 18 '19 13:10 cyberbalsa

Are you sure that's a heat issue? I mean: have you tried cooling it down somehow to confirm it dosen't happen if kept cold?

Have you got a minmal setup (a scapy/whatever script) that I can use to repreoduce the issue by myself?

In the syslog you posted I found a slightly different error : brcmfmac: brcmf_sdio_hostmail: mailbox indicates firmware halted. Cold you try replace sdio.c with this, recompile ad look at the log to see, when it'll crash again, if it prints something useful?

Thank you

DrSchottky avatar Oct 18 '19 20:10 DrSchottky

Can't seem to load that Hastebin, can you Gist it.

Thanks

cyberbalsa avatar Oct 18 '19 21:10 cyberbalsa

https://gist.github.com/DrSchottky/e012fb50e11d4e843d316df3763a5b34

DrSchottky avatar Oct 18 '19 21:10 DrSchottky

Hi. I think I have a similar problem. I'm using the last raspbian release 2019-09-26.

I followed the steps in the readme and all works well, everything works fine until the mon0 interface is up.

I also build the drivers for the RPi3B (not the plus) using the bcm43430a1/7_45_41_46 using the folder brcmfmac_4.19.y-nexmon of this driver and works perfect.

I put the commands and the dmesg output: original firmware:

brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Feb 27 2018 03:15:32 version 7.45.154 (r684107 CY) FWID 01-4fbe0b04

rmmod brcmfmac:

brcmfmac: brcmf_proto_bcdc_query_dcmd: brcmf_proto_bcdc_msg failed w/status -5
brcmfmac: brcmf_cfg80211_get_tx_power: error (-5)
brcmfmac: brcmf_fil_cmd_data: bus is down. we have nothing to do.
brcmfmac: brcmf_link_down: WLC_DISASSOC failed (-5)
brcmfmac: brcmf_fil_cmd_data: bus is down. we have nothing to do.
brcmfmac: brcmf_fil_cmd_data: bus is down. we have nothing to do.
brcmfmac: brcmf_fil_cmd_data: bus is down. we have nothing to do.
brcmfmac: brcmf_cfg80211_get_channel: chanspec failed (-5)
usbcore: deregistering interface driver brcmfmac

insmod brcmfmac_4.19.y-nexmon/brcmfmac.ko:

brcmfmac: loading out-of-tree module taints kernel.
brcmfmac: F1 signature read @0x18000000=0x15264345
brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
usbcore: registered new interface driver brcmfmac
brcmfmac: brcmf_sdio_bus_preinit: before brcmf_sdio_debugfs_create
brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Oct 22 2019 10:15:04 version 7.45.154 (nexmon.org: 2.2.2-269-g4921d-1) FWID 01-4fbe0b04
brcmfmac: brcmf_bus_started: before brcmf_debugfs_add_entry

iw phy `iw dev wlan0 info | gawk '/wiphy/ {printf "phy" $2}'` interface add mon0 type monitor:

brcmfmac: brcmf_vif_add_validate: Attempt to add a MONITOR interface...
brcmfmac: brcmf_mon_add_vif: brcmf_mon_add_vif called
brcmfmac: brcmf_mon_add_vif: Adding vif "mon0"

ifconfig mon0 up:

brcmfmac: brcmf_sdio_hostmail: mailbox indicates firmware halted

TheNextLVL avatar Oct 22 '19 08:10 TheNextLVL

Which hardware are you using? If you're on 3B+/4B just run make and make install in bcm43455c0/7_45_189/

DrSchottky avatar Oct 22 '19 16:10 DrSchottky

Hi @DrSchottky I try both versions with the same result. I follow the README guide.

The driver works, but when I up the mon0 interface the driver halts. I try disabling the interface (ifconfig wlan0 down) and then up the mon0 with the same result.

I don't know if this affects, but I don't disable dhcpd or wpa_supplicant service.

TheNextLVL avatar Oct 23 '19 16:10 TheNextLVL

try to uninstall wpa_supplicant

DrSchottky avatar Oct 23 '19 22:10 DrSchottky

Hi @DrSchottky , I made a fresh raspbian installation and uninstalling wpa_supplicant before install the firmware made work the interface in mode monitor 😄 . Is there any way to connect to AP and monitor with the mon0 interface like the RPi3B?

TheNextLVL avatar Oct 28 '19 11:10 TheNextLVL

@TheNextLVL never tried but it doesn't sound like it. https://github.com/seemoo-lab/nexmon/issues/317

DrSchottky avatar Oct 28 '19 12:10 DrSchottky

In README bcm43455c0 is marked as not supporting injection. Is this wrong? @DrSchottky @JRWR What the real difference between bcm4345 and bcm43455c0? Is there any way to find this C0 on a chip marking?

image

zhovner avatar Feb 10 '20 10:02 zhovner

Hi Pavel,

in the past injection was not properly working from the host, however, from the firmware you can normally always inject. Unfortunately, I am not sure about the current status. bcm43455 and bcm43455c0 were found in different devices with different firmware versions. If the ROMs of both devices are the same, we could merge those chip versions into one version. Feel free to dump and compare them and let me know.

Matthias

On 10. Feb 2020, at 11:25, Pavel Zhovner [email protected] wrote:

In README bcm43455c0 is marked as not supporting injection. Is this wrong? @DrSchottky https://github.com/DrSchottky @JRWR https://github.com/JRWR What the real difference between bcm4345 and bcm43455c0? Is there any way to find this C0 on a chip marking?

https://user-images.githubusercontent.com/774290/74141701-7b483300-4c08-11ea-99fb-fb8abf11b0da.png — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/seemoo-lab/nexmon/issues/335?email_source=notifications&email_token=ACZ773UXFBIDP5O4OW2GN3DRCETSRA5CNFSM4JBEKXDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELH7LKY#issuecomment-584054187, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZ773SYWGK5UFMBQY2UMETRCETSRANCNFSM4JBEKXDA.

matthiasseemoo avatar Feb 18 '20 20:02 matthiasseemoo

I'm seeing this with 7_45_206 on a Raspberry Pi 4 (with pwnagotchi).

Although I'm not sure what to make of the error about Direct firmware load. Did the firmware load but just not directly? [ 197.571662] brcmfmac mmc1:0001:1: Direct firmware load for brcm/brcmfmac43455-sdio.raspberrypi,4-model-b.txt failed with error -2

root@raspberrypi:/home/pi# dmesg | grep brc
<snip>
[  197.570461] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[  197.571662] brcmfmac mmc1:0001:1: Direct firmware load for brcm/brcmfmac43455-sdio.raspberrypi,4-model-b.txt failed with error -2
[  197.572008] usbcore: registered new interface driver brcmfmac
[  197.753387] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[  197.765297] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Sep 22 2020 21:58:35 version 7.45.206 (nexmon.org: 2.2.2-326-g050d-dirty-34) FWID 01-88ee44ea
[  205.621179] brcmfmac: brcmf_cfg80211_set_power_mgmt: power save enabled
[  208.783621] brcmfmac: brcmf_vif_add_validate: Attempt to add a MONITOR interface...
[  208.783644] brcmfmac: brcmf_mon_add_vif: brcmf_mon_add_vif called
[  208.783658] brcmfmac: brcmf_mon_add_vif: Adding vif "mon0"
[  324.312363] ieee80211 phy2: brcmf_fw_crashed: Firmware has halted or crashed
[  328.140993] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4097, -110
[  335.591685] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4100, -110
[  343.360257] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4102, -110
[  351.046731] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[  358.811672] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4105, -110
[  361.693042] ieee80211 phy2: brcmf_proto_bcdc_query_dcmd: brcmf_proto_bcdc_msg failed w/status -110
[  361.693063] ieee80211 phy2: brcmf_cfg80211_get_channel: chanspec failed (-110)
[  364.254228] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4097, -110
[  367.295472] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4098, -110
[  370.336577] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110
[  373.377489] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4100, -110
[  376.418377] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4101, -110
[  379.459105] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4102, -110
[  382.499740] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[  385.540312] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4104, -110
[  388.580695] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4105, -110
[  391.621150] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4106, -110
[  394.661423] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4107, -110
<snip>

meliodasren avatar Sep 23 '20 02:09 meliodasren

@DrSchottky , do you know where the nexmon version numbers come from for bcm43455c0? i.e. 7_45_154, 7_45_189, and 7_45_206?

They seem to coincide with version numbers that I'm finding elsewhere, maybe published by Broadcom/Cypress?

Is the nexmon firmware derived from or based on some formally released firmware? As you can tell, I'm completely new to nexmon.

The reason I'm asking, is that this bug looks very similar to another one found in iiab (the link to it is below). Long story short, it looks like all of the firmware versions since 2018-02-05 (7.45.132) all crash. Whereas only the first one used seems to be fine and not crash, from 2015-03-01 (7.45.18.0). It's almost like the last firmware from Broadcom worked fine and every release from Cypress since then crashes.

Based on their table, is it safe to conclude that this issue is not relating to packet injection or monitor mode but is instead based on load? i.e. Would using the Raspberry Pi as an AP require packet injection or monitor mode type behaviour?

As far as I can tell, iiab is just heavily loading the WiFi by having up to 32 clients connect to it as an AP.

Here's the table I'm referring to: https://github.com/iiab/iiab/issues/823#issuecomment-662285202

I'm wondering if it would be possible to make a nexmon version for bcm43455c0 based off of 7.45.18.0 and see if it fixes this bug.

As a reference, it looks like iiab is still using that original firmware from 2015 just to get the stability/throughput that they need to have 32 clients connected to the Raspberry Pi's WiFi simultaneously.

meliodasren avatar Sep 25 '20 00:09 meliodasren

@meliodasren yes nexmon fw numbering matches the original Cypress fw they're based on

DrSchottky avatar Sep 26 '20 10:09 DrSchottky

Same problem for me. After run owl -i mon0 -N, the wifi driver report brcmf_fw_crashed: Firmware has halted or crashed Testing on rpi4 with 7_45_206 patches

qdk0901 avatar Nov 02 '20 08:11 qdk0901

I have also this problem on a Raspberry Pi 4B. I have tested with both 7_45_189 and 7_45_206 patches. Basically what I do is installing the firmware (make install-firmware), creating the mon0 monitor interface, putting it up, changing it to channel 1, and then just aireplay-ng --test mon0. The aireplay test starts working, but it crashes really fast (about a second after starting). The journal messages shown during the process described above are:

Dec 04 08:56:40 raspberrypi kernel: ieee80211 phy0: brcmf_cfg80211_get_tx_power: error (-5)
Dec 04 08:56:41 raspberrypi dhcpcd[521]: wlan0: removing interface
Dec 04 08:56:41 raspberrypi systemd[1]: Starting Load/Save RF Kill Switch Status...
Dec 04 08:56:41 raspberrypi systemd[1]: Started Load/Save RF Kill Switch Status.
Dec 04 08:56:41 raspberrypi kernel: usbcore: deregistering interface driver brcmfmac
Dec 04 08:56:41 raspberrypi dhcpcd-run-hooks[3333]: wlan0: stopping wpa_supplicant
Dec 04 08:56:41 raspberrypi kernel: brcmfmac: loading out-of-tree module taints kernel.
Dec 04 08:56:41 raspberrypi kernel: brcmfmac: F1 signature read @0x18000000=0x15264345
Dec 04 08:56:41 raspberrypi kernel: brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
Dec 04 08:56:41 raspberrypi kernel: brcmfmac mmc1:0001:1: Direct firmware load for brcm/brcmfmac43455-sdio.raspberrypi,4-model-b.txt failed with error -2
Dec 04 08:56:41 raspberrypi kernel: usbcore: registered new interface driver brcmfmac
Dec 04 08:56:42 raspberrypi kernel: brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
Dec 04 08:56:42 raspberrypi kernel: brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Dec  4 2020 08:56:27 version 7.45.206 (nexmon.org: 2.2.2-329-g5dbc-dirty-7) FWID 01-88ee44ea
Dec 04 08:56:42 raspberrypi systemd-udevd[3315]: Using default interface naming scheme 'v240'.
Dec 04 08:56:42 raspberrypi dhcpcd-run-hooks[3360]: wlan0: starting wpa_supplicant
Dec 04 08:56:42 raspberrypi kernel: brcmfmac: brcmf_cfg80211_set_power_mgmt: power save enabled
Dec 04 08:56:42 raspberrypi dhcpcd[521]: wlan0: connected to Access Point `'
Dec 04 08:56:42 raspberrypi dhcpcd[521]: wlan0: waiting for carrier
Dec 04 08:56:42 raspberrypi dhcpcd[521]: wlan0: carrier acquired
Dec 04 08:56:42 raspberrypi dhcpcd[521]: wlan0: IAID 32:d0:bc:ba
Dec 04 08:56:42 raspberrypi dhcpcd[521]: wlan0: adding address fe80::bdab:2267:9a2c:3d59
Dec 04 08:56:42 raspberrypi avahi-daemon[347]: Joining mDNS multicast group on interface wlan0.IPv6 with address fe80::bdab:2267:9a2c:3d59.
Dec 04 08:56:42 raspberrypi avahi-daemon[347]: New relevant interface wlan0.IPv6 for mDNS.
Dec 04 08:56:42 raspberrypi avahi-daemon[347]: Registering new address record for fe80::bdab:2267:9a2c:3d59 on wlan0.*.
Dec 04 08:56:42 raspberrypi dhcpcd[521]: wlan0: carrier lost
Dec 04 08:56:42 raspberrypi dhcpcd[521]: wlan0: deleting address fe80::bdab:2267:9a2c:3d59
Dec 04 08:56:42 raspberrypi avahi-daemon[347]: Withdrawing address record for fe80::bdab:2267:9a2c:3d59 on wlan0.
Dec 04 08:56:42 raspberrypi avahi-daemon[347]: Leaving mDNS multicast group on interface wlan0.IPv6 with address fe80::bdab:2267:9a2c:3d59.
Dec 04 08:56:42 raspberrypi avahi-daemon[347]: Interface wlan0.IPv6 no longer relevant for mDNS.
Dec 04 08:56:47 raspberrypi systemd[1]: systemd-rfkill.service: Succeeded.
Dec 04 08:57:01 raspberrypi kernel: brcmfmac: brcmf_vif_add_validate: Attempt to add a MONITOR interface...
Dec 04 08:57:01 raspberrypi kernel: brcmfmac: brcmf_mon_add_vif: brcmf_mon_add_vif called
Dec 04 08:57:01 raspberrypi kernel: brcmfmac: brcmf_mon_add_vif: Adding vif "mon0"
Dec 04 08:57:01 raspberrypi kernel: brcmfmac: nexmon_nl_ioctl_handler: NEXMON: nexmon_nl_ioctl_handler: Enter
Dec 04 08:57:01 raspberrypi kernel: brcmfmac: nexmon_nl_ioctl_handler: NEXMON: nexmon_nl_ioctl_handler: 0058454e 32 48
Dec 04 08:57:01 raspberrypi kernel: brcmfmac: nexmon_nl_ioctl_handler: NEXMON: nexmon_nl_ioctl_handler: calling brcmf_fil_cmd_data_set, cmd: 263
Dec 04 08:57:01 raspberrypi kernel: brcmfmac: nexmon_nl_ioctl_handler: NEXMON: nexmon_nl_ioctl_handler: Exit
Dec 04 08:57:10 raspberrypi kernel: device mon0 entered promiscuous mode
Dec 04 08:57:18 raspberrypi kernel: ieee80211 phy1: brcmf_fw_crashed: Firmware has halted or crashed

When this happens, I have to reboot the board. Reloading the driver does not work.

I forgot to write I am using latest RaspberryPi OS, that supposedly supports injection with 7_45_206 patches.

doragasu avatar Dec 04 '20 09:12 doragasu

Please post the output of sudo -u root cat /sys/kernel/debug/ieee80211/$(iw wlan0 info | gawk '/wiphy/ {printf "phy" $2}')/forensics when reporting crashes.

When injecting frames fast sometimes a null pointer reference crashes the chip while packets get dequeued. A quick fix is to check for the null reference and exit the dequeue loop in case there is one. The next injected frame will trigger queue processing again.

TLDR: put below into sendframe.c

__attribute__((naked))
void
check_scb(void)
{
     asm(
        "cmp r6, #0\n"             // check if pkt->scb is null
        "bne nonnull\n"
        "add lr,lr,0x178\n"        // if null adapt lr to jump out of pkt dequeue loop
        "b return\n"
        "nonnull:\n"
        "ldr.w r3,[r7,#0xe8]\n"    // get scb->cfg->flags (crashed the chip when scb was null)
        "return:\n"
        "push {lr}\n"
        "pop {pc}\n"
    );  
}

__attribute__((at(0x1AF378, "", CHIP_VER_BCM43455c0, FW_VER_7_45_189)))
__attribute__((at(0x1AABB0, "", CHIP_VER_BCM43455c0, FW_VER_7_45_206)))
__attribute__((naked))
void
patch_null_pointer_scb(void)
{
    asm(
        "bl check_scb\n"    // branch to null pointer check instead of accessing possibly invalid cfg
    );  
}

jlinktu avatar Dec 04 '20 17:12 jlinktu

Thanks a lot for help!

I have tried the patch, and it helps, but does not completely remove the problem. Without the patch, if I inject packets continuously, was able to crash the chip instantly. With this patch it takes several seconds, but it finally crashes. This is the requested crash info:

$ sudo -u root cat /sys/kernel/debug/ieee80211/$(iw wlan0 info | gawk '/wiphy/ {printf "phy" $2}')/forensics
dongle trap info: type 0x4 @ epc 0x0008b5a4
  cpsr 0x2000019f spsr 0x200001bf sp 0x0025fe08
  lr   0x001a4fcf pc   0x0008b5a4 offset 0x25fdb0
  r0   0xffffffff r1   0x0020f818 r2 0x0025fe64 r3 0x00000000
  r4   0x0022bbbc r5   0x0025fe64 r6 0xffffffff r7 0x0022bbbc
ar "rsdb_mode"
000000.221 wl0: wlc_iovar_op: rsdb_mode BCME -23 (Unsupported)
000000.235 wl0: wlc_phy_set_regtbl_on_femctrl: FIXME bt_coex
000000.288 wl0: wlc_iovar_op: bw_cap BCME -5 (Not down)
000000.381 wl0: unable to find iovar "toe_ol"
000000.381 wl0: wlc_iovar_op: toe_ol BCME -23 (Unsupported)
000000.382 wl0: wl_open
000000.491 wl0: wl_open
000000.500 wl0: wlc_phy_set_regtbl_on_femctrl: FIXME bt_coex
000000.558 wl0: unable to find iovar "nd_hostip_clear"
000000.558 wl0: wlc_iovar_op: nd_hostip_clear BCME -23 (Unsupported)
000000.579 wl0: unable to find iovar "nd_hostip_clear"
000000.579 wl0: wlc_iovar_op: nd_hostip_clear BCME -23 (Unsupported)
000097.085 wl0: unable to find iovar "toe_ol"
000097.085 wl0: wlc_iovar_op: toe_ol BCME -23 (Unsupported)
000097.095 wl0: wlc_phy_set_regtbl_on_femctrl: FIXME bt_coex
000113.439 wl0: wlc_phy_chan2freq_20691: channel 255 not found in channel table
000113.506 wl0: wlc_phy_chan2freq_20691: channel 255 not found in channel table
000113.743 wl0: wlc_phy_chan2freq_20691: channel 255 not found in channel table
000134.212 wl0: wlc_phy_chan2freq_20691: channel 255 not found in channel table
000137.204 
FWID 01-88ee44ea
flags 1
000137.204 
TRAP 4(25fdb0): pc 8b5a4, lr 1a4fcf, sp 25fe08, cpsr 2000019f, spsr 200001bf
000137.204   dfsr 8, dfar ffffffff
000137.204   r0 ffffffff, r1 20f818, r2 25fe64, r3 0, r4 22bbbc, r5 25fe64, r6 ffffffff
000137.204   r7 22bbbc, r8 20f818, r9 20f818, r10 25fecc, r11 0, r12 0
000137.204 
   sp+0 0022bbbc 00000000 00000000 0022bbbc
000137.204   sp+10 00000008 001aba83 00000002 0025c094

000137.204 sp+14 001aba83
000137.204 sp+34 0019b30f
000137.204 sp+4c 0019b59d
000137.204 sp+84 0019c1df
000137.204 sp+98 00002001
000137.204 sp+bc 001bdc85
000137.204 sp+c8 00000a81
000137.204 sp+d4 0019da49
000137.204 sp+e4 00010103
000137.204 sp+100 00007f19
000137.204 sp+124 001c5a29
000137.204 sp+14c 001a1da1
000137.204 sp+15c 001a1ddd
000137.204 sp+16c 0019a5b9
000137.204 sp+174 0019ac89

doragasu avatar Dec 14 '20 10:12 doragasu

I am willing to spend some time trying to fix this, but unfortunately figuring this out might be too much for me. Any advice on how to work on this? I hope I do not need to read and understand a complete PhD thesis plus reverse engineer the firmware for the RPi4 WiFi chipset.

doragasu avatar Mar 02 '21 10:03 doragasu

I can confirm this; I am trying to use the internal chip for Wifibroadcast (video over raw wifi) and with the patch it starts up, I can "ping" the other side, but as soon as I try to transmit any meaningful amount of data (eg a video stream), the firmware crashes.

Log attached: brcm-crash.log

IGNNE avatar May 09 '21 16:05 IGNNE

I'm wondering if it would be possible to make a nexmon version for bcm43455c0 based off of 7.45.18.0 and see if it fixes this bug.

Does this help @DrSchottky / @IGNNE / @doragasu ? @meliodasren raised a good point, just wondering if any of you had a chance to test this?

davenicoll avatar May 02 '22 17:05 davenicoll

Unfortunately I do not have time to spend on this anymore. I worked around the issue by using an external WiFi dongle.

doragasu avatar May 03 '22 07:05 doragasu

Hi, I've tried using nexmon on rpi 4, but injection does not work at all

ElDavoo avatar Aug 31 '22 08:08 ElDavoo