firmware
firmware copied to clipboard
raspberrypi
Network Boot different DHCP and TFTP Server
I'm having a very similar issue to #871 which was closed nearly two years ago. However the solution didn't work for me!
I'm using a dhcp proxy (dnsmasq) to only provide pxe information to the raspberry pi. If I set the boot menu text to "Raspberry Pi Boot" it request the tftp files from the dhcp server and not the tftp server. If I do not set this menu text, the Raspberry pi is just retrying sending dhcp discoveries. What am I doing wrong?
tcpdump
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
20:46:02.406872 IP (tos 0x0, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 350)
0.0.0.0.68 > 255.255.255.255.67: [no cksum] BOOTP/DHCP, Request from b8:27:eb:2f:aa:eb, length 322, xid 0x26f30339, Flags [none] (0x0000)
Client-Ethernet-Address b8:27:eb:2f:aa:eb
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Discover
Parameter-Request Option 55, length 14:
Vendor-Option, Vendor-Class, BF, Option 128
Option 129, Option 130, Option 131, Option 132
Option 133, Option 134, Option 135, TFTP
Subnet-Mask, Default-Gateway
ARCH Option 93, length 2: 0
NDI Option 94, length 3: 1.2.1
GUID Option 97, length 17: 0.235.170.47.178.235.170.47.178.235.170.47.178.235.170.47.178
Vendor-Class Option 60, length 32: "PXEClient:Arch:00000:UNDI:002001"
20:46:02.423471 IP (tos 0x0, ttl 128, id 62593, offset 0, flags [none], proto UDP (17), length 328)
192.168.10.96.67 > 255.255.255.255.68: [udp sum ok] BOOTP/DHCP, Reply, length 300, xid 0x26f30339, Flags [none] (0x0000)
Your-IP 192.168.11.21
Server-IP 192.168.10.96
Client-Ethernet-Address b8:27:eb:2f:aa:eb
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Offer
Subnet-Mask Option 1, length 4: 255.255.254.0
RN Option 58, length 4: 345600
RB Option 59, length 4: 604800
Lease-Time Option 51, length 4: 691200
Server-ID Option 54, length 4: 192.168.10.96
Default-Gateway Option 3, length 4: 192.168.10.1
20:46:03.408984 IP (tos 0xc0, ttl 64, id 36009, offset 0, flags [none], proto UDP (17), length 351)
192.168.10.20.67 > 255.255.255.255.68: [bad udp cksum 0xcc18 -> 0x5c86!] BOOTP/DHCP, Reply, length 323, xid 0x26f30339, Flags [Broadcast] (0x8000)
Client-Ethernet-Address b8:27:eb:2f:aa:eb
Vendor-rfc1048 Extensions
Magic Cookie 0x63825363
DHCP-Message Option 53, length 1: Offer
Server-ID Option 54, length 4: 192.168.10.20
Vendor-Class Option 60, length 9: "PXEClient"
GUID Option 97, length 17: 0.235.170.47.178.235.170.47.178.235.170.47.178.235.170.47.178
Vendor-Option Option 43, length 41: 6.1.3.10.4.0.80.88.69.8.7.0.1.1.192.168.10.24.9.20.0.1.17.82.97.115.112.98.101.114.114.121.32.80.105.32.66.111.111.116.255
20:46:03.410570 IP (tos 0x0, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 49)
192.168.11.21.49153 > 192.168.10.20.69: [no cksum] 21 RRQ "bootcode.bin" octet
20:46:14.353519 IP (tos 0x0, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 48)
192.168.11.21.49154 > 192.168.10.20.69: [no cksum] 20 RRQ "bootsig.bin" octet
dnsmasq config
port=0
log-dhcp
dhcp-range=192.168.10.24,proxy
pxe-service=0, "Raspberry Pi Boot", 1, 192.168.10.24
dhcp-option=option:server-ip-address,192.168.10.24
dhcp-option=option:tftp-server,192.168.10.24
cc @ghollingworth as he answered the linked issue.
Incidentally, I observed the same issue with my Pi4, and there was a bug that has just been fixed in the latest EEPROM beta version:
https://github.com/raspberrypi/rpi-eeprom/blob/master/firmware/beta/pieeprom-2019-10-16.bin
Since you're apparently using a Pi3, I wonder if you would have some luck when you put the IP address of your TFTP server into DHCP option 66.
IIRC Pi3 expects the Option43 response to come from the TFTP server not the DHCP server. If the DHCP server sets Option66 to the dotted decimal IPV4 address then that should work. You might want to ensure Option66 appears after Option43
RPi 3B
DNSMasq running on router 10.0.1.1 TFTP Server running on 10.0.1.10
Never seems to hit the TFTP server. Tried commenting out option 43 and restarted dnsmasq.
dnsmasq.conf
dhcp-mac=rpi-boot,b8:27:eb:d9:fb:9d
#dhcp-option-force=net:rpi-boot,43,"Raspberry Pi Boot "
dhcp-option-force=net:rpi-boot,66,10.0.1.10
#Following not needed, RPi always requests "bootcode.bin"
#dhcp-option-force=net:rpi-boot,67,"bootcode.bin"
#dhcp-option-force=net:rpi-boot,encap:175,176,1b
Tested with various things above RPi 3B seems to be getting the DHCP options but not aware what it is doing after that.
Router (10.0.1.1)
sudo systemctl status dnsmasq
router001 dnsmasq-dhcp[2648]: 653460281 vendor class: PXEClient:Arch:00000:UNDI:002001
router001 dnsmasq-dhcp[2648]: 653460281 DHCPDISCOVER(enp8s0) b8:27:eb:ce:fa:ce
router001 dnsmasq-dhcp[2648]: 653460281 tags: rpi-boot, enp8s0
router001 dnsmasq-dhcp[2648]: 653460281 DHCPOFFER(enp8s0) 10.0.1.157 b8:27:eb:ce:fa:ce
router001 dnsmasq-dhcp[2648]: 653460281 requested options: 43:vendor-encap, 60:vendor-class, 67:bootfile-name,
router001 dnsmasq-dhcp[2648]: 653460281 requested options: 128, 129, 130, 131, 132, 133, 134, 135, 66:tftp-server
router001 dnsmasq-dhcp[2648]: 653460281 next server: 10.0.1.1
router001 dnsmasq-dhcp[2648]: 653460281 sent size: 1 option: 53 message-type 2
router001 dnsmasq-dhcp[2648]: 653460281 sent size: 4 option: 54 server-identifier 10.0.1.1
router001 dnsmasq-dhcp[2648]: 653460281 sent size: 4 option: 51 lease-time 12h
router001 dnsmasq-dhcp[2648]: 653460281 sent size: 4 option: 58 T1 6h
router001 dnsmasq-dhcp[2648]: 653460281 sent size: 4 option: 59 T2 10h30m
router001 dnsmasq-dhcp[2648]: 653460281 sent size: 4 option: 1 netmask 255.255.255.0
router001 dnsmasq-dhcp[2648]: 653460281 sent size: 4 option: 28 broadcast 10.0.1.255
router001 dnsmasq-dhcp[2648]: 653460281 sent size: 12 option: 66 tftp-server 10.0.1.10
TFTP Server (10.0.1.10)
sudo journalctl -xe
#Testing with a TFTP client on another machine (not RPi) grabs file ok...
tftpserver001 in.tftpd[30761]: RRQ from ::ffff:10.0.1.101 filename bootcode.bin
#Nothing after that...
Edit:
So I got this working if I put an SD Card in the RPi with only bootcode.bin on it. But not when no SD-Card is present. (I've enabled the flag so it should work.)
vcgencmd otp_dump | grep 17:
17:3020000a
Also working without the SD-Card from a Windows 10 machine running tftpd64/dhcp and WinNFSd. (Both tftp/dhcp running on same system)
Not ideal in both cases as I was hoping to run without the sd-card on a linux server.
Can confirm this issue with a new 3B+.
While the Pi requests option 66 it seems to only use option 54. A packet capture of a conversation showing the DHCPREQUEST, OFFER and the TFTP request is attached: rpi_3bplus_pxe.log (it's a binary pcap, .log extension due to github file extension whitelisting)
As a workaround I set the DHCP server to override option 54 (dhcp-server-identifier) with the IP of the TFTP server.
EDIT: I did not see before or (yet) try https://github.com/raspberrypi/firmware/issues/871
Seems #871 doesn't work anymore because of changes in the bootcode.bin, I tried it with a bootcode.bin from 2017 (for example https://github.com/raspberrypi/firmware/blob/23047785b7414111cb7cb80aa9d0042c99fae437/boot/bootcode.bin ) and my 3B did get the files from the TFTP server my Option66 was pointing to.
In case anyone still has this issue:
On the raspberry pi 4/CM-4, it will correctly query the IP give in option 66 of the DHCP offer/ack, even if different from option 54. Tested it with both servers running on the same interface (assigned two different IPs) so if the issue still persists for others that might mean the error might be related to MAC addresses.
for this test I only set options: 1 (subnet), 54 (DHCP Server Identifier), 51 (IP Address lease time), 28 (Broadcast Address), 53 (DHCP Message type), 66 (TFTP Server Name), 255 (End) in that order. Despite what the documentation says in some places, option 43 or PXE related options are not needed on the 4. This might also work for the 3.
