How to PXE boot a Raspberry

So far, we’ve built an NFS server and a TFTP server. Next, we’re going to populate those servers with the files that are needed to boot a Raspberry. The contents of /boot will be served by the TFTP server and the entirety of /root will be served by the NFS server. Download the latest Raspberry Pi OS image and mount it so we can copy the files into place. Choose your favorite image from here. IF you right-click on one of the blue download boxes and copy the link address, you can use wget to download the image right to your TFTP/NFS server.

wget https://downloads.raspberrypi.org/raspios_lite_arm64/images/raspios_lite_arm64-2022-01-28/2022-01-28-raspios-bullseye-arm64-lite.zip

Next, we’ll unzip it and temporarily mount the two volumes so we can copy them into place:

unzip 2022-01-28-raspios-bullseye-arm64-lite.zip

mkdir bootpart
mkdir rootpart
mkdir boot
mkdir root
kpartx -av *.img
mount /dev/mapper/loop0p1 bootpart/
mount /dev/mapper/loop0p2 rootpart/

The first two directories are just temporary mount points. The next two directories are the locations that will be served by NFS. kpartx finds the partitions in the image and makes them available as loop filesystems, which we can mount and copy the files out of:

cp -rp bootpart/.   boot
cp -rp rootpart/.   root

Copying root takes a couple of minutes. This latest arm64-lite OS is 1.2G. Be patient! Don’t skip the “-rp” part! We need a recursive copy with all the permissions!

I think I’d like to let each Raspberry that boots from here have its own root directory. So, I’ll let these be the virgin, master copies and I’ll make a separate copy for each client. We’ll need to name those directories with the serial number of the Pi using it. Yeah, the serial number. Jump on the raspberry that’s going to be netbooting and grab the serial number:

cat /proc/cpuinfo | grep Serial | awk -F ': ' '{print $2}' | tail -c 9
ad87da29

Now, embed that 8 digit number in the path on the NFS server, each with a boot directory in it. The serial of MY RPi is ad87da29. Substitute YOUR serial number in all of the following!

mkdir /data1/ad87da29
mkdir /data1/rpi4-ad87da29

AND copy from the “masters” into the “personalized directories”:

cd /data1
cp -r boot/. ad87da29
cp -r root/. rpi4-ad87da29
cp -r boot/. rpi4-ad87da29/boot

Then, let’s just clean up a little. We’ve got our pristine copies in root and boot for our next clients, so we don’t need the image or the split-out parts.

rm 2022-01-28-raspios-bullseye-arm64-lite.*
umount *part
rm *part -rf

Reboot your server to restart all the services and make sure everything is functional. Let’s set our sites on the first netboot client. Back on the Raspberry that we got the serial number from, let’s check the boot order. Newer RPi 3’s and 4’s should have netboot capability. We just need to make sure that netboot is included in the “boot order” or “list of places to look for an OS”. The command is “vcgencmd
bootloader_config”:

# vcgencmd bootloader_config
BOOT_UART=0
WAKE_ON_GPIO=1
POWER_OFF_ON_HALT=0

[all]
BOOT_ORDER=0xf14

That’s not going to work! We need 0xf21 for that last bit. We can fix this in raspi-config.

Advanced Options > Boot Loader Version > Latest

<OK>

Advanced Options > Boot Order > Network Boot

<OK>

ESC and reboot.

OK. Now we need to do a couple of tweaks to the files. Our intent is to have the files that get sent to the netboot client set up to mount the OS into the root partition. Sneaky business. The file that ends up in /etc/fstab will need to know that the / partition is an NFS mount. The /boot/cmdline.txt file needs to get screwed around into believing that the OS is out in NFS land. IF you were paying attention, you saw that the boot partition is getting sent to the netboot client via TFTP and the root partition will get mounted via NFS for the life of the boot.

The worst part of this whole scenario is that NFS v4 is the standard now, but we need to dial back to v3 for most of this. So, there are a couple of points where we need to explicitly configure v3 for all of this to work! So, a couple more tweaks on the TFTP/NFS server that I’ve affectionately named “boots”.

Edit both these files:

boots:/data1/ad87da29/cmdline.txt  -and-
boots:/data1/rpi4-ad87da29/boot/cmdline.txt

One of the files gets used as the boot directory through TFTP. The second one gets referenced once you’re up and running. The contents of the file should look like this, with a couple of customizations for your situation. “nfsroot” should be the IP address of your NFS server, and the directory where you copied “rootpart” rpi4-ad87da29 is my root directory copy. You’ll need to change that last part to YOUR serial number:

console=serial0,115200 console=tty root=/dev/nfs nfsroot=192.168.1.44:/data1/rpi4-ad87da29,vers=3 rw ip=dhcp rootwait elevator=deadline

The file “boots:/data1/rpi4-ad87da29/etc/fstab” needs to get edited the same way. Take out everything but the proc line and add the following, edit to match your NFS settings:

192.168.1.44:/data1/rpi4-ad87da29       /       nfs     defaults,tcp    0       0

Your NFS settings? They go in boots:/etc/exports DO NOT put a space between the network definition and the (NFS options)! You’ll end up with a read-only share or no share at all!

/data1  192.168.1.0/24(rw,sync,no_subtree_check,no_root_squash)

OK. Last little tweak. You need to configure your DHCP server to tell that we have a TFTP boot server available! MY DHCP server is pi-hole so I just created a new options file in:

pi-hole:/etc/dnsmasq.d/02-pihole-dhcp-options.conf

dhcp-option=66,192.168.1.44
dhcp-boot=pxelinux.0,,192.168.1.44

Don’t be tempted to edit one of the existing files in /etc/dnsmasq.d. It could get overwritten! You’re not supposed to be in there…

OK Pull out the SD card and the thumb drive. Netboot your Raspberry!