FreeBSD Handbook
Prev	Chapter 19 Advanced Networking	Next

19.7 Diskless Operation

Updated by Jean-François Dockès.

A FreeBSD machine can boot over the network and operate without a local disk, using filesystems mounted from an NFS server. No system modification is necessary, beyond standard configuration files. Such a system is easy to set up because all the necessary elements are readily available:

There are at least two possible methods to load the kernel over the network:
- PXE: The Intel® Preboot Execution Environment system is a form of smart boot ROM built into some networking cards or motherboards. See pxeboot(8) for more details.
- The etherboot port (net/etherboot) produces ROM-able code to boot kernels over the network. The code can be either burnt into a boot PROM on a network card, or loaded from a local floppy (or hard) disk drive, or from a running MS-DOS® system. Many network cards are supported.
A sample script (/usr/share/examples/diskless/clone_root) eases the creation and maintenance of the workstation's root filesystem on the server. The script will probably require a little customization but it will get you started very quickly.
Standard system startup files exist in /etc to detect and support a diskless system startup.
Swapping, if needed, can be done either to an NFS file or to a local disk.

There are many ways to set up diskless workstations. Many elements are involved, and most can be customized to suit local taste. The following will describe the setup of a complete system, emphasizing simplicity and compatibility with the standard FreeBSD startup scripts. The system described has the following characteristics:

The diskless workstations use a shared read-only root filesystem, and a shared read-only /usr.

The root filesystem is a copy of a standard FreeBSD root (typically the server's), with some configuration files overridden by ones specific to diskless operation or, possibly, to the workstation they belong to.

The parts of the root which have to be writable are overlaid with mfs(8) filesystems. Any changes will be lost when the system reboots.
The kernel is loaded by etherboot , using DHCP (or BOOTP) and TFTP.

Caution: As described, this system is insecure. It should live in a protected area of a network, and be untrusted by other hosts.

19.7.1 Setup Instructions

19.7.1.1 Configuring DHCP/BOOTP

There are two protocols that are commonly used to boot a workstation that retrieves its configuration over the network: BOOTP and DHCP. They are used at several points in the workstation bootstrap:

etherboot uses DHCP (by default) or BOOTP (needs a configuration option) to find the kernel. (PXE uses DHCP).
The kernel uses BOOTP to locate the NFS root.

It is possible to configure a system to use only BOOTP. The bootpd(8) server program is included in the base FreeBSD system.

However, DHCP has a number of advantages over BOOTP (nicer configuration files, possibility of using PXE, plus many others not directly related to diskless operation), and we shall describe both a pure BOOTP, and a BOOTP+DHCP configuration, with an emphasis on the latter, which will use the ISC DHCP software package.

19.7.1.1.1 Configuration Using ISC DHCP

The isc-dhcp server can answer both BOOTP and DHCP requests.

As of release 4.4, isc-dhcp 3.0 is not part of the base system. You will first need to install the net/isc-dhcp3 port or the corresponding package. Please refer to Chapter 4 for general information about ports and packages.

Once isc-dhcp is installed, it needs a configuration file to run, (normally named /usr/local/etc/dhcpd.conf). Here follows a commented example:

          default-lease-time 600;
          max-lease-time 7200;
          authoritative;

          option domain-name "example.com";
          option domain-name-servers 192.168.4.1;
          option routers 192.168.4.1;

          subnet 192.168.4.0 netmask 255.255.255.0 {
            use-host-decl-names on; 
            option subnet-mask 255.255.255.0;
            option broadcast-address 192.168.4.255;

            host margaux {
              hardware ethernet 01:23:45:67:89:ab;
              fixed-address margaux.example.com;
              next-server 192.168.4.4;
              filename "/tftpboot/kernel.diskless";
              option root-path "192.168.4.4:/data/misc/diskless";
            }
          }

: This option tells dhcpd to send the value in the host declarations as the hostname for the diskless host. An alternate way would be to add an option host-name margaux inside the host declarations.
: The next-server directive designates the TFTP server (the default is to use the same host as the DHCP server).
: The filename directive defines the file that etherboot will load as a kernel.

Note: PXE appears to prefer a relative file name, and it loads pxeboot, not the kernel (option filename "pxeboot").
: The root-path option defines the path to the root filesystem, in usual NFS notation.

19.7.1.1.2 Configuration Using BOOTP

Here follows an equivalent bootpd configuration. This would be found in /etc/bootptab.

Please note that etherboot must be compiled with the non-default option NO_DHCP_SUPPORT in order to use BOOTP, and that PXE needs DHCP. The only obvious advantage of bootpd is that it exists in the base system.

          .def100:\
            :hn:ht=1:sa=192.168.4.4:vm=rfc1048:\
            :sm=255.255.255.0:\
            :ds=192.168.4.1:\
            :gw=192.168.4.1:\
            :hd="/tftpboot":\
            :bf="/kernel.diskless":\
            :rp="192.168.4.4:/data/misc/diskless":

          margaux:ha=0123456789ab:tc=.def100

19.7.1.2 Preparing a Boot Program with Etherboot

Etherboot's Web site contains extensive documentation mainly intended for Linux systems, but nonetheless containing useful information. The following will just outline how you would use etherboot on a FreeBSD system.

You must first install the net/etherboot package or port. The etherboot port can normally be found in /usr/ports/net/etherboot. If the ports tree is installed on your system, just typing make in this directory should take care of everything. Else refer to Chapter 4 for information about ports and packages.

For our setup, we shall use a boot floppy. For other methods (PROM, or dos program), please refer to the etherboot documentation.

To make a boot floppy, insert a floppy in the drive on the machine where you installed etherboot, then change your current directory to the src directory in the etherboot tree and type:

     # gmake bin32/devicetype.fd0

devicetype depends on the type of the Ethernet card in the diskless workstation. Refer to the NIC file in the same directory to determine the right devicetype.

19.7.1.3 Configuring the TFTP and NFS Servers

You need to enable tftpd on the TFTP server:

Create a directory from which tftpd will serve the files, e.g. /tftpboot.
Add this line to your /etc/inetd.conf:
```
tftp    dgram   udp     wait    root  /usr/libexec/tftpd    tftpd -s /tftpboot
```
Note: It appears that at least some PXE versions want the TCP version of TFTP. In this case, add a second line, replacing dgram udp with stream tcp.
Tell inetd to reread its configuration file:
```
# kill -HUP `cat /var/run/inetd.pid`
```

You can place the tftpboot directory anywhere on the server. Make sure that the location is set in both inetd.conf and dhcpd.conf.

You also need to enable NFS and export the appropriate filesystem on the NFS server.

Add this to /etc/rc.conf:
```
nfs_server_enable="YES"
```
Export the filesystem where the diskless root directory is located by adding the following to /etc/exports (adjust the volume mount point and replace margaux with the name of the diskless workstation):
```
/data/misc -alldirs -ro margaux
```
Tell mountd to reread its configuration file. If you actually needed to enable NFS in /etc/rc.conf at the first step, you probably want to reboot instead.
```
# kill -HUP `cat /var/run/mountd.pid`
```

19.7.1.4 Building a Diskless Kernel

Create a kernel configuration file for the diskless client with the following options (in addition to the usual ones):

          options     BOOTP          # Use BOOTP to obtain IP address/hostname
          options     BOOTP_NFSROOT  # NFS mount root filesystem using BOOTP info
          options     BOOTP_COMPAT   # Workaround for broken bootp daemons.

You may also want to use BOOTP_NFSV3 and BOOTP_WIRED_TO (refer to LINT).

Build the kernel (See Chapter 9), and copy it to the tftp directory, under the name listed in dhcpd.conf.

19.7.1.5 Preparing the Root Filesystem

You need to create a root filesystem for the diskless workstations, in the location listed as root-path in dhcpd.conf.

The easiest way to do this is to use the /usr/share/examples/diskless/clone_root shell script. This script needs customization, at least to adjust the place where the filesystem will be created (the DEST variable).

Refer to the comments at the top of the script for instructions. They explain how the base filesystem is built, and how files may be selectively overridden by versions specific to diskless operation, to a subnetwork, or to an individual workstation. They also give examples for the diskless /etc/fstab and /etc/rc.conf files.

The README files in /usr/share/examples/diskless contain a lot of interesting background information, but, together with the other examples in the diskless directory, they actually document a configuration method which is distinct from the one used by clone_root and /etc/rc.diskless[12], which is a little confusing. Use them for reference only, except if you prefer the method that they describe, in which case you will need customized rc scripts.

19.7.1.6 Configuring Swap

If needed, a swap file located on the server can be accessed via NFS. The exact bootptab or dhcpd.conf options are not clearly documented at this time. The following configuration suggestions have been reported to work in some installations using isc-dhcp 3.0rc11.

Add the following lines to dhcpd.conf:
```
              # Global section
              option swap-path code 128 = string;
              option swap-size code 129 = integer 32;

              host margaux {
                ... # Standard lines, see above
                option swap-path "192.168.4.4:/netswapvolume/netswap";
                option swap-size 64000;
              }
       
```
The idea is that, at least for a FreeBSD client, DHCP/BOOTP option code 128 is the path to the NFS swap file, and option code 129 is the swap size in kilobytes. Older versions of dhcpd allowed a syntax of option option-128 "..., which does not seem to work any more.

/etc/bootptab would use the following syntax instead:

T128="192.168.4.4:/netswapvolume/netswap":T129=64000

On the NFS swap file server, create the swap file(s)

              # mkdir /netswapvolume/netswap
              # cd /netswapvolume/netswap
              # dd if=/dev/zero bs=1024 count=64000 of=swap.192.168.4.6
              # chmod 0600 swap.192.168.4.6

192.168.4.6 is the IP address for the diskless client.

On the NFS swap file server, add the following line to /etc/exports:
```
         /netswapvolume  -maproot=0:10 -alldirs margaux
       
```
Then tell mountd to reread the exports file, as above.

19.7.1.7 Miscellaneous Issues

19.7.1.7.1 Running with a Read-only `/usr`

If the diskless workstation is configured to run X, you will have to adjust the xdm configuration file, which puts the error log on /usr by default.

19.7.1.7.2 Using a Non-FreeBSD Server

When the server for the root filesystem is not running FreeBSD, you will have to create the root filesystem on a FreeBSD machine, then copy it to its destination, using tar or cpio.

In this situation, there are sometimes problems with the special files in /dev, due to differing major/minor integer sizes. A solution to this problem is to export a directory from the non-FreeBSD server, mount this directory onto a FreeBSD machine, and run MAKEDEV on the FreeBSD machine to create the correct device entries (FreeBSD 5.0 and later use devfs(5) to allocate device nodes transparently for the user, running MAKEDEV on these versions is useless).

Prev	Home	Next
NFS	Up	ISDN

This, and other documents, can be downloaded from ftp://ftp.FreeBSD.org/pub/FreeBSD/doc/.

For questions about FreeBSD, read the documentation before contacting <questions@FreeBSD.org>.
For questions about this documentation, e-mail <doc@FreeBSD.org>.