xf.is blog

After doing a pfSense upgrade where the underlying base system has been upgraded like from 2.4.5p1 (FreeBSD 11.3) to 2.5.1 (FreeBSD 12.2) it is wise to upgrade the ZFS feature level.

If booting from GPT (UEFI) with ZFS zroot the zfs feature level upgrade process might have some gotchas that can cause the system not to boot.

Before doing this take configuration backup and have restore image/usb ready. This example is with a single disk setup no zpool mirror/stripe.

First check the zpool status:

[2.5.1-RELEASE][[email protected]]/root: zpool status
  pool: zroot
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: scrub repaired 0 in 0 days 00:00:10 with 0 errors on Mon Apr 26 16:06:54 2021
config:

        NAME        STATE     READ WRITE CKSUM
        zroot       ONLINE       0     0     0
          mfid0p4   ONLINE       0     0     0

Upgrade the zroot with:

[2.5.1-RELEASE][[email protected]]/root: zpool upgrade zroot
This system supports ZFS pool feature flags.

Enabled the following features on 'zroot':
  large_dnode
  allocation_classes

If you boot from pool 'zroot', don't forget to update boot code.
Assuming you use GPT partitioning and da0 is your boot disk
the following command will do it:

        gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0

Here is the tricky bit. The command that is shown might not be correct. Do not run it yet.

First check where the freebsd-boot boot partition is with “gpart show”:

[2.5.1-RELEASE][[email protected]]/root: gpart show
=>       40  266338224  mfid0  GPT  (127G)
         40     409600      1  efi  (200M)
     409640       1024      2  freebsd-boot  (512K)
     410664        984         - free -  (492K)
     411648    4194304      3  freebsd-swap  (2.0G)
    4605952  261730304      4  freebsd-zfs  (125G)
  266336256       2008       - free -  (1.0M)

First check the disk (in this case mfid0 but it might be for example da0) then find the index number for freebsd-boot (in this case 2). The upgrade command is then

[2.5.1-RELEASE][[email protected]]/root: gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 mfid0
partcode written to mfid0p2
bootcode written to mfid0

If the index (-i) is wrong (for example 1) it will cause the system to not boot.

Another way is to check the original BSD install log for bootcode if it is available:

[2.5.1-RELEASE][[email protected]]/root: grep "gpart bootcode" /var/log/bsdinstall_log
DEBUG: zfs_create_diskpart: gpart bootcode -p "/boot/boot1.efifat" -i 1 "mfid0"
DEBUG: zfs_create_diskpart: gpart bootcode -b "/boot/pmbr" -p "/boot/gptzfsboot" -i 2 "mfid0"

In this case it is the second line that has to be run (correct disk and index).

After that reboot the system and verify the system comes up normally.