Occasion:
Actually, my one wanne build such a boot environment on BTRFS Raid1 because of my experience with timeshift in the desktop field, but while my experimentation the question comes up, of course, is this possible with zfs. And when one research the web for zfs snapshot boot menu solution than the focus is really fast on the tool "zfsbootmenu" (zbm). This is an absolut awesome tool that every admin wants to use. And here follows how to integrate this in PVE.
Table of Contents:
1. My goals for this implementation:
2. Requirements:
A normal PVE 8 or 9 zfs (Raid1) installation. The PVE install disks are sda and sdb.
3. Configuration steps necessary 4 both Bios or UEFI boot:
4. UEFI install
5. BIOS install:
zbm Bios images erstellen
6. Configuration steps for autosnapshot necessary 4 both Bios and UEFI boot:
7. Back to proxmox-boot environment from UEFI:
8. Back to proxmox-boot environment from Bios:
9. Sources
Actually, my one wanne build such a boot environment on BTRFS Raid1 because of my experience with timeshift in the desktop field, but while my experimentation the question comes up, of course, is this possible with zfs. And when one research the web for zfs snapshot boot menu solution than the focus is really fast on the tool "zfsbootmenu" (zbm). This is an absolut awesome tool that every admin wants to use. And here follows how to integrate this in PVE.
Table of Contents:
- My goals for this implementation
- Requirements
- Configuration steps necessary 4 both Bios or UEFI boot
- UEFI install
- BIOS install
- Configuration steps for autosnapshot necessary 4 both Bios and UEFI boot
- Back to proxmox-boot environment from UEFI
- Back to proxmox-boot environment from Bios
- Sources
1. My goals for this implementation:
- as little change as possible to the original installation
- changes to new bootloader are based of a PVE ISO standard installation
- in a few steps back to the original PVE boot environment
- kernel image files stored on the root partition for snapshoting
- kernel pinning possible
- easy snapshot rollback, also when OS does not boot anymore
- snapshot boot menu
- MemTest im Bootmenu available
- system boots even if one drive is offline
- GRUB is used for all boot combinations except no loader is needed
2. Requirements:
A normal PVE 8 or 9 zfs (Raid1) installation. The PVE install disks are sda and sdb.
3. Configuration steps necessary 4 both Bios or UEFI boot:
- change zpool parameter
zfs set canmount=noauto rpoolzfs set canmount=noauto rpool/ROOTzfs set -u canmount=noauto rpool/ROOT/pve-1zfs set org.zfsbootmenu:active=on rpool/ROOTzpool set bootfs=rpool/ROOT/pve-1 rpool- mount ESPs
mkdir /boot/efi/sda2mkdir /boot/efi/sdb2mount /dev/sda2 /boot/efi/sda2mount /dev/sdb2 /boot/efi/sdb24. UEFI install
- install zbm to the ESP (EFI System Partition)
mkdir /boot/efi/sda2/EFI/zbmmkdir /boot/efi/sdb2/EFI/zbmwget -O /boot/efi/sda2/EFI/zbm/zbm.efi https://get.zfsbootmenu.org/eficp /boot/efi/sda2/EFI/zbm/zbm.efi /boot/efi/sdb2/EFI/zbm/- make zbm to the default Bootloader, when all UEFI profiles are cleared
cp /boot/efi/sda2/EFI/zbm/zbm.efi /boot/efi/sda2/EFI/BOOT/BOOTX64.EFIcp /boot/efi/sda2/EFI/zbm/zbm.efi /boot/efi/sdb2/EFI/BOOT/BOOTX64.EFI- copy Memtest-File to ESPs
cp /boot/memtest86+x64.efi /boot/efi/sda2/cp /boot/memtest86+x64.efi /boot/efi/sdb2/- create UEFI Boot Entries for zbm and Memtest
efibootmgr -c -d /dev/sda -p 2 -l "memtest86+x64.efi" -L "MemTest86 RAM Error Check - sda"efibootmgr -c -d /dev/sdb -p 2 -l "memtest86+x64.efi" -L "MemTest86 RAM Error Check - sdb"efibootmgr -c -d /dev/sdb -p 2 -l "EFI\zbm\zbm.efi" -u "spl.spl_hostid=0xe1b3ccf3" -L "Proxmox VE ZBM - sdb"efibootmgr -c -d /dev/sda -p 2 -l "EFI\zbm\zbm.efi" -u "spl.spl_hostid=0xe1b3ccf3" -L "Proxmox VE ZBM - sda"efibootmgr -t 5--> one get "spl.spl_hostid" with command "hostid" and has to add "0x" in front
5. BIOS install:
- change grub default config
nano /etc/default/grubGRUB_DEFAULT=4
nano /etc/grub.d/40_custommenuentry "PVE Boot with ZBM Bootloader" {
insmod part_gpt
insmod search_fs_uuid
insmod fat
search --fs-uuid --no-floppy --set=root 3AD9-98BA --set=root 3ADC-FB26
#set root=(hd0,gpt2)
linux ($root)/zbm/vmlinuz-bootmenu spl.spl_hostid=0x54c1c9d6
initrd ($root)/zbm/initramfs-bootmenu.img
}
--> one can choose between UUID or path variante for "set root"; my one prefer the UUID one
--> if the disk with the 1. ESP-UUID is missing than grub shows an error for a few seconds, but after the boot process continous
--> one get "spl.spl_hostid" with command "hostid" and has to add "0x" in front
--> UUIDs of the ESPs one find in the file "/etc/kernel/proxmox-boot-uuids" or with "blkid"
- copy zbm boot files and memtest file to the ESPs; on github exist zbm archives packed with different kernels, so choose what u like; if one wanne create the bios files manually your one can find the process under the next sub headline
cp /boot/memtest86+x64.bin /boot/efi/sda2/cp /boot/memtest86+x64.bin /boot/efi/sdb2/
mkdir /boot/efi/sda2/zbmmkdir /boot/efi/sdb2/zbmwget https://github.com/zbm-dev/zfsbootmenu/releases/download/v3.0.1/zfsbootmenu-release-x86_64-v3.0.1-linux6.12.tar.gztar -xf zfsbootmenu-release-x86_64-v3.0.1-linux6.12.tar.gz -C /boot/efi/sda2/zbm/ --strip-components=1cp /boot/efi/sda2/zbm/* /boot/efi/sdb2/zbm/- recreate grub boot menu
roxmox-boot-tool refreshzbm Bios images erstellen
- install void Linux Distro (cmd version is sufficient)
- void updaten
xbps-install -Su- install zbm
xbps-install -S zfsbootmenu- configure zbm
nano /etc/zfsbootmenu/config.yamlGlobal:
ManageImages: true
BootMountPoint:
Components:
Enabled: true
Versions: false
ImageDir: /mnt
- generate kernel images
generate-zbm- Done. Image files are stored in /mnt. Copy them with scp, sftp or USB to your PVE Host.
6. Configuration steps for autosnapshot necessary 4 both Bios and UEFI boot:
- installation
apt install zfs-auto-snapshot- configuration; my one wants only automatic snapshot after reboot with keep 20 retention; additional to the standard configuration one can create pre/post hooks for apt to have snapshots when install new packets
nano /etc/cron.d/zfs-auto-snapshotchange to:
@reboot root which zfs-auto-snapshot > /dev/null || exit 0 ; zfs-auto-snapshot -k 20 -p Boot_auto-snap rpool/ROOT/pve-1- delete all "zfs-auto-snapshot" files in /etc/cron.hourly..[daily]...
7. Back to proxmox-boot environment from UEFI:
- note UEFI bootnum from zbm and Memetest (1. colum)
efibootmgr- deactivate these nums
efibootmgr -Ab 000Befibootmgr -Ab 000Aaso.
- copy systemd efi file back to standard BOOT
cp /boot/efi/sda2/EFI/systemd/systemd-bootx64.efi /boot/efi/sda2/EFI/BOOT/BOOTX64.EFIcp /boot/efi/sdb2/EFI/systemd/systemd-bootx64.efi /boot/efi/sdb2/EFI/BOOT/BOOTX64.EFI8. Back to proxmox-boot environment from Bios:
- edit grub config
nano /etc/default/grubGRUB_DEFAULT=0
- recreate grub menu
roxmox-boot-tool refresh9. Sources
Last edited:
