zpool replace doesnt replace bios and efi

jerrymoushu

New Member
Mar 29, 2021
9
0
1
43
i build a server with 4 disks
first 2 disk raid 1 zfs for proxmox os
the rest 2 disk raid 1 zfs for vm

i have no problem to replace zfs pool for vm however when i do the same for the proxmox os (found both bios and efi) partition missing
any easy way of doing natively? thanks
 
You need to manually partition the new system disk, copy over the ESP/grub and only to a zpool replace on partition 3. See here:

Changing a failed bootable device​

Depending on how Proxmox VE was installed it is either using proxmox-boot-tool [1] or plain grub as bootloader (see Host Bootloader). You can check by running:
# proxmox-boot-tool status
The first steps of copying the partition table, reissuing GUIDs and replacing the ZFS partition are the same. To make the system bootable from the new disk, different steps are needed which depend on the bootloader in use.
# sgdisk <healthy bootable device> -R <new device>
# sgdisk -G <new device>
# zpool replace -f <pool> <old zfs partition> <new zfs partition>
Use the zpool status -v command to monitor how far the resilvering process of the new disk has progressed.
With proxmox-boot-tool:
# proxmox-boot-tool format <new disk's ESP>
# proxmox-boot-tool init <new disk's ESP>
ESP stands for EFI System Partition, which is setup as partition #2 on bootable disks setup by the Proxmox VE installer since version 5.4. For details, see Setting up a new partition for use as synced ESP.
With grub:
# grub-install <new disk>
 
  • Like
Reactions: Windows7ge
in my case, if my raid 1 just degraded - i just have to sgdisk to replace everything over to the new disk and then zpool replace the part-3 (zfs) will do
as i assumed there will be no raid happening on bios and efi re-silvering process happen
 
Yes, there will be no raid for ESP/grub but you still want to copy that over from the healthy disk. Because otherwise your host won't boot anymore if you just resilver the ZFS partition and then the healthy disk fails which is the only one that got a bootloader.
 
almost there but failed to boot up.

proxmox-boot-tool format /dev/sdb2
UUID="52E5-2DB5" SIZE="536870912" FSTYPE="vfat" PARTTYPE="c12a7328-f81f-11d2-ba4b-00a0c93ec93b" PKNAME="sdb" MOUNTPOINT=""
E: '/dev/sdb2' contains a filesystem ('vfat') - exiting (use --force to override)

so i continue with --force

root@pve20:~# proxmox-boot-tool init /dev/sdb2 --force
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
UUID="04C1-EB05" SIZE="536870912" FSTYPE="vfat" PARTTYPE="c12a7328-f81f-11d2-ba4b-00a0c93ec93b" PKNAME="sdb" MOUNTPOINT=""
Mounting '/dev/sdb2' on '/var/tmp/espmounts/04C1-EB05'.
Installing grub i386-pc target..
Installing for i386-pc platform.
Installation finished. No error reported.
Unmounting '/dev/sdb2'.
Adding '/dev/sdb2' to list of synced ESPs..
Refreshing kernels and initrds..
Running hook script 'proxmox-auto-removal'..
Running hook script 'zz-proxmox-boot'..
WARN: /dev/disk/by-uuid/021C-3CC9 does not exist - clean '/etc/kernel/proxmox-boot-uuids'! - skipping
Copying and configuring kernels on /dev/disk/by-uuid/04C1-EB05
Copying kernel 5.11.22-4-pve
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.11.22-4-pve
Found initrd image: /boot/initrd.img-5.11.22-4-pve
done
Copying and configuring kernels on /dev/disk/by-uuid/8DE4-E75E
Copying kernel 5.11.22-4-pve
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.11.22-4-pve
Found initrd image: /boot/initrd.img-5.11.22-4-pve
done
WARN: /dev/disk/by-uuid/8DE5-854A does not exist - clean '/etc/kernel/proxmox-boot-uuids'! - skipping
WARN: /dev/disk/by-uuid/FEC2-58C1 does not exist - clean '/etc/kernel/proxmox-boot-uuids'! - skipping

for reference

Disk /dev/sda: 465.76 GiB, 500107862016 bytes, 976773168 sectors
Disk model: Samsung SSD 850
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 4F3578CD-3197-4143-AF39-1041C86FB2C6

Device Start End Sectors Size Type
/dev/sda1 34 2047 2014 1007K BIOS boot
/dev/sda2 2048 1050623 1048576 512M EFI System
/dev/sda3 1050624 976773134 975722511 465.3G Solaris /usr & Apple ZFS


Disk /dev/sdb: 465.76 GiB, 500107862016 bytes, 976773168 sectors
Disk model: Samsung SSD 860
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 44C6B86D-63AB-4F6D-981B-48FF2C340943

Device Start End Sectors Size Type
/dev/sdb1 34 2047 2014 1007K BIOS boot
/dev/sdb2 2048 1050623 1048576 512M EFI System
/dev/sdb3 1050624 976773134 975722511 465.3G Solaris /usr & Apple ZFS

everything just look fine, not sure which part i missed? could you spare your hands, thanks
 
@Dunuin It might be out of nowhere for you but a year later your post just make my late night and the rest of the week-end a heck of a lot less stressful.

One of my mirrored boot drives failed, luckily not the one with the boot-loader (NVMe so no hot swap). I learned tonight that ZFS really only copies the data partition which is disappointing but your list of commands enabled me to copy and configure the EFI partitions from the original drive over to a pair of replacements by swapping them out one at a time.

Words cannot express my gratitude. I was really preparing for the worst as soon as I discovered this ZFS caveat.
 
  • Like
Reactions: Dunuin

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!