repair boot (zfs) - proxmox-boot-tool didnt work.

daemonix · Aug 3, 2022

The ASUS bm cant find the boot drive so Im trying to repair the boot manager.
Its a fully updated proxmox with ZFS on the boot drive so RTFM-ing regarding proxmox-boot-tool didnt get me somewhere.
I have attached the screenshot of the error I got when I booted of ubuntu and mounted the main pool (I think... .. Its the first time I use ZFS on the boot disk...)

What am I doing wrong?!?!
Thats the correct way to rebuild boot on a ZFS/systemd proxmox?
Thanks

Dunuin · Aug 3, 2022

Did you tried to write a new systemd/grub bootloader? Partition 1 is for grub and partiton 2 for systemd boot.

See the wiki article for replacing a failed boot disk which also describes how to write a new bootloader: https://pve.proxmox.com/wiki/ZFS_on_Linux#_zfs_administration

With proxmox-boot-tool:
# proxmox-boot-tool format <new disk's ESP>
# proxmox-boot-tool init <new disk's ESP>

ESP stands for EFI System Partition, which is setup as partition #2 on bootable disks setup by the Proxmox VE installer since version 5.4. For details, see Setting up a new partition for use as synced ESP.

With plain grub:
# grub-install <new disk>

plain grub is only used on systems installed with Proxmox VE 6.3 or earlier, which have not been manually migrated to using proxmox-boot-tool yet.

daemonix · Aug 3, 2022

Dunuin said:
Did you tried to write a new systemd/grub bootloader? Partition 1 is for grub and partiton 2 for systemd boot.

See the wiki article for replacing a failed boot disk which also describes how to write a new bootloader: https://pve.proxmox.com/wiki/ZFS_on_Linux#_zfs_administration

Hi,
Sorry I dont want to do anything silly so let me double check!


zpool import -R /mnt -f rpool
zfs mount -a 
mount --rbind /dev /mnt/dev
mount --rbind /proc /mnt/proc
mount --rbind /sys /mnt/sys chroot /mnt

chroot /mnt

And then I run "proxmox-boot-tool init" on /dev/nvm**** (second partition)??

daemonix · Aug 3, 2022

it didnt like it :S

Code:

root@ubuntu:/# proxmox-boot-tool format /dev/nvme0n1p2
UUID="" SIZE="536870912" FSTYPE="" PARTTYPE="" PKNAME="nvme0n1" MOUNTPOINT=""
Setting partition type of '/dev/nvme0n1p2' to 'c12a7328-f81f-11d2-ba4b-00a0c93ec93b'..
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
Calling 'udevadm settle'..
Running in chroot, ignoring request.
Formatting '/dev/nvme0n1p2' as vfat..
mkfs.fat 4.2 (2021-01-31)
Done.
root@ubuntu:/# proxmox-boot-tool init /dev/nvme0n1p2
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
UUID="" SIZE="536870912" FSTYPE="" PARTTYPE="" PKNAME="nvme0n1" MOUNTPOINT=""
E: '/dev/nvme0n1p2' has wrong partition type (!= c12a7328-f81f-11d2-ba4b-00a0c93ec93b).
root@ubuntu:/#

daemonix · Aug 4, 2022

Anyone has an ideas how to get out of this situation?

Partition 2 is correctly formatted and the right EFI type. (re-applied it with fdisk just to be sure...)


Disk /dev/nvme0n1: 3.64 TiB, 4000787030016 bytes, 7814037168 sectors
Disk model: Corsair MP600 PRO XT
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: C3ABADBC-30C1-4C5A-8461-11CD126BA1CD

Device              Start        End    Sectors   Size Type
/dev/nvme0n1p1         34       2047       2014  1007K BIOS boot
/dev/nvme0n1p2       2048    1050623    1048576   512M EFI System
/dev/nvme0n1p3    1050624 1048576000 1047525377 499.5G Solaris /usr & Apple ZFS
/dev/nvme0n1p4 1048578048 7814037134 6765459087   3.2T Solaris /usr & Apple ZFS


root@ubuntu:/# proxmox-boot-tool init /dev/nvme0n1p2
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
UUID="" SIZE="536870912" FSTYPE="" PARTTYPE="" PKNAME="nvme0n1" MOUNTPOINT=""
E: '/dev/nvme0n1p2' has wrong partition type (!= c12a7328-f81f-11d2-ba4b-00a0c93ec93b).

"E: '/dev/nvme0n1p2' has wrong partition type (!= c12a7328-f81f-11d2-ba4b-00a0c93ec93b)."
I cant find anything about this... and "c12a7328-f81f-11d2-ba4b-00a0c93ec93b" seems to be the EFI id.

leesteken · Aug 4, 2022

Maybe try gdisk for drives with a GPT instead of fdisk?

daemonix · Aug 4, 2022

leesteken said:
Maybe try gdisk for drives with a GPT instead of fdisk?

just did but I get the same error..

leesteken · Aug 5, 2022

daemonix said:
just did but I get the same error..

Did you remove and recreate the partition with gdisk and save the changes? I'm just trying to make sure you did what works for me.

daemonix · Aug 5, 2022

leesteken said:
Did you remove and recreate the partition with gdisk and save the changes? I'm just trying to make sure you did what works for me.

nop. Ill try it.

daemonix · Aug 5, 2022

daemonix said:
nop. Ill try it.

Nothing

its crazy...
after recreating the partition I did a couple more tries but no-go...

Code:

ubuntu@ubuntu:~$ blkid
/dev/mapper/pve-root: UUID="d63481d3-2315-4a28-86b2-fbcf90a3be52" BLOCK_SIZE="4096" TYPE="ext4"
/dev/nvme0n1p2: UUID="3911-0CFA" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="EFI system partition" PARTUUID="c0c89a29-1a62-4a7f-843d-1d0771581f81"
/dev/nvme0n1p3: LABEL="rpool" UUID="8656547487326107648" UUID_SUB="2395244843362434488" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="9cad09c1-4a81-4db0-92d4-73783e382b04"
/dev/nvme0n1p4: LABEL="nv4t-data" UUID="11501586213567198848" UUID_SUB="8747858444768344940" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="f9a9221a-af5b-b545-a29c-b3123430c8cc"
/dev/nvme1n1p2: UUID="4D9A-1220" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="d6a861e6-4e16-452d-92b2-f121b15aa1f8"
/dev/sda1: BLOCK_SIZE="2048" UUID="2022-04-19-10-23-19-00" LABEL="Ubuntu 22.04 LTS amd64" TYPE="iso9660" PARTLABEL="ISO9660" PARTUUID="a09db2b8-b5f6-43ae-afb2-91e0a90189a1"

ubuntu@ubuntu:~$ sudo su -
root@ubuntu:~# zpool import -R /mnt -f rpool
root@ubuntu:~# zfs mount -a
root@ubuntu:~# mount --rbind /dev /mnt/dev
root@ubuntu:~# mount --rbind /proc /mnt/proc
root@ubuntu:~# mount --rbind /sys /mnt/sys
root@ubuntu:~# chroot /mnt

root@ubuntu:/# proxmox-boot-tool kernel list
Manually selected kernels:
None.

Automatically selected kernels:
5.15.30-2-pve
5.15.39-1-pve
root@ubuntu:/# proxmox-boot-tool init /dev/nvme0n1p2
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
UUID="" SIZE="536870912" FSTYPE="" PARTTYPE="" PKNAME="nvme0n1" MOUNTPOINT=""
E: '/dev/nvme0n1p2' has wrong partition type (!= c12a7328-f81f-11d2-ba4b-00a0c93ec93b).

###
Expert command (? for help): c
Partition number (1-4): 2
Enter the partition's new unique GUID ('R' to randomize): c12a7328-f81f-11d2-ba4b-00a0c93ec93b
New GUID is C12A7328-F81F-11D2-BA4B-00A0C93EC93B
10.10.0.200 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPmW2ajDUDmyxNgLWbj5qF7AF5aqygOjTtiFLXEBBwY/

Expert command (? for help): w
###
reboot
###
/dev/nvme0n1p2: UUID="3911-0CFA" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="EFI system partition" PARTUUID="c12a7328-f81f-11d2-ba4b-00a0c93ec93b"
/dev/nvme0n1p3: LABEL="rpool" UUID="8656547487326107648" UUID_SUB="2395244843362434488" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="9cad09c1-4a81-4db0-92d4-73783e382b04"
/dev/nvme0n1p4: LABEL="nv4t-data" UUID="11501586213567198848" UUID_SUB="8747858444768344940" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="f9a9221a-af5b-b545-a29c-b3123430c8cc"

###
root@ubuntu:/# proxmox-boot-tool init /dev/nvme0n1p2
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
UUID="" SIZE="536870912" FSTYPE="" PARTTYPE="" PKNAME="nvme0n1" MOUNTPOINT=""
E: '/dev/nvme0n1p2' has wrong partition type (!= c12a7328-f81f-11d2-ba4b-00a0c93ec93b).

leesteken · Aug 5, 2022

Did you only do c with gdisk (as shown in your post)? I really think that actually deleting (d) the second partition and creating a new one (n) with the same start and end sector and type EF00 with gdisk is necessary.

EDIT: Or boot with GParted Live and use their GUI to remove and recreate the partition.

daemonix · Aug 5, 2022

leesteken said:
Did you only do c with gdisk (as shown in your post)? I really think that actually deleting (d) the second partition and creating a new one (n) with the same start and end sector and type EF00 with gdisk is necessary.

EDIT: Or boot with GParted Live and use their GUI to remove and recreate the partition.

sorry, I did a full p2 delete, then "n" (same geometry as before).set EFI id etc. then reboot. then proxmox-boot-tool format. reboot.
init (failed).
gdisk x/c/2 and set id again. reboot. (c in expert more is to change the partid by hand)
init again (failed..)

leesteken · Aug 5, 2022

daemonix said:
sorry, I did a full p2 delete, then "n" (same geometry as before).set EFI id etc. then reboot. then proxmox-boot-tool format. reboot.
init (failed).
gdisk x/c/2 and set id again. reboot. (c in expert more is to change the partid by hand)
init again (failed..)

And gdisk does show the drive as GPT; present (and MBR: protective)?

daemonix · Aug 5, 2022

leesteken said:
And gdisk does show the drive as GPT; present (and MBR: protective)?

yes


root@ubuntu:/# gdisk /dev/nvme0n1
GPT fdisk (gdisk) version 1.0.6

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

escoreal · Apr 10, 2023

For the next one finding this topic, maybe a solution. Worked for me.

Code:

mount --bind /run /mnt/run

shoaib42 · Aug 20, 2023

escoreal said:
For the next one finding this topic, maybe a solution. Worked for me.

Code:

mount --bind /run /mnt/run

Dude you saved me a 4 hour headache!!, This needs to be bubbled up on reddit too!

RolandK · Jan 21, 2024

i have run into this multiple times and this also caused me a lot of pain/hours

you need to have /run mounted into the chroot, as lsblk is a little bit stupid and does not seem to recheck partition type directly from device as manpage tells, when there is no /run/udev/data/ available

i would suggest to add some check/improvement into proxmox-boot-tool to check when parttype is empty if /run/udev/data is available, and if not, proxmox-boot-tool could print a hint "when inside chroot - please mount /run, as lsblk does need that" @Stoiko Ivanov

proxmox-boot-tool checks partition like this:
lsblk --bytes --pairs -o 'UUID,SIZE,FSTYPE,PARTTYPE,PKNAME,MOUNTPOINT' /dev/sda2
UUID="52ED-341F" SIZE="536870912" FSTYPE="vfat" PARTTYPE="c12a7328-f81f-11d2-ba4b-00a0c93ec93b" PKNAME="sda" MOUNTPOINT=""

manpage:
lsblk lists information about all available or the specified block devices. The lsblk command reads the sysfs filesystem and udev db to gather
information. If the udev db
is not available or lsblk is compiled without udev support, then it tries to read LABELs, UUIDs and filesystem types from the block device. In this case root permissions
are necessary.

Search

Search

repair boot (zfs) - proxmox-boot-tool didnt work.

daemonix

Member

Dunuin

Distinguished Member

daemonix

Member

daemonix

Member

daemonix

Member

leesteken

Distinguished Member

daemonix

Member

leesteken

Distinguished Member

daemonix

Member

daemonix

Member

leesteken

Distinguished Member

daemonix

Member

leesteken

Distinguished Member

daemonix

Member

escoreal

Renowned Member

shoaib42

New Member

RolandK

Renowned Member