Transferring boot copies to other NVMe disks

[luison]

Hi, we are having issues with a local server running various NVMe drives and SSDs. The UEFI boot partitions were installed originally to the SSD drives that have been superseeded now by NVMe disks.
We just realised that removing the SSD drive leaves the system unbootable as booting from any of the other NVMe drives returns a grub error.

We are hoping to clone all 3 NVM disks to be able to boot when selected from BIOS in case of a failure of the others.

The main current boot SSD drive is like this:
gdisk -l /dev/sda
GPT fdisk (gdisk) version 1.0.6

Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sda: 1953525168 sectors, 931.5 GiB
Model: CT1000BX500SSD1
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): DB696B42-E5B8-48A3-973A-C4D4FF8354C3
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 1953525134
Partitions will be aligned on 2048-sector boundaries
Total free space is 128365 sectors (62.7 MiB)

Number Start (sector) End (sector) Size Code Name
1 2048 4095 1024.0 KiB EF02 biosboot_ssd0
2 1052672 1251053567 596.0 GiB FD00 data_ssd0
3 1251053568 1893631999 306.4 GiB 8E00 data2_ssd0
4 1893632000 1953398783 28.5 GiB FD00 SINUSO
5 4096 1052671 512.0 MiB EF00 boot_ssd1

NVMe disks is:
Disk /dev/nvme0n1: 1953525168 sectors, 931.5 GiB
Model: Force MP600
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): C336E644-7F5A-4EB0-9A3D-0268AE5EEE41
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 1953525134
Partitions will be aligned on 2048-sector boundaries
Total free space is 19973485 sectors (9.5 GiB)

Number Start (sector) End (sector) Size Code Name
1 2048 4095 1024.0 KiB 8300
2 1052672 1251053567 596.0 GiB 8300 lvm
3 1251053568 1893631999 306.4 GiB FD00 data2_nvm0
4 1893646336 1933567999 19.0 GiB 8300 nvme0n1p6_lvm2
5 4096 1052671 512.0 MiB 8300

So identical partitions for grub init and UEFI boot which cloned at the time from the main SSD one.


After rereading now the documentation, I understood that

proxmox-boot-tool init /dev/nvme0n1p5

would solve it but ir returns the error:
E: '/dev/nvme0n1p5' has wrong partition type (!= c12a7328-f81f-11d2-ba4b-00a0c93ec93b).

So we tried to format it with
proxmox-boot-tool format /dev/nvme0n1p5

But same result.
I understand this tool replaces "grub-install" so not sure how to proceed.

status is:
proxmox-boot-tool status
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
System currently booted with uefi
2C6E-A8ED is configured with: uefi (versions: 5.15.136-1-pve, 5.15.143-1-pve, 5.3.10-1-pve, 5.4.203-1-pve, grubx64.efi)


Any help would be appreciated.

 

[luison]

Some additional clarifications to my previous question, which we still can't figure out.

• pve-manager/7.4-17/513c62be (running kernel: 5.15.143-1-pve)
• this is not a ZFS system as it was upgraded from previous installations, I guess will have to move to ZFS on next upgrade
• I've checked the partition types and corrected one of the types, still same issue

Number Start (sector) End (sector) Size Code Name
1 2048 4095 1024.0 KiB EF02
2 1052672 1251053567 596.0 GiB 8300 lvm
3 1251053568 1893631999 306.4 GiB FD00 data2_nvm0
4 1893646336 1933567999 19.0 GiB 8300 nvme0n1p6_lvm2
5 4096 1052671 512.0 MiB EF00

As for my understanding, I should be able to use:

proxmox-boot-tool init /dev/nvme0n1p5

on that drive to make it also bootable.

We also tried as backup to install to an external usb drive... with a different error

Number  Start (sector)    End (sector)  Size       Code  Name
   1              48            2048   1000.5 KiB  EF02  BIOS boot partition
   2            2064         1048576   511.0 MiB   EF00  EFI system partition

root@e20home:/home/luison# proxmox-boot-tool format /dev/sdd2
UUID="" SIZE="535814656" FSTYPE="" PARTTYPE="c12a7328-f81f-11d2-ba4b-00a0c93ec93b" PKNAME="sdd" MOUNTPOINT=""
Formatting '/dev/sdd2' as vfat..
mkfs.fat 4.2 (2021-01-31)
Done.

root@e20home:/home/luison# proxmox-boot-tool init /dev/sdd2
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
UUID="3DAB-78C2" SIZE="535814656" FSTYPE="vfat" PARTTYPE="c12a7328-f81f-11d2-ba4b-00a0c93ec93b" PKNAME="sdd" MOUNTPOINT=""
Mounting '/dev/sdd2' on '/var/tmp/espmounts/3DAB-78C2'.
Installing systemd-boot..
Created "/var/tmp/espmounts/3DAB-78C2/EFI/systemd".
Created "/var/tmp/espmounts/3DAB-78C2/EFI/BOOT".
Created "/var/tmp/espmounts/3DAB-78C2/loader".
Created "/var/tmp/espmounts/3DAB-78C2/loader/entries".
Created "/var/tmp/espmounts/3DAB-78C2/EFI/Linux".
Copied "/usr/lib/systemd/boot/efi/systemd-bootx64.efi" to "/var/tmp/espmounts/3DAB-78C2/EFI/systemd/systemd-bootx64.efi".
Copied "/usr/lib/systemd/boot/efi/systemd-bootx64.efi" to "/var/tmp/espmounts/3DAB-78C2/EFI/BOOT/BOOTX64.EFI".
Random seed file /var/tmp/espmounts/3DAB-78C2/loader/random-seed successfully written (512 bytes).
Failed to create EFI Boot variable entry: Invalid argument

 

[luison]

Answering my on question (sort of!) the issue got solved partially at least but not sure how. Maybe rechecking the flags of the "clone" partitions on gdisk or after a reboot, suddenly allowed us to correctly use proxmox-boot-tool init and now the 3 partitions are correctly been updating when system updates, but still have other issues that prevent booting when I remove one of the drives. I am basically trying that one disk failure does not prevent the system from booting.

1.- EFI MOUNT ON FSTAB
I still have another issue with the fstab mount for "the" efi partition. I understand that there is no raid involved or possible here and only one can be mounted in fstab which in my case is /dev/sda. So the initial part of my fstab is:

## BOOT
UUID=2C6E-A8ED /boot/efi vfat defaults 0 1

## SYSTEM
proc /proc proc defaults 0 0
/dev/pve/root / ext4 errors=remount-ro 0 1

In my understanding this mount is only relevant for system updates which then proxmox-boot-tool would copy to the other mirrors. Being so, I've added a "nofail" to that mount to prevent a boot error in case of that disks failure which I think works.

• Can someone confirm if this is correct? (UUID=2C6E-A8ED /boot/efi vfat defaults,nofail 0 1)
• As /boot is in / in my case, does the order of loading it affect? Should the /boot/efi be defined after?

2.- ADDING our own boot options to EFI boot menu howto
We had some personalized boot options for rescue disks configured before when using GRUB, I was expecting that if recovered those including them in /etc/grub.d/40_custom, these would also appear on the EFI boot menus. I can now tell they don't. In this case, how do I add say a local rescue partition or ISO to this menu?

Thanks for any help.