Using dd to clone a Proxmox main/boot disk

[Morgan]

I want to move Proxmox Virtual Environment 5.4-13 from a hard disk to a slightly larger SSD.

I tried booting into a live cd and using dd to copy the entire disk / block device to the other disk.

dd if=/dev/sda of=/dev/sdb bs=64M

This seems to work okay. All the partitions enumerate correctly and seem intact on the new SSD.

However the bootloader on proxmox is somewhat unusual and I cannot get grub (which was copied over as part of dd) to find the boot info anymore. I get the "grub unknown filesystem" error. This is somewhat expected because the identity of the disk has changed. I'm also not entirely sure if all of the grub information was copied over.

I expect to be able to recover from this by temporarily telling grub which partition /boot and / (root) is located on, then booting in and reinstalling grub. The usual workaround from within grub rescue is:

set root=(hd0,6)
set prefix=(hd0,6)/boot/grub
insmod normal
normal

However all of the filesystems also give the unknown filesystem error.

grub rescue> ls
(lvm/pve-root) (lvm/pve-swap) (hd0) (hd0,gpt3) (hd0,gpt2) (hd0,gpt1)

Listing any of these filesystems or trying to run "insmod normal" returns an unknown filesystem error.

List of partitions on the disk including lvm:

root@olddisk:~# lsblk
#other disks omitted / irrelevant
sdl                            8:176  0 465.8G  0 disk
├─sdl1                         8:177  0     1M  0 part
├─sdl2                         8:178  0   256M  0 part /boot/efi
└─sdl3                         8:179  0 465.5G  0 part
  ├─pve-swap                 253:0    0     8G  0 lvm  [SWAP]
  ├─pve-root                 253:1    0    96G  0 lvm  /
  ├─pve-data_tmeta           253:9    0   3.5G  0 lvm
  │ └─pve-data-tpool         253:11   0 338.6G  0 lvm
  │   ├─pve-data             253:12   0 338.6G  0 lvm
  │   ├─pve-vm--100--disk--1 253:13   0    50G  0 lvm
  │   ├─pve-vm--101--disk--1 253:14   0    50G  0 lvm
  │   ├─pve-vm--101--disk--2 253:15   0    32G  0 lvm
  │   ├─pve-vm--102--disk--1 253:16   0   128G  0 lvm
  │   └─pve-vm--103--disk--1 253:17   0   128G  0 lvm
  └─pve-data_tdata           253:10   0 338.6G  0 lvm
    └─pve-data-tpool         253:11   0 338.6G  0 lvm
      ├─pve-data             253:12   0 338.6G  0 lvm
      ├─pve-vm--100--disk--1 253:13   0    50G  0 lvm
      ├─pve-vm--101--disk--1 253:14   0    50G  0 lvm
      ├─pve-vm--101--disk--2 253:15   0    32G  0 lvm
      ├─pve-vm--102--disk--1 253:16   0   128G  0 lvm
      └─pve-vm--103--disk--1 253:17   0   128G  0 lvm

Is there a special process for fixing grub for Proxmox or for LVM based root?

 

[fabian]

I notice your disk is enumerated as /dev/sdl - is it possible that it is not passed along as bootable by the bios and Grub does not see it? did you remove the original disk (LVM does not like duplicate VG names, not sure how Grub would handle it)?

 

[Morgan]

I notice your disk is enumerated as /dev/sdl - is it possible that it is not passed along as bootable by the bios and Grub does not see it? did you remove the original disk (LVM does not like duplicate VG names, not sure how Grub would handle it)?

When I try to boot the SSD, the only disk in the system is the SSD. I physically ejected all the other disks cause I didn't want to worry about them when trying to get into early linux boot. e.g. /dev/sda thru /dev/sdk are removed. The HDD is obviously not inserted; the SSD is used in its place.

The properties of the partitions are identical on the SSD and HDD. This I already checked after cloning the disks.

root@olddisk:~# fdisk -l /dev/sdl
Disk /dev/sdl: 465.8 GiB, 500107862016 bytes, 976773168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 13EA940D-2372-4033-8AFD-0DBD83E2615A

Device      Start       End   Sectors   Size Type
/dev/sdl1    2048      4095      2048     1M BIOS boot
/dev/sdl2    4096    528383    524288   256M EFI System
/dev/sdl3  528384 976773134 976244751 465.5G Linux LVM


root@olddisk:~# parted /dev/sdl
GNU Parted 3.2
Using /dev/sdl
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p
Model: ATA Hitachi HDS72105 (scsi)
Disk /dev/sdl: 500GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name  Flags
1      1049kB  2097kB  1049kB                     bios_grub
2      2097kB  271MB   268MB   fat32              boot, esp
3      271MB   500GB   500GB                      lvm

As you can see, the boot flag is set on the second partition.

 

[fabian]

okay - what does ls (lvm0,1) and ls (lvm0,1)/ and the same for 0,2 and 0,3 output in the grub shell?

set root=(hd0,6)
set prefix=(hd0,6)/boot/grub
insmod normal
normal

can't work, since there is no 6th partition on hd0

 

[Morgan]

All possibilities for 'ls' on all partitions and all partitions plus a slash return "unknown filesystem"

Also, the options are actually (not reciting from memory this time)

grub rescue> ls
(lvm/pve-root) (lvm/pve-swap) (hd0) (hd0,gpt3) (hd0,gpt2) (hd0,gpt1)

And I did use real disk names when running the commands...(hd0,6) is a copy paste from other instructions.

 

[fabian]

next step would be to boot from a livecd, activate the LVs, mount the root LV somewhere, bind mount /sys /proc and /dev into the mounted root LV into their respective paths, then chroot into the mounted root LV and run

update-grub
grub-install /dev/PATH-OF-YOUR-DISK

basically, the procedure from https://howtoubuntu.org/how-to-repair-restore-reinstall-grub-2-with-a-ubuntu-live-cd but with our root filesystem on an LV, not directly in a partition.

 

[Morgan]

Thanks. This worked.

That's a much better set of instructions for using the chroot method than I was able to find elsewhere online.