Installed PVE on a single ZFS SSD. How to add a second SSD as mirror?

[dakazze]

I am running PVE on bare metal and installed it on a single ZFS SSD because the delivery of the second SSD was delayed. Now I finally got the second SSD and installed it.
Since I already set up all my VMs and stuff I am scared to lose all of my progress by mistake. I found a few posts on how to add a second drive to the boot pool but some of them are older and might be outdated and on top of that I am a complete noob when it comes to all of this.
So I would really appreciate if anyone could explain how to do this, the easiest and safest way!

 

[waltar]

Look at "Changing a failed device" in https://pve.proxmox.com/wiki/ZFS_on_Linux#_zfs_administration , you need
# sgdisk <healthy bootable device> -R <new device>
# sgdisk -G <new device>
# proxmox-boot-tool format <new disk's ESP>
# proxmox-boot-tool init <new disk's ESP> [grub]
# zpool attach rpool /dev/disk/by-id/<id_of_running_os_ssd> /dev/disk/by-id/<id_of_new_ssd_for_mirror>

 

[dakazze]

Thanks a lot guys!

@Joshua44 yea, thats exactly why I was asking here. In one of the posts I found, the OP wrote that there was something wrong with the official documentation and he lost all of his data ^^
I already BUed the VMs and for proxmox it should be enough to copy /etc right?

@waltar thank you very much!
Just to be 100% sure because of the issue I mentioned above:
# sgdisk <my old working SSD> -R <my new unused SSD>
so in my case with the already running install on NVMe and the new SSD:
# sgdisk /dev/nvme0n1 -R /dev/sda


and I can do this from the webinterface-shell during normal operation or should I hook up a screen and a keyboard to do it in "safe mode"?

 

[dakazze]

Okay I tried my luck and now I am facing this issue:

root@proxmox:/etc# sgdisk /dev/nvme0n1 -R /dev/sda
Caution! Secondary header was placed beyond the disk's limits! Moving the
header, but other problems may occur!

Warning! Secondary partition table overlaps the last partition by
62512128 blocks!
You will need to delete this partition or resize it in another utility.

Problem: partition 3 is too big for the disk.
Aborting write operation!
Aborting write of new partition table.

I have searched google and this issue seems to come up a lot. Most posts are even marked as "solved" but please stick with me, I tried but I cant make sense of the solutions posted....
It looks like the first SSD is bigger by around 30GB. I would not mind losing 30gig because I dont use that much space anyway on my boot drive. Is there any chance I can resize the first SSD to make it fit?

Device           Start        End   Sectors   Size Type
/dev/nvme0n1p1      34       2047      2014  1007K BIOS boot
/dev/nvme0n1p2    2048    2099199   2097152     1G EFI System
/dev/nvme0n1p3 2099200 1000215182 998115983 475.9G Solaris /usr & Apple ZFS

Or should I just reinstall the whole thing? Is there an easy way to BU and restore the hypervisor config or is it just manually backing up /etc for the CM config files?

 

[LnxBil]

It looks like the first SSD is bigger by around 30GB. I would not mind losing 30gig because I dont use that much space anyway on my boot drive. Is there any chance I can resize the first SSD to make it fit?

Good question, I only increased the size of vdevs, yet never reduced.
The easiest way for most people would be to backup everything to an external disk or mount, reinstall as you like and reinstall PVe again. Yet I do not know if you can install on two different sized disks. That is something you would normally do not do. Nevertheless, you can reinstall on the smaller SSD and add the bigger SSD afterwards.

 

[dakazze]

Nevertheless, you can reinstall on the smaller SSD and add the bigger SSD afterwards.

Haha I would have never thought about that, thank you!!

So I guess reinstall it is then -.-

 

[waltar]

But if doing a new installation (after saving vm/lxc images and /etc/pve/ on other host) you can even do immedently the zfs mirror ... but otherwise the single disk inst and afterwards the extension to mirror is even nice unusal way to go so why not (easy is normal, the complicated is the interesting one)