ZFS VM Storage upgrade process

[justanton]

hey all, I currently have pair of 256gb NVME ssds mirrored in ZFS. This was done through GUI. This pool is used for VMS and running out of space.

I need to now upgrade it to a pair of larger drives. I found this article as advice on how to do single drive upgrade:
http://www.microhowto.info/howto/replace_one_of_the_physical_volumes_in_an_lvm_volume_group.html

Can someone point me the right direction on how to do this with mirroring the drives too? What should I do to upgrade from existing two drives to two larger ones while keeping mirror for redundancy.

 

[Dunuin]

hey all, I currently have pair of 256gb NVME ssds mirrored in ZFS. This was done through GUI. This pool is used for VMS and running out of space.

Does PVE boots of that pool or is it for storing guests only?

I need to now upgrade it to a pair of larger drives. I found this article as advice on how to do single drive upgrade:
http://www.microhowto.info/howto/replace_one_of_the_physical_volumes_in_an_lvm_volume_group.html

That is for LVM not ZFS.

Can someone point me the right direction on how to do this with mirroring the drives too? What should I do to upgrade from existing two drives to two larger ones while keeping mirror for redundancy.

Do you have two free drive slots and ports? If yes I would just buy 2 more drives and add them so you got a striped mirror (like raid10) which would increase the capacity and performance while still keeping redundancy.

If you don't got free drive slots or ports you could try to replace them like discussed there: https://forum.proxmox.com/threads/grow-individual-mirrored-vdev-in-zpool.105569/#post-454544

 

[justanton]

Does PVE boots of that pool or is it for storing guests only?

That is for LVM not ZFS.

Do you have two free drive slots and ports? If yes I would just buy 2 more drives and add them so you got a striped mirror (like raid10) which would increase the capacity and performance while still keeping redundancy.

If you don't got free drive slots or ports you could try to replace them like discussed there: https://forum.proxmox.com/threads/grow-individual-mirrored-vdev-in-zpool.105569/#post-454544

Thanks for coming back to me so quickly.

It is for storing guests only.

I have spare ports, but I need to do the replacement as I have a cluster set-up and need them to be consistent as they use same naming for the pools.

 

[Dunuin]

Thanks for coming back to me so quickly.

It is for storing guests only.

I have spare ports, but I need to do the replacement as I have a cluster set-up and need them to be consistent as they use same naming for the pools.

Better ask for a second opinion before buying stuff, but far as I heard, it should be possible to turn a mirror into a striped mirror on the fly without rebooting, needing to destroy the pool or taking it offline.

 

[justanton]

Better ask for a second opinion before buying stuff, but far as I heard, it should be possible to turn a mirror into a striped mirror on the fly without rebooting, needing to destroy the pool or taking it offline.

Oh I've already got the drives, just looking to do the upgrade. no problem on turning off the machine by the way. I've done pool drive changes in ZFS(freenas) before by just removing drives and resilvering just like that link you sent. It should work the same way then?

 

[LnxBil]

It should work the same way then?

Yes

but far as I heard, it should be possible to turn a mirror into a striped mirror on the fly without rebooting, needing to destroy the pool or taking it offline.

You heard right

root@dyn-043 ~ > zpool list -v
NAME                                       SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
testpool                                  4.75G   186K  4.75G        -         -     0%     0%  1.00x    ONLINE  -
  mirror                                  4.75G   186K  4.75G        -         -     0%  0.00%      -  ONLINE
    scsi-0QEMU_QEMU_HARDDISK_drive-scsi1      -      -      -        -         -      -      -      -  ONLINE
    scsi-0QEMU_QEMU_HARDDISK_drive-scsi2      -      -      -        -         -      -      -      -  ONLINE

root@dyn-043 ~ > zpool add testpool mirror /dev/sdd /dev/sde

root@dyn-043 ~ > zpool list -v
NAME                                       SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
testpool                                  9.25G  1.15M  9.25G        -         -     0%     0%  1.00x    ONLINE  -
  mirror                                  4.75G  1.15M  4.75G        -         -     0%  0.02%      -  ONLINE
    scsi-0QEMU_QEMU_HARDDISK_drive-scsi1      -      -      -        -         -      -      -      -  ONLINE
    scsi-0QEMU_QEMU_HARDDISK_drive-scsi2      -      -      -        -         -      -      -      -  ONLINE
  mirror                                  4.50G      0  4.50G        -         -     0%  0.00%      -  ONLINE
    sdd                                       -      -      -        -         -      -      -      -  ONLINE
    sde                                       -      -      -        -         -      -      -      -  ONLINE

It is also easy to add redundancy if you want to convert a stripped (even single disk) to mirrored setup.

 

[justanton]

Yes


You heard right

root@dyn-043 ~ > zpool list -v
NAME                                       SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
testpool                                  4.75G   186K  4.75G        -         -     0%     0%  1.00x    ONLINE  -
  mirror                                  4.75G   186K  4.75G        -         -     0%  0.00%      -  ONLINE
    scsi-0QEMU_QEMU_HARDDISK_drive-scsi1      -      -      -        -         -      -      -      -  ONLINE
    scsi-0QEMU_QEMU_HARDDISK_drive-scsi2      -      -      -        -         -      -      -      -  ONLINE

root@dyn-043 ~ > zpool add testpool mirror /dev/sdd /dev/sde

root@dyn-043 ~ > zpool list -v
NAME                                       SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
testpool                                  9.25G  1.15M  9.25G        -         -     0%     0%  1.00x    ONLINE  -
  mirror                                  4.75G  1.15M  4.75G        -         -     0%  0.02%      -  ONLINE
    scsi-0QEMU_QEMU_HARDDISK_drive-scsi1      -      -      -        -         -      -      -      -  ONLINE
    scsi-0QEMU_QEMU_HARDDISK_drive-scsi2      -      -      -        -         -      -      -      -  ONLINE
  mirror                                  4.50G      0  4.50G        -         -     0%  0.00%      -  ONLINE
    sdd                                       -      -      -        -         -      -      -      -  ONLINE
    sde                                       -      -      -        -         -      -      -      -  ONLINE

It is also easy to add redundancy if you want to convert a stripped (even single disk) to mirrored setup.

thanks! will do it right now.

By the way - my system right now is a mirror - should I change anything about it?

 

[Dunuin]

thanks! will do it right now.

By the way - my system right now is a mirror - should I change anything about it?

No, all fine then expanding that 2 disk mirror to a 4 or even 6/8/... disk striped mirror.

 

[justanton]

No, all fine then expanding that 2 disk mirror to a 4 or even 6/8/... disk striped mirror.

noted, thanks.