Expanding ZFS mirrors

[aarcane]

So I'm migrating my cluster to something a little smaller, and moving from Ceph cluster down to single node with ZFS. One major feature I like from Ceph is that when I add a new drive or twelve, I get data automatically rebalanced across the cluster.

Part of my goal is to spend minimal money now, with some growth options going forward to save on power bill going forward (Summer was brutally expensive!), so I can't justify buying *all* the ZFS VM pool drives right now. I want to add mirror pairs later, and start with a mirror pair of two 1TB drives that I have available for the purpose.

So here's the thing, ZFS doesn't natively support data rebalancing. This is generally a good thing, as it relates to how ZFS stores data on disk in an effectively immutable way to detect write errors. This isn't great for expansion, and the old way to rebalance data was basically to take a data set offline, and then send and receive it into the same pool, or backup and restore it. This is a bit tedious, and as you know with VMs, can cause moderate downtime, which may not be ideal.

Does Proxmox 6 or 7 have any built in configurables or features or tools to rebalance my ZFS pools when I add additional vdevs, or is this not a feature currently implemented?

 

[aarcane]

So I've discovered one potential workaround solution, which is not ideal, but is suitable. if you go to a VM's "Hardware" tab and click on one of the virtual drives, you can "Move disk" up top. Using this to move the disks from the ZFS VM storage pool over to the OS root pool and then back again should effectively rebalance each VM's storage across the new vdev(s) with little to no down time. The two main flaws with this approach are that I have to do it one VM at a time, and that it won't automatically handle any snapshots or other data at rest that proxmox may be taking or keeping behind the scenes