Boot & VM data on same zpool or seperate?

[jayg30]

Hello,

I'm wondering what the proxmox teams recommendations are for ZFS. I see proxmox now support UEFI boot and supposedly rpool supports all ZFS feature flags in this configuration.

During installation of Proxmox the "rpool" is generated. The ROOT is there for booting. And rpool/data is created to store your containers and VMs.

Is the recommendation to just expand your rpool and store everything there? Or build a separate zpool so that Proxmox and the boot environment is kept separate?

My experience has always been 2 pools and separation. Then again most of my systems used USB drives for rpool or PXE booted. Proxmox is pretty much a full fat Debian install, and I suspect there is more disk activity because of it (not like FreeNAS where it's installed once and then the OS is read into RAM).

I suspect the argument against separation of the pools would be so you can aggregate all your available disks. You don't lose out on any extra storage (possibly dedicating 2 disks to a RAID1 for the rpool). Possibly Proxmox would be more "responsive" if it can run on a pool with more disks and possibly a SLOG.

And I suspect the argument for separation would be issues with re-installation of the OS. What do I do if I want to perform a fresh install of Proxmox but don't want to wipe my pool with all my containers and VMs. I don't know how you'd do this with 1 pool to be honest.

OpenZFS Dev Summit from this year had a comment on this topic. It even specifically noted Proxmox saying 1 pool with feature flags enabled is "where we want to be".
https://youtu.be/Rc-9QmUlDXs?t=290

So I'm curious to hear what others, especially the Proxmox team, have to say.

 

[LnxBil]

The ROOT is there for booting.

Technically, UEFI is there for booting, the ROOT holds the root filesystem which is read after the booting.

Is the recommendation to just expand your rpool and store everything there?

The default is to use all available space in ZFS, so no expansion necessary. Maybe you set a lower limit?

Or build a separate zpool so that Proxmox and the boot environment is kept separate?

It's already separated in a ZFS dataset. You can set quota on it in order to reserve blocks for it, so that a full pool cannot mess with your installation. That is IMHO the only risk you have in a shared setup.

Proxmox is pretty much a full fat Debian install, and I suspect there is more disk activity because of it (not like FreeNAS where it's installed once and then the OS is read into RAM).

Unfortunately, yes.

My experience has always been 2 pools and separation.

There are people that also favor this, I do not and most of other ZFS users also do not. The reason behind ZFS war that you only need one pool with all your disks so that you have the most speed. Nowadays, you often have to use separate pools if you have SSD and HDD. You cannot combine them for storage (technically yes, but it does not make sense to split data between SSD and HDD performance-wise).