Config for better performance and backups (ZFS to VM)

[jaykavathe]

I have secondary NVME disk (A) for VM install and a 50TB RaidZ1 ZFS pool with 6 drives in my proxmox host (beside OS disk and ISO storage).
To create nextcloud VM or Alpine-portainer, what is recommended way if I need my ZFS pool only for nextcloud and nothing else?

Option 1) Create ZFS on host, (and then create directory on host using this zfs pool? Do I need to do this?) and when building VM, specify ZFS pool/directory as available 50TB space for whole Nextcloud VM.

Option 2) Create ZFS on host, create VM with 40GB space on VM install NVME and "Add disk" to add secondary disc (I assume I add my 50TB zfs pool or directory here).

Option 3) Pass on my 6 disks to VM during creation and let VM manage zpool?

Option 4) Something better than this? I dont see the need for samba share as this ZFS will be exclusive to single nextcloud VM.

Q1 - What is better strategy? Since I am planning to do another backup on a separate remote NAS, I am not hugely worried about Raid stategy. Is Z1 better? Also would this 50TB ZFS pool would make my backup slower that it will backup whole 50TB even if its blank?

Q2 - I read that you can have a flexible pool where you can keep feeding storage to VM as needed. I dont have primarily need for this unless this makes backup faster.

 

[Stefan_R]

If you want to use PVE's backup, snapshot or migration features, you will need to create the ZFS on the host. You do not need to create a directory (or directory storage) for the ZFS, PVE will use zvol assignments which are automatically created when you assign them to the VM. You can assign the VM all 50 TB immediately, as you said, assignments are "flexible" (thin-provisioned), so only storage actually used by the VM will be allocated - this does *not* cost any performance in ZFS.

RAID Z1 can be an issue with consumer drives, as resilvering can take a toll on them, and while a drive is missing there is no redundancy, so no guarantee about data correctness. This might not be an issue for you, but is something to look out for - there's a lot of information about the different RAIDZ levels out there, also consider a simple striped mirror layout.

If you want to put your VM on the ZFS entirely or put its boot drive on your SSD depends only on your performance needs. Giving the VM space on the SSD will most likely make it much faster, but of course adds some administrative burden.

 

[jaykavathe]

If you want to use PVE's backup, snapshot or migration features, you will need to create the ZFS on the host. You do not need to create a directory (or directory storage) for the ZFS, PVE will use zvol assignments which are automatically created when you assign them to the VM. You can assign the VM all 50 TB immediately, as you said, assignments are "flexible" (thin-provisioned), so only storage actually used by the VM will be allocated - this does *not* cost any performance in ZFS.

RAID Z1 can be an issue with consumer drives, as resilvering can take a toll on them, and while a drive is missing there is no redundancy, so no guarantee about data correctness. This might not be an issue for you, but is something to look out for - there's a lot of information about the different RAIDZ levels out there, also consider a simple striped mirror layout.

If you want to put your VM on the ZFS entirely or put its boot drive on your SSD depends only on your performance needs. Giving the VM space on the SSD will most likely make it much faster, but of course adds some administrative burden.

Perfect, you did cover all my questions in the way I understood it.. thank you

One quick question though, when I create zpool on host, I don't see that share available to select when vm creation. I went to "Datacenter" and added ZFS manually there with blocksize of 64K and was able to install VM on it. Is that the same thing as zvol assignment?

 

[Stefan_R]

One quick question though, when I create zpool on host, I don't see that share available to select when vm creation. I went to "Datacenter" and added ZFS manually there with blocksize of 64K and was able to install VM on it. Is that the same thing as zvol assignment?

If you create the zpool manually, PVE can't know about it (or, well, we deliberately ignore it, since it might not be related to PVE at all). You can either create the zpool via our GUI (Node -> Disks -> ZFS), or add it manually, as you have done. See our storage documentation for more on that: https://pve.proxmox.com/pve-docs/chapter-pvesm.html

It is not the same as what I've referred to as "zvol assigment" above (honestly a bit of a misnomer there). What I meant was that the zvol gets created automatically when you add a storage to a VM (or create a new VM). You can see the created zvol with 'zfs list' or by looking at the "Content" tab on your zpool storage on the PVE GUI.