Is ZFS recommended for single node with heavy use of passthrough ?

[nva]

Finally got all the parts for my home server and currently testing. I plan to have SR-IOV passthrough for VM networking, HBA passthrough for storage VM. This means i can't take live snapshots and because it's just one node i don't need replication.

Without using 2 major benefits of ZFS, should i just stick to LVM and ext4? AFAIK, LVM-thin can also take snapshot, just not live snapshots.

By the way, is default LVM on proxmox boot disk LVM-thin? It's single NVMe so VM disks are on same drive with PVE boot. I don't remember having option choosing between LVM and LVM-thin during installation.

 

[Dunuin]

By the way, is default LVM on proxmox boot disk LVM-thin?

Both LV+ext4 for the root filesystem as well as LVM thin pool for guests.

Without using 2 major benefits of ZFS, should i just stick to LVM and ext4? AFAIK, LVM-thin can also take snapshot, just not live snapshots.

Biggest benefit in my opinion is the data integrity with its checksumming of all files and blocks. But without parity (and you don'T plan to use a mirror) you only get bit rot detection and no bit rot protection (so ZFS can't fix corrupted data).

 

[nva]

Both LV+ext4 for the root filesystem as well as LVM thin pool for guests.

Biggest benefit in my opinion is the data integrity with its checksumming of all files and blocks. But without parity (and you don'T plan to use a mirror) you only get bit rot detection and no bit rot protection (so ZFS can't fix corrupted data).

It seems ZFS doesn't have much benefits on my intended settings. But if an proxmox update went wrong it would be great to have the ability to quickly rollback to a snapshot of 'rpool/ROOT/pve-1'.

Just one more question: if i still decide for ZFS on proxmox, how do i avoid write amplification in case a guest (e.g. TrueNAS) really want to have its own CoW filesystem on top virtual disk assigned by Proxmox? AFAIF, zvol is block storage from zfs and CoW on CoW should be avoided.

 

[Dunuin]

You can't avoid that write amplification. You usually don't use TrueNAS but something like OMV that supports simple filesystems like ext4 in case you want to work with virtual disks. Or in case you still want TrueNAS because of ZFS and its features, you usually get dedicated disks for it and pass them (or a whole HBA) through.

 

[gfngfn256]

Just one more question: if i still decide for ZFS on proxmox, how do i avoid write amplification in case a guest (e.g. TrueNAS) really want to have its own CoW filesystem on top virtual disk assigned by Proxmox? AFAIF, zvol is block storage from zfs and CoW on CoW should be avoided.

Since you mentioned earlier that you'll be using "HBA passthrough for storage VM" I don't believe TrueNAS will do much writing to its own OS disk.
Maybe I misunderstood your working scenario.

 

[nva]

You can't avoid that write amplification. You usually don't use TrueNAS but something like OMV that supports simple filesystems like ext4 in case you want to work with virtual disks. Or in case you still want TrueNAS because of ZFS and its features, you usually get dedicated disks for it and pass them (or a whole HBA) through.

Since you mentioned earlier that you'll be using "HBA passthrough for storage VM" I don't believe TrueNAS will do much writing to its own OS disk.
Maybe I misunderstood your working scenario.

Yes i intent to passthrough the HBA for the disk array, but TrueNAS boot disk is planned to be on VM storage because it wouldn't take much space anyway. It appears that TrueNAS also install ZFS on boot disk, so Proxmox VM storage would have to be ext4 to avoid this write amplification.

 

[gfngfn256]

So I did understand correctly - so as I already said I believe TrueNAS doesn't write much to its own System disk. So you can install Zfs On Zfs without trouble.

 

[Dunuin]

Yes i intent to passthrough the HBA for the disk array, but TrueNAS boot disk is planned to be on VM storage because it wouldn't take much space anyway. It appears that TrueNAS also install ZFS on boot disk, so Proxmox VM storage would have to be ext4 to avoid this write amplification.

It isn't writing that much as long as you move TrueNASs "system dataset" to the pool on the HBA. So logs and so on will end up on the HBA disks and not the virtual disk.