[GUIDE] Proxmox with ZFS2

[AngryAdm]

Hello, this is a non-production friendly guide to how to get proxmox to work with ZFS2.0 and persistent L2ARC cache for those of you who use GPU passthrough and use proxmox as basis for a workstation that is shutdown every day.

Please read it all before you venture down this path.
It can be noted, I have been down this path since RC2 any issues or crashes.

Notes: For Debian 10: it is not required to remove packages prior to install of 2.0 unless a previous version of ZFS was installed on the machine. Read the last command of this guide before continuing.

Note2: If you are using your proxmox host for GPU passthrough it is adviced to set options zfs l2arc_mfuonly = 1 to not fill the cache with all sorts of crap if you are doing backups. It will most likely trash your SSD unless it is enterprise grade SLC within a few weeks.

Note3: This does NOT work for a proxmox or debian install with a root zfs pool!!!!!!!!!

WARNING: NOT SAFE FOR PRODUCTION

Here are the commands to enter in order for persistent l2arc to work with proxmox (debian based)

apt-get update

apt-get upgrade

apt-get dist-upgrade

apt-get install build-essential autoconf automake libtool gawk alien fakeroot dkms libblkid-dev uuid-dev libudev-dev libssl-dev zlib1g-dev libaio-dev libattr1-dev libelf-dev pve-headers python3 python3-dev python3-setuptools python3-cffi libffi-dev git

Note: Beware of apt-get upgrade upgrading ZFS modules... If upgrade does upgrade ZFS modules it is adviced to reinstall the packages custom to ZFS 2.0 BEFORE rebooting! This also applies to Proxmox kernel upgrades.

Enjoy!

git clone https://github.com/zfsonlinux/zfs

cd zfs

git checkout zfs-2.0.0-rc7

sh autogen.sh

./configure

make deb

apt-get remove zfsutils-linux

rm kmod-zfs-devel_2.0.0-0_amd64.deb

dpkg -i *.deb

apt-get install zfsutils-linux

Last command removes some of the 2.0 installed packages but without this the pool wont mount on reboot. It does not limit the functionality of ZFS2 to my knowledge. Press Y
Read this for reason: https://github.com/openzfs/zfs/issues/10941

 

[michaelvv]

I did the exact things but it hangs on reboot.

1) I booted into an older kernel.
2) I did a apt update , apt upgrade
3) I can now boot on 5.4.78-2-pve
4) I extracted the files from zfs_2.0.0-1_amd64.deb and put them into a new folder /zfs20
5) /zfs20/zpool upgrade rpool , upgraded my rpool to zfs 2.0 features.

Just did a reboot and everything works fine , thanks for your guide...

 

[Taylan]

Can you report any negative behaviour? It seems you are testing the new version for 4 weeks now.

 

[michaelvv]

Hi.

I have been testing like crazy for 2 days , not a long time , nothing negative so far.

I'm only running 6 LXC Containers but 2 of them a very cpu intense , always using 15% Cpu.

Hardware , Desktop grade , Amd Ryzen 3600 , 32 GB Memory , 1 TB Nvme + 14 TB HD , 2 Intel Gb LAN.

Have done 2 full ZFS Send backups to External server using sanoid , syncoid software , while destroy / restore / backup
LXC Containers , mean while running scrub , having loadavg over +14 ...

Have set compression = zstd on my LXC , no dedup , will try with dedup after a longer testing period.

I also have e Ubuntu kernel 5.9.12 with ZFS 2.0 receiving the backup with old hardware , 2 TB HD 12 years old , 32 Gb Memory old Intel i3.
Here I can for sure say that ZFS 2.0 is performing much better , had problem with ZFS 0.8.3 when receiving Backups from multiple PC's.

In my experience ZFS 2.0 is truly a significant upgrade , on my old PC with a never kernel 5.9.12 just amazing so far...

 

[AngryAdm]

Can you report any negative behaviour? It seems you are testing the new version for 4 weeks now.

I've run ZFS2 since RC2 with no negative behaviour except hangs when removing cache or log devices if the pool is in use.
Workaround: Prevent whatever using the pool. Do a reboot remove the devices when pool is not being accessed by "whatever"

 

[AngryAdm]

I just retried my method and I do not get hanging

I did the exact things but it hangs on reboot.

1) I booted into an older kernel.
2) I did a apt update , apt upgrade
3) I can now boot on 5.4.78-2-pve
4) I extracted the files from zfs_2.0.0-1_amd64.deb and put them into a new folder /zfs20
5) /zfs20/zpool upgrade rpool , upgraded my rpool to zfs 2.0 features.

Just did a reboot and everything works fine , thanks for your guide...

I just tried my guide to verify your findings on a different system, following it to the letter. After a reboot I did not encounter a hang.
I did not have to copy stuff to /zfs20 it just worked
Not sure what was different about your system