Linux Kernel 5.13, ZFS 2.1 for Proxmox VE

t.lamprecht

Proxmox Staff Member
Staff member
Jul 28, 2015
5,176
1,491
164
South Tyrol/Italy
shop.proxmox.com
The upcoming Proxmox VE 7.1 (Q4/2021) will use a 5.13 based Linux kernel including ZFS 2.1. You can test this kernel now, install with:

Code:
apt update && apt install pve-kernel-5.13

This version will replace the currently used 5.11 kernel and will be maintained until the Proxmox VE 7.2 release.
The kernel is available on the pve-no-subscription repository at time of writing, so it's not required to enable the pvetest repository.

We invite you to test your hardware with this kernel, and we are thankful for receiving your feedback (please include HW & setup details, especially on issues).

A note regarding ZFS versions:
ZFS is separated in user space tools and kernel module, both have a separate version. The user space tooling is backward compatible to older ZFS kernel module versions, so it's fine if you get 2.1 based user space tools but still use the PVE 5.11 kernel (which is build with ZFS 2.0). You can check both versions with zfs -V, for example:

ZFS 2.1 user space, 5.11 kernel with 2.0 ZFS module:
Code:
zfs -V
zfs-2.1.1-pve1
zfs-kmod-2.0.6-pve1

ZFS 2.1 user space, 5.13 kernel with 2.1 ZFS module:
Code:
zfs -V
zfs-2.1.1-pve1
zfs-kmod-2.1.1-pve1
 
Last edited:

vesalius

Active Member
Aug 19, 2020
121
29
28
A bit off topic, but I am seeing ZFS 2.1.1 as an upgrade on PVE 7.0 from the no-subscription repository with the stock 5.11 kernel. I never saw 2.0.6. Is that expected? or does this new kernel upgrade come with the zfs-kmod-2.0.6-pve1 and ZFS-2.1.1 user space?

1636460877931.png
 
Last edited:

t.lamprecht

Proxmox Staff Member
Staff member
Jul 28, 2015
5,176
1,491
164
South Tyrol/Italy
shop.proxmox.com
A bit off topic, but I am seeing ZFS 2.1.1 as an upgrade on PVE 7.0 from the no-subscription repository with the stock 5.11 kernel. I never saw 2.0.6. Is that expected? or does this new kernel upgrade come with the zfs-kmod-2.0.6-pve1 and ZFS-2.1.1 user space?
As said:
ZFS is separated in user space tools and kernel module, both have a separate version. The user space tooling is backward compatible to older ZFS kernel module versions, so it's fine if you get 2.1 based user space tools but still use the PVE 5.11 kernel (which is build with ZFS 2.0). You can check both versions with zfs -V, for example:
 
  • Like
Reactions: vesalius

t.lamprecht

Proxmox Staff Member
Staff member
Jul 28, 2015
5,176
1,491
164
South Tyrol/Italy
shop.proxmox.com

olidal

New Member
Mar 26, 2021
6
1
3
The upcoming Proxmox VE 7.1 (Q4/2021) will use a 5.13 based Linux kernel including ZFS 2.1. You can test this kernel now, install with:

Code:
apt update && apt install pve-kernel-5.13

This version will replace the currently used 5.11 kernel and will be maintained until the Proxmox VE 7.2 release.
The kernel is available on the pve-no-subscription repository at time of writing, so it's not required to enable the pvetest repository.

We invite you to test your hardware with this kernel, and we are thankful for receiving your feedback (please include HW & setup details, especially on issues).

A note regarding ZFS versions:
ZFS is separated in user space tools and kernel module, both have a separate version. The user space tooling is backward compatible to older ZFS kernel module versions, so it's fine if you get 2.1 based user space tools but still use the PVE 5.11 kernel (which is build with ZFS 2.0). You can check both versions with zfs -V, for example:

ZFS 2.1 user space, 5.11 kernel with 2.0 ZFS module:
Code:
zfs -V
zfs-2.1.1-pve1
zfs-kmod-2.0.6-pve1

ZFS 2.1 user space, 5.13 kernel with 2.1 ZFS module:
Code:
zfs -V
zfs-2.1.1-pve1
zfs-kmod-2.1.1-pve1
Thanks for this great news!

Does it come with AUFS module support?
(see my post here: https://forum.proxmox.com/threads/l...ailed-driver-not-supported.97851/#post-429892)

Rationale for supporting AUFS: OVERLAYFS is not compatible with ZFS, but AUFS is, and AUFS also comes with unique additional features (eg. writes can be directed to covered FS instead of all going to upper file system)

Thanks
 

t.lamprecht

Proxmox Staff Member
Staff member
Jul 28, 2015
5,176
1,491
164
South Tyrol/Italy
shop.proxmox.com
Does it come with AUFS module support?
No, I'm afraid not, see https://forum.proxmox.com/threads/kernel-5-11.86225/page-4#post-390667 for a bit more info.

Rationale for supporting AUFS: OVERLAYFS is not compatible with ZFS, but AUFS is, and AUFS also comes with unique additional features (eg. writes can be directed to covered FS instead of all going to upper file system)
The overlayfs situation is not ideal, but, there's some semi-active work going on to support OverlayFS in ZFS, e.g. see this comment for describing current efforts somewhat: https://github.com/openzfs/zfs/pull/9414#issuecomment-939201415
 

olidal

New Member
Mar 26, 2021
6
1
3
No, I'm afraid not, see https://forum.proxmox.com/threads/kernel-5-11.86225/page-4#post-390667 for a bit more info.


The overlayfs situation is not ideal, but, there's some semi-active work going on to support OverlayFS in ZFS, e.g. see this comment for describing current efforts somewhat: https://github.com/openzfs/zfs/pull/9414#issuecomment-939201415
Thank you Thomas.

This is a very bad news on my side as we are developing a Proxmox-based product that depends on some of aufs' unique features.

I guess I am left with the only option of maintaining my own promox kernel variant and release it through our own repo (which does not exist yet). Unless we could find a way to distribute this alternate kernel build through the Proxmox repo?

I would be very happy to share that effort with the Proxmox community, and that would save me quite some efforts trying to maintain a debian repo just for that kernel.

Just let me know,

Kindest regards,
Olivier.
 

t.lamprecht

Proxmox Staff Member
Staff member
Jul 28, 2015
5,176
1,491
164
South Tyrol/Italy
shop.proxmox.com
I guess I am left with the only option of maintaining my own promox kernel variant and release it through our own repo (which does not exist yet).
Hmm, I mean you could go for the dkms or "build-only the module" approach, but you'd need to maintain that yourself - but it may be less work required than rebuilding every kernel (albeit I'm guesstimating this one).

Unless we could find a way to distribute this alternate kernel build through the Proxmox repo?
Well, our repos are not accessible from the public and the signing key is locked down on an HSM.
Contributing the changes required to add aufs (again) and maintain that could be a possibility on the other hand, it would be contained in our kernel package then. But, I'd like to wait out for Q2'22 to make a decision for accepting such a thing:
  • as Proxmox VE 6.4 is still supported till 2022-07 and it has still aufs support (so you could use that in the meantime)
  • ZFS may get overlayfs support until then (knocks wood), making such extra "maintain-aufs" effort superfluous
 

olidal

New Member
Mar 26, 2021
6
1
3
Hmm, I mean you could go for the dkms or "build-only the module" approach, but you'd need to maintain that yourself - but it may be less work required than rebuilding every kernel (albeit I'm guesstimating this one).
Can I "build only the module"? I tried but was never able to insmod it, because the module was not matching with the kernel. I'd be happy to try this option if you can help or give some tips on how to match the two parts.

Well, our repos are not accessible from the public and the signing key is locked down on an HSM.
The safe way for sure.
Contributing the changes required to add aufs (again) and maintain that could be a possibility on the other hand, it would be contained in our kernel package then. But, I'd like to wait out for Q2'22 to make a decision for accepting such a thing:
  • as Proxmox VE 6.4 is still supported till 2022-07 and it has still aufs support (so you could use that in the meantime)
The other issue I have is that older 5.4 kernel do not support some H/W such as the I219-V intel NIC, and you dont include AUFS in the 5.11 backport for PVE6.4, so I guess moving on now to PVE7 is still my best option.

  • ZFS may get overlayfs support until then (knocks wood), making such extra "maintain-aufs" effort superfluous
Sadly, that will never solve my problem as I am using aufs for some of its unique features, not planned for inclusion in overlayfs.

Thanks again, please let me know if we can try to work together on the "build-only the module" option, which I would be happy to maintain and contribute for the Proxmox community.

Olivier.
 

BuFf0k

New Member
Jul 27, 2021
7
1
3
36

gerby

Member
Nov 26, 2019
8
0
6
36
First gen EPYC with a Windows 10 Guest and HOST cpu type; guest failed to boot. Setting CPU type EPYC resolved the issue.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!