Proxmox 9 and nftables status

[swiperman]

Hi, I am wondering about nftables, in the admin guide it says to use nftables you need to install proxmox-firewall instead of pve-firewall (https://pve.proxmox.com/pve-docs/pve-admin-guide.html#pve_firewall_nft). It also says that it is currently in tech preview and not suitable for production use.

But!

I have a production system with 3 hosts in a cluster, I migrate production VM's between hosts when I need to restart hosts because of kernel upgrade or something similar.
After upgrading to pve 9 then when I select a host to migrate I more often then not get a dialog that does not have the usual warning of "Migration with local disk might take long" and if I start the migration at that time it will fail with the following message:

conntrack state migration not supported or disabled, active connections might get dropped

Sometimes when I choose migrate then the warning about migration taking a long time shows up as normal and if I press "Migrate" immediately then it works.

If I do not get the warning then I can wait for about 30 seconds and then I get the usual warning about it taking a long time but also another warning about conntrack

Cannot migrate conntrack state, target node is lacking support. Active network connections might get dropped

If I press migrate at theat point the migration succeeds as normal.

As far as I can understand then this happens because nftables is not enabled on the host, which I have not done because its a production server and the official guide says to not use nftables in production.

So the question is, should we turn on nftables on production hosts?
If not, then how to get rid of this conntrack migration issue?

 

[cheiss]

First, please post the output of pveversion -v, best from all three nodes.

Does /usr/libexec/qemu-server/dbus-vmstate exist on your systems? This is a prerequisite helper for conntrack state migration.

As far as I can understand then this happens because nftables is not enabled on the host,

No, conntrack state migration is possible with both the old pve-firewall and the newer, nftables-based proxmox-firewall.

If I do not get the warning then I can wait for about 30 seconds and then I get the usual warning about it taking a long time

While not directly related to the firewall - what kind of storage do you have? Checking the disks should not take that long.

So the question is, should we turn on nftables on production hosts?

This is something you must decide, there's no general recommendation (currently) - but the old pve-firewall is planned be deprecated at some point in the future AFAIK.
Also, new and/or advanced feature may only work with nftables, see also https://pve.proxmox.com/pve-docs/pve-admin-guide.html#pve_firewall_nft

 

[swiperman]

Hi Christoph, thanks for your reply.

Here is the output of pveversion -v

proxmox-ve: 9.2.0 (running kernel: 7.0.6-2-pve)
pve-manager: 9.2.2 (running version: 9.2.2/b9984c6d90a4bd80)
proxmox-kernel-helper: 9.2.0
proxmox-kernel-7.0: 7.0.6-2
proxmox-kernel-7.0.6-2-pve-signed: 7.0.6-2
proxmox-kernel-7.0.2-7-pve-signed: 7.0.2-7
proxmox-kernel-6.17: 6.17.13-13
proxmox-kernel-6.17.13-13-pve-signed: 6.17.13-13
proxmox-kernel-6.8.12-20-pve-signed: 6.8.12-20
proxmox-kernel-6.8: 6.8.12-20
pve-kernel-5.13.19-6-pve: 5.13.19-15
ceph-fuse: 19.2.3-pve1
corosync: 3.1.10-pve2
criu: 4.1.1-1
frr-pythontools: 10.6.1-1+pve2
ifupdown: residual config
ifupdown2: 3.3.0-1+pmx12
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-5
libproxmox-acme-perl: 1.7.1
libproxmox-backup-qemu0: 2.0.2
libproxmox-rs-perl: 0.4.1
libpve-access-control: 9.1.1
libpve-apiclient-perl: 3.4.2
libpve-cluster-api-perl: 9.1.5
libpve-cluster-perl: 9.1.5
libpve-common-perl: 9.1.12
libpve-guest-common-perl: 6.0.3
libpve-http-server-perl: 6.0.5
libpve-network-perl: 1.6.6
libpve-notify-perl: 9.1.5
libpve-rs-perl: 0.15.3
libpve-storage-perl: 9.1.5
libspice-server1: 0.15.2-1+b1
lvm2: 2.03.31-2+pmx1
lxc-pve: 7.0.0-2
lxcfs: 7.0.0-pve1
novnc-pve: 1.7.0-1
proxmox-backup-client: 4.2.0-1
proxmox-backup-file-restore: 4.2.0-1
proxmox-backup-restore-image: 1.0.0
proxmox-firewall: 1.2.3
proxmox-kernel-helper: 9.2.0
proxmox-mail-forward: 1.0.3
proxmox-mini-journalreader: 1.6
proxmox-offline-mirror-helper: 0.7.4
proxmox-widget-toolkit: 5.2.3
pve-cluster: 9.1.5
pve-container: 6.1.10
pve-docs: 9.2.2
pve-edk2-firmware: 4.2025.05-2
pve-esxi-import-tools: 1.0.1
pve-firewall: 6.0.4
pve-firmware: 3.18-4
pve-ha-manager: 5.2.4
pve-i18n: 3.7.4
pve-qemu-kvm: 11.0.0-3
pve-xtermjs: 6.0.0-1
qemu-server: 9.1.15
smartmontools: 7.5-pve2
spiceterm: 3.4.2
swtpm: 0.8.0+pve3
vncterm: 1.9.2
zfsutils-linux: 2.4.2-pve1

pveversion -v shows the same on all hosts, they are all up to date.

The file /usr/libexec/qemu-server/dbus-vmstate is on all three hosts.

We have 2 3.84 TB NVME disks using zfs in a mirror (so the zpool is 3.84 TB). the warning about "Migration with local disk might take long" has been there all along since we started using Proxmox 5 or 6 years ago so I am not worried about that

As for that there is no general recommendation about switching over to proxmox-firewall, is the admin-guide then not up to date? Because it says there not to use that one in production

 

[swiperman]

Should I perhaps put up a separate question about why I get this error with an updated system using pve-firewall?