Failed to load nf_conntrack module

[Fathi]

Hello,
I am having trouble since I've upgraded a CentOS 7.6 container to CentOS 7.7 running on ProxMox 6.0.
The problem is that firewalld no more starts complaining about nf_conntrack module as follows.
ERROR: Failed to load nf_conntrack module: modprobe: ERROR: could not find module by name='nf_conntrack'
modprobe: ERROR: could not insert 'nf_conntrack': Function not implemented
modprobe: ERROR: Error running install command for nf_conntrack...
ERROR: Raising SystemExit in run_server

The output of modinfo nf_conntrack is
modinfo: ERROR: Module alias nf_conntrack not found.

While on a VPS running at french provider OVH, i have as output:
filename: /lib/modules/3.10.0-1062.1.1.el7.x86_64/kernel/net/netfilter/nf_conntrack.ko.xz
license: GPL
retpoline: Y
rhelversion: 7.7
srcversion: 03A8408E58BFA6E173F2FE6
depends: libcrc32c
intree: Y
vermagic: 3.10.0-1062.1.1.el7.x86_64 SMP mod_unload modversions
signer: CentOS Linux kernel signing key
sig_key: 34:1A:1E:7B:066:87:15:3E:3A:E9:8D:3E:B5:6E:0E:CD:30B:79
sig_hashalgo: sha256
parm: tstamp:Enable connection tracking flow timestamping. (bool)
parm: acct:Enable connection tracking flow accounting. (bool)
parm: nf_conntrack_helper:Enable automatic conntrack helper assignment (default 1) (bool)
parm: expect_hashsize:uint

On the ProxMox server side, i have also a positive output:
filename: /lib/modules/5.0.21-2-pve/kernel/net/netfilter/nf_conntrack.ko
license: GPL
alias: nf_conntrack-10
alias: nf_conntrack-2
alias: ip_conntrack
srcversion: ECF2FC78962840323375B8C
depends: nf_defrag_ipv6,libcrc32c,nf_defrag_ipv4
retpoline: Y
intree: Y
name: nf_conntrack
vermagic: 5.0.21-2-pve SMP mod_unload modversions
parm: tstamp:Enable connection tracking flow timestamping. (bool)
parm: acct:Enable connection tracking flow accounting. (bool)
parm: nf_conntrack_helper:Enable automatic conntrack helper assignment (default 0) (bool)
parm: expect_hashsize:uint


The output of rpm -qf /lib/modules/3.10.0-1062.1.1.el7.x86_64/kernel/net/netfilter/nf_conntrack.ko.xz on the centos 7.7 vps is
kernel-3.10.0-1062.1.1.el7.x86_64

Now, on a proxmox container, there is no kernel installed as it is a container.
How could i let the firewalld daemon on the centos container use the nf_conntrack module of the proxmox hypervisor ?
TIA.

 

[t.lamprecht]

Hey,

Hmm the single thing I could immagine is that the "nf_conntrack" module is not loaded on the Proxmox VE host (CT are normally not allowed to load modules not already loaded).

I cannot reproduce this here on my local Proxmox VE 6 with a CentOS 7.7 CT, granted I freshly installed it (no upgrade) but still..

[root@CT139 ~]# cat /etc/centos-release
CentOS Linux release 7.7.1908 (Core)
[root@CT139 ~]# lsmod | grep nf_conn
nf_conncount           24576  1 openvswitch
nf_conntrack          139264  5 nf_nat,nf_nat_ipv6,nf_nat_ipv4,openvswitch,nf_conncount
nf_defrag_ipv6         24576  2 nf_conntrack,openvswitch
nf_defrag_ipv4         16384  1 nf_conntrack
libcrc32c              16384  4 nf_conntrack,nf_nat,dm_persistent_data,openvswitch

Can you check if the module is loaded on the PVE host with lsmod | grep nf_conn and modprobe it there if not?
Then it should be available in the CT too.

 

[t.lamprecht]

firewalld.service seems to work here just fine too:

[root@CT139 ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2019-09-27 05:00:26 UTC; 6s ago
     Docs: man:firewalld(1)
 Main PID: 575 (firewalld)
   CGroup: /system.slice/firewalld.service
           └─575 /usr/bin/python2 -Es /usr/sbin/firewalld --nofork --nopid

Sep 27 05:00:26 CT139 systemd[1]: Starting firewalld - dynamic firewall daemon...
Sep 27 05:00:26 CT139 systemd[1]: Started firewalld - dynamic firewall daemon.
Sep 27 05:00:27 CT139 firewalld[575]: ERROR: Failed to read file "/proc/sys/net/netfilter/nf_conntrack_helper": [Errno 2] No such file or directory: '/proc/sys/net/netfilter/nf_conntrack_helper'
Sep 27 05:00:27 CT139 firewalld[575]: WARNING: Failed to get and parse nf_conntrack_helper setting

The last error seems to come from an older firewalld version issue.

But there are some other search hits comming up, e.g. from CentOS/Fedora world:
https://bugzilla.redhat.com/show_bug.cgi?id=1686654

But my guess is still that the module is just not loaded on the host (albeit available, as your modinfo command showed)

 

[Fathi]

Hello,
By the way, as this is an urgent case, I removed firewalld firewalld-filesystem and python-firewall all 0.6.x branch and reinstalled from centos vault those of the 7.6 version of centos (0.5.3-5). And now the firewalld service starts.

Now, on the centos 7.7 container:
lsmod | grep nf_
nf_reject_ipv4 16384 1 ipt_REJECT
nf_reject_ipv6 20480 1 ip6t_REJECT
nf_nat_ipv6 16384 2 ip6table_nat,ip6t_MASQUERADE
nf_nat_ipv4 16384 2 ipt_MASQUERADE,iptable_nat
nf_nat 36864 2 nf_nat_ipv6,nf_nat_ipv4
nf_conntrack 139264 6 xt_conntrack,nf_nat,ip6t_MASQUERADE,nf_nat_ipv6,ipt_MASQUERADE,nf_nat_ipv4
nf_defrag_ipv6 24576 1 nf_conntrack
nf_defrag_ipv4 16384 1 nf_conntrack
libcrc32c 16384 4 nf_conntrack,nf_nat,dm_persistent_data,btrfs

and on the proxmox hypervisor:
lsmod | grep nf_
nf_reject_ipv4 16384 1 ipt_REJECT
nf_reject_ipv6 20480 1 ip6t_REJECT
nf_nat_ipv6 16384 2 ip6table_nat,ip6t_MASQUERADE
nf_nat_ipv4 16384 2 ipt_MASQUERADE,iptable_nat
nf_nat 36864 2 nf_nat_ipv6,nf_nat_ipv4
nf_conntrack 139264 6 xt_conntrack,nf_nat,ip6t_MASQUERADE,nf_nat_ipv6,ipt_MASQUERADE,nf_nat_ipv4
nf_defrag_ipv6 24576 1 nf_conntrack
nf_defrag_ipv4 16384 1 nf_conntrack
libcrc32c 16384 4 nf_conntrack,nf_nat,dm_persistent_data,btrfs

So nf_conntrack seems to be loaded on both the hypervisor and the container and beening useable by firewalld 0.5.3 but not by firewalld 0.6.3.

 

[t.lamprecht]

So nf_conntrack seems to be loaded on both the hypervisor and the container and beening useable by firewalld 0.5.3 but not by firewalld 0.6.3.

Hmm, OK, maybe it then make sense to report this upstream in a GitHub issue in firewalld or in the centos firewalld package bug tracker? Aa IMO this is then not an issue of Proxmox VE and or it's LXC Implementation.

 

[Fathi]

Thank you,
Submitted on the firewalld github as issue #519.