Running keepalived in lxc container

[AndAsh]

Hello, please help me to run keepalived in lxc container.
keepalived needs system kernel modules, how do I configure the container to
could keepalived use kernel modules?

 

[Stoiko Ivanov]

No experience with keepalived - but lxc-containers share the kernel with the pve-node.
just load the modules in the node and see if this works

I hope this helps!

 

[AndAsh]

on keepalived startup I get:

Aug 31 14:52:36 haproxy-01 Keepalived_vrrp[1074]: (Line 23) Truncating auth_pass to 8 characters
Aug 31 14:52:36 haproxy-01 Keepalived_vrrp[1074]: Initializing ipvs
Aug 31 14:52:36 haproxy-01 modprobe[1075]: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/5.11.22-4-pve/modules.dep.bin'
Aug 31 14:52:36 haproxy-01 modprobe[1075]: FATAL: Module ip_vs not found in directory /lib/modules/5.11.22-4-pve
Aug 31 14:52:36 haproxy-01 Keepalived_vrrp[1074]: IPVS: Can't initialize ipvs: Permission denied (you must be root)
Aug 31 14:52:36 haproxy-01 Keepalived_vrrp[1074]: Stopped
Aug 31 14:52:36 haproxy-01 Keepalived[1066]: Keepalived_vrrp exited with permanent error FATAL. Terminating
Aug 31 14:52:36 haproxy-01 systemd[1]: keepalived.service: Succeeded.
Aug 31 14:52:36 haproxy-01 Keepalived[1066]: Stopped Keepalived v2.0.10 (11/12,2018)
Aug 31 14:52:36 haproxy-01 systemd[1]: keepalived.service: Consumed 60ms CPU time.

added to container config:

lxc.cgroup.devices.allow: a
lxc.cap.drop:
lxc.mount.auto: proc:rw sys:rw
lxc.mount.entry: /lib/modules lib/modules none ro,bind 0 0

on keepalived startup I get:

Aug 31 15:03:11 haproxy-01 Keepalived_vrrp[1090]: (Line 20) (VI_1) Specifying lvs_sync_daemon_interface against a vrrp is deprecated.
Aug 31 15:03:11 haproxy-01 Keepalived_vrrp[1090]: (Line 20)         Please use global lvs_sync_daemon
Aug 31 15:03:11 haproxy-01 Keepalived_vrrp[1090]: (Line 23) Truncating auth_pass to 8 characters
Aug 31 15:03:11 haproxy-01 Keepalived_vrrp[1090]: Initializing ipvs
Aug 31 15:03:11 haproxy-01 Keepalived_vrrp[1090]: IPVS: Can't initialize ipvs: Permission denied (you must be root)
Aug 31 15:03:11 haproxy-01 Keepalived_vrrp[1090]: Stopped
Aug 31 15:03:11 haproxy-01 Keepalived[1081]: Keepalived_vrrp exited with permanent error FATAL. Terminating
Aug 31 15:03:11 haproxy-01 Keepalived[1081]: Stopped Keepalived v2.0.10 (11/12,2018)
Aug 31 15:03:11 haproxy-01 systemd[1]: keepalived.service: Succeeded.

I can't figure out what permissions are missing.

 

[Dunuin]

I run keepalived inside my unprivileged Pihole LXCs without editing anything special to the LXCs config file. Atleast with Debian 11 LXCs it works fine here out of the box.

 

[leesteken]

added to container config:

lxc.cgroup.devices.allow: a
lxc.cap.drop:
lxc.mount.auto: proc:rw sys:rw
lxc.mount.entry: /lib/modules lib/modules none ro,bind 0 0

This is the opposite of what was suggested:

just load the modules in the node and see if this works

Try running modprobe ip_vs on the Proxmox host before starting the container and then try keepalived. Maybe you need more modules (I don't know keepalived) but you can load them the same way. You can add all required modules to /etc/modules once you have it working so it will automatically load those.

 

[EmilioMoreno]

I run keepalived inside my unprivileged Pihole LXCs without editing anything special to the LXCs config file. Atleast with Debian 11 LXCs it works fine here out of the box.

Proxmox VE6 or ProxmoxVE7 ? Any special config on the host also?

 

[Colin 't Hart]

We've been running this just fine with Ubuntu 18.04 and 20.04 inside the (unprivileged) containers. We did find we needed to add use_vmac to the instances inside the keepalived.conf to prevent both nodes in the keepalive pair from obtaining the virtual IP. Currently still on Proxmox 6.4 on the hosts. Upgrading to 7.3 after the new year.

 

[Dunuin]

Proxmox VE6 or ProxmoxVE7 ? Any special config on the host also?

Proxmox VE7.

 

[tengz]

Problem is that Keepalived requires ip_vs module to available in /lib/modules/*** for running virtual_server in DR (direct routing) mode.
ip_vs module is enabled on the host by modprobe command.

pct set 100 --mp0 /lib/modules,mp=/lib/modules,ro=1

Adding the mount point by this command doesn't work, container just doesn't run after that.
Can it be fixed?
I'm using latest Proxmox with 5.15.85-1 kernel.

 

[toxic]

did someone find a solution ?
I struggle with keepalived as I want to enable `lvs_sync_daemon eth0 VRRP1` but this requires the kernel module ip_vs, which I enabled on the host, but keepalived still looks for it in /lib/modules and I'm unable to pass the /lib/modules to the CT, it always fails, either without message or with the following message :

open_without_symlink: 1289 Too many levels of symbolic links - lib in /usr/lib/x86_64-linux-gnu/lxc/rootfs/lib/modules was a symbolic link!