LXC update breaking up write permission

DzAirmaX

Member
Sep 8, 2016
10
3
23
Hi,

I updated my Proxmox host to the latest versions 2 days ago.

------------------------------------------------------------------------------------
proxmox-ve: 6.3-1 (running kernel: 5.4.78-2-pve)
pve-manager: 6.3-3 (running version: 6.3-3/eee5f901)
pve-kernel-5.4: 6.3-3
pve-kernel-helper: 6.3-3
pve-kernel-5.4.78-2-pve: 5.4.78-2
pve-kernel-5.4.78-1-pve: 5.4.78-1
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.0.4-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
libjs-extjs: 6.0.1-10
libknet1: 1.16-pve1
libproxmox-acme-perl: 1.0.7
libproxmox-backup-qemu0: 1.0.2-1
libpve-access-control: 6.1-3
libpve-apiclient-perl: 3.1-3
libpve-common-perl: 6.3-2
libpve-guest-common-perl: 3.1-4
libpve-http-server-perl: 3.1-1
libpve-storage-perl: 6.3-5
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.6-1
lxcfs: 4.0.6-pve1
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.0.6-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.4-4
pve-cluster: 6.2-1
pve-container: 3.3-2
pve-docs: 6.3-1
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-3
pve-firmware: 3.1-3
pve-ha-manager: 3.1-1
pve-i18n: 2.2-2
pve-qemu-kvm: 5.1.0-8
pve-xtermjs: 4.7.0-3
pve-zsync: 2.0-4
qemu-server: 6.3-3
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 0.8.5-pve1
------------------------------------------------------------------------------------


Since then I noticed that none of my lxc containers were able to mount wireguard interfaces anymore.

The error I have while mounting the devices in the LXC appeared after the host upgrade.

Here is what is going on:

wg-quick up wg0
.....
[#] sysctl -q net.ipv4.conf.all.src_valid_mark=1
sysctl: setting key "net.ipv4.conf.all.src_valid_mark": Read-only file system
.....

To install wireguard on those LXC container, I followed the tutorial provided on this forum by another member:

https://nixvsevil.com/posts/wireguard-in-proxmox-lxc/

The lxc container is running in Unprivileged mode. I tried to use the privileged mode: same result.

Everything was working well before updating to lxc-pve: 4.0.6-1 on the host.

Maybe I missing something... At your disposition if you need anything from my end.
 
Last edited:

onion83

New Member
Dec 6, 2020
2
1
3
20
Try downgrade lxc-pve and reboot , I fix it myself ... it was P0 accident :eek:

apt install proxmox-widget-toolkit=2.4-3 libpve-storage-perl=6.3-4 lxc-pve=4.0.3-1 pve-manager=6.3-2

-- Update --
 
Last edited:
  • Like
Reactions: archerious

fragpic

New Member
Feb 7, 2021
12
0
1
31
Same issue.
Downgrading fixed it. This is pretty major as it broke all my containers running docker.
 

kobuki

Well-Known Member
Dec 30, 2008
470
25
48
Same issue here with a CT running OpenVPN. It needs to enable IP forwarding in the kernel (e.g. /proc/sys/net/ipv4/ip_forward). Fails with write permission error. Quite the issue, it kills most network/routing oriented containers. Will try downgrading in the meantime.

Downgrading lxc-pve=4.0.3-1 is a viable workaround until the inevitable fix.
 
Last edited:

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
4,425
529
118
Same issue here with a CT running OpenVPN. It needs to enable IP forwarding in the kernel (e.g. /proc/sys/net/ipv4/ip_forward). Fails with write permission error. Quite the issue, it kills most network/routing oriented containers. Will try downgrading in the meantime.

Downgrading lxc-pve=4.0.3-1 is a viable workaround until the inevitable fix.
here my openvpn container works with the latest lxc-pve 4.0.6-1

could you post the container configuration?
 

avw

Renowned Member
May 31, 2020
1,025
177
63
Same issue here with a CT running OpenVPN. It needs to enable IP forwarding in the kernel (e.g. /proc/sys/net/ipv4/ip_forward). Fails with write permission error. Quite the issue, it kills most network/routing oriented containers. Will try downgrading in the meantime.

Downgrading lxc-pve=4.0.3-1 is a viable workaround until the inevitable fix.
I thought adding the following line to unprivileged containers was the fix for OpenVPN and such: lxc.mount.entry: /dev/net/openvpn dev/net/tun none bind,create=file and creating /dev/net/openvpn on the host with the right permissions (every reboot). Does that not also fix wireguard (which works fine for me)?
 

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
4,425
529
118
I thought adding the following line to unprivileged containers was the fix for OpenVPN and such: lxc.mount.entry: /dev/net/openvpn dev/net/tun none bind,create=file and creating /dev/net/openvpn on the host with the right permissions (every reboot). Does that not also fix wireguard (which works fine for me)?
yes, this approach should work.

also check here for reference [0]

[0]: https://pve.proxmox.com/wiki/OpenVPN_in_LXC
 

kobuki

Well-Known Member
Dec 30, 2008
470
25
48
here my openvpn container works with the latest lxc-pve 4.0.6-1

could you post the container configuration?

Sure, here it is:

Code:
# pct config 101
arch: amd64
cores: 1
hostname: vpn.xxx.private
memory: 512
net0: name=eth0,bridge=vmbr0,hwaddr=8A:D3:78:C4:11:22,type=veth
onboot: 1
ostype: debian
rootfs: thinhdd:vm-101-disk-1,size=8G
swap: 128
lxc.cgroup.devices.allow: c 10:200 rwm
lxc.mount.entry: /dev/net/tun dev/net/tun none bind,optional,create=file

There're no issues with OpenVPN itself - it starts up without issues as usual and we can connect to it. But packets are not forwarded so the LAN behind it is inaccessible. As others, I had to downgrade for now.
 
  • Like
Reactions: JoeSix

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
4,425
529
118
config looks okay, have you restarted the container after the lxc upgrade?

But packets are not forwarded so the LAN behind it is inaccessible.
this works here with basically the same configuration you sent, i can access the lan behind it just fine

maybe you should remove the optional in your mount.entry
 

kobuki

Well-Known Member
Dec 30, 2008
470
25
48
maybe you should remove the optional in your mount.entry

It has nothing to do with the problem at hand. It's just the tun device config, which is working fine. There's a small, but very definite difference in the CT vs. the older version of pve-lxc. The following mount entry is not present in the newer version:

Code:
proc on /proc/sys/net type proc (rw,nosuid,nodev,noexec,relatime)

Other folks in this thread also report different network processes not working, most likely related to the exact same difference. NB that I use the above config in multiple PVE servers and only this one is problematic, where I've upgraded to the latest 6.x version (from 5.x).
 

oguz

Proxmox Staff Member
Staff member
Nov 19, 2018
4,425
529
118
The following mount entry is not present in the newer version:

Code:
proc on /proc/sys/net type proc (rw,nosuid,nodev,noexec,relatime)
can you enable the nesting option? this will mount /proc with rw
 

kobuki

Well-Known Member
Dec 30, 2008
470
25
48
can you enable the nesting option? this will mount /proc with rw
Not now, as the system is live and people started working on it, but as an idea, I might try later. But i was under the impression that nesting is only necessary when processes try to create security namespaces and other security measures using cgroup inside CTs so my gut feeling says it won't help.
 

fragpic

New Member
Feb 7, 2021
12
0
1
31
can you enable the nesting option? this will mount /proc with rw
I have it enabled on my container but /sys is still being mounted at read only.
I am able to get docker to work by remounting it as rw inside the container, but I couldnt do that for all containers so I ended up downgrading.
 

DzAirmaX

Member
Sep 8, 2016
10
3
23
Thanks for checking in @oguz.

Nesting option did not not solve the issue on my end too.

Is /sys supposed to be mounted in ro in a LXC container (privileged / unprivileged / nested) ?
 

Fabian_E

Proxmox Staff Member
Staff member
Aug 1, 2019
1,485
231
68
Hi,
please try upgrading to lxc-pve 4.0.6-2 which is available in our testing repository. It should fix the issues present with 4.0.6-1. Simply add the repository, use apt update && apt install lxc-pve and disable the repository again.
 

archerious

New Member
Feb 12, 2021
4
0
1
24
Try downgrade lxc-pve and reboot , I fix it myself ... it was P0 accident :eek:

apt install proxmox-widget-toolkit=2.4-3 libpve-storage-perl=6.3-4 lxc-pve=4.0.3-1 pve-manager=6.3-2

-- Update --
That fixed it for me, thank you so much.
 

kobuki

Well-Known Member
Dec 30, 2008
470
25
48
It seems lxc-pve 4.0.6-2 is in the no-subs repo, so instead of downgrading it should be enough to upgrade to latest... Though I couldn't try it, yet.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!