Trouble with OpenVPN likely because it's a privileged lxc

R

Rabid3east

Member
Jan 20, 2022
15
0
6
38
I've gone through the forums and every wiki I can find but so far nothing has solved my issue. Need to be able to route some docker compose containers through a vpn.

version: "3.5" services: transmission-openvpn: container_name: transmission-openvpn cap_add: - NET_ADMIN volumes: - '/mnt/downloads:/data' environment: - OPENVPN_PROVIDER=SURFSHARK - OPENVPN_CONFIG=ar-bua_udp - OPENVPN_USERNAME=***** - OPENVPN_PASSWORD=***** - LOCAL_NETWORK=192.168.0.0/16 - PGID=1000 - PUID=1000 logging: driver: json-file options: max-size: 10m restart: always ports: - 9091:9091

When I try to run docker compose up I get this error message.

⠿ Container transmission-openvpn Created 0.0s Attaching to transmission-openvpn transmission-openvpn | Starting container with revision: 901c0d2def15fcbd2e2cde2bddd1799678e85a31 transmission-openvpn | Creating TUN device /dev/net/tun transmission-openvpn | Using OpenVPN provider: SURFSHARK transmission-openvpn | Running with VPN_CONFIG_SOURCE auto transmission-openvpn | No bundled config script found for SURFSHARK. Defaulting to external config transmission-openvpn | Downloading configs from https://github.com/haugene/vpn-configs-contrib/archive/main.zip into /tmp/tmp.CtAKnU3WWA transmission-openvpn | Extracting configs to /tmp/tmp.IZ6zrvAgc8 transmission-openvpn | Found configs for SURFSHARK in /tmp/tmp.IZ6zrvAgc8/vpn-configs-contrib-main/openvpn/surfshark, will replace current content in /etc/openvpn/surfshark transmission-openvpn | Cleanup: deleting /tmp/tmp.CtAKnU3WWA and /tmp/tmp.IZ6zrvAgc8 transmission-openvpn | Starting OpenVPN using config ar-bua_udp.ovpn transmission-openvpn | Modifying /etc/openvpn/surfshark/ar-bua_udp.ovpn for best behaviour in this container transmission-openvpn | Modification: Point auth-user-pass option to the username/password file transmission-openvpn | Modification: Change ca certificate path transmission-openvpn | Modification: Change ping options transmission-openvpn | Modification: Update/set resolv-retry to 15 seconds transmission-openvpn | Modification: Change tls-crypt keyfile path transmission-openvpn | Modification: Set output verbosity to 3 transmission-openvpn | Modification: Remap SIGUSR1 signal to SIGTERM, avoid OpenVPN restart loop transmission-openvpn | Setting OpenVPN credentials... transmission-openvpn | adding route to local network 192.168.0.0/16 via 192.168.128.1 dev eth0 transmission-openvpn | Tue Aug 2 12:24:03 2022 OpenVPN 2.4.7 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Mar 22 2022 transmission-openvpn | Tue Aug 2 12:24:03 2022 library versions: OpenSSL 1.1.1f 31 Mar 2020, LZO 2.10 transmission-openvpn | Tue Aug 2 12:24:03 2022 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts transmission-openvpn | Tue Aug 2 12:24:03 2022 Outgoing Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication transmission-openvpn | Tue Aug 2 12:24:03 2022 Incoming Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication transmission-openvpn | Tue Aug 2 12:24:03 2022 TCP/UDP: Preserving recently used remote address: [AF_INET]91.206.168.56:1194 transmission-openvpn | Tue Aug 2 12:24:03 2022 Socket Buffers: R=[212992->212992] S=[212992->212992] transmission-openvpn | Tue Aug 2 12:24:03 2022 UDP link local: (not bound) transmission-openvpn | Tue Aug 2 12:24:03 2022 UDP link remote: [AF_INET]91.206.168.56:1194 transmission-openvpn | Tue Aug 2 12:24:03 2022 TLS: Initial packet from [AF_INET]91.206.168.56:1194, sid=077d6f4d 147d1343 transmission-openvpn | Tue Aug 2 12:24:03 2022 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this transmission-openvpn | Tue Aug 2 12:24:04 2022 VERIFY OK: depth=2, C=VG, O=Surfshark, CN=Surfshark Root CA transmission-openvpn | Tue Aug 2 12:24:04 2022 VERIFY OK: depth=1, C=VG, O=Surfshark, CN=Surfshark Intermediate CA transmission-openvpn | Tue Aug 2 12:24:04 2022 VERIFY KU OK transmission-openvpn | Tue Aug 2 12:24:04 2022 Validating certificate extended key usage transmission-openvpn | Tue Aug 2 12:24:04 2022 ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication transmission-openvpn | Tue Aug 2 12:24:04 2022 VERIFY EKU OK transmission-openvpn | Tue Aug 2 12:24:04 2022 VERIFY OK: depth=0, CN=ar-bua-v022.prod.surfshark.com transmission-openvpn | Tue Aug 2 12:24:04 2022 WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1633', remote='link-mtu 1581' transmission-openvpn | Tue Aug 2 12:24:04 2022 WARNING: 'cipher' is used inconsistently, local='cipher AES-256-CBC', remote='cipher AES-256-GCM' transmission-openvpn | Tue Aug 2 12:24:04 2022 WARNING: 'auth' is used inconsistently, local='auth SHA512', remote='auth [null-digest]' transmission-openvpn | Tue Aug 2 12:24:04 2022 Control Channel: TLSv1.2, cipher TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384, 2048 bit RSA transmission-openvpn | Tue Aug 2 12:24:04 2022 [ar-bua-v022.prod.surfshark.com] Peer Connection Initiated with [AF_INET]91.206.168.56:1194 transmission-openvpn | Tue Aug 2 12:24:05 2022 SENT CONTROL [ar-bua-v022.prod.surfshark.com]: 'PUSH_REQUEST' (status=1) transmission-openvpn | Tue Aug 2 12:24:06 2022 PUSH: Received control message: 'PUSH_REPLY,dhcp-option DNS 162.252.172.57,dhcp-option DNS 149.154.159.92,redirect-gateway def1,sndbuf 524288,rcvbuf 524288,explicit-exit-notify,block-outside-dns,route-gateway 10.8.8.1,topology subnet,ping 60,ping-restart 180,ifconfig 10.8.8.4 255.255.255.0,peer-id 2,cipher AES-256-GCM' transmission-openvpn | Tue Aug 2 12:24:06 2022 Options error: Unrecognized option or missing or extra parameter(s) in [PUSH-OPTIONS]:7: block-outside-dns (2.4.7) transmission-openvpn | Tue Aug 2 12:24:06 2022 OPTIONS IMPORT: timers and/or timeouts modified transmission-openvpn | Tue Aug 2 12:24:06 2022 OPTIONS IMPORT: explicit notify parm(s) modified transmission-openvpn | Tue Aug 2 12:24:06 2022 OPTIONS IMPORT: --sndbuf/--rcvbuf options modified transmission-openvpn | Tue Aug 2 12:24:06 2022 Socket Buffers: R=[212992->425984] S=[212992->425984] transmission-openvpn | Tue Aug 2 12:24:06 2022 OPTIONS IMPORT: --ifconfig/up options modified transmission-openvpn | Tue Aug 2 12:24:06 2022 OPTIONS IMPORT: route options modified transmission-openvpn | Tue Aug 2 12:24:06 2022 OPTIONS IMPORT: route-related options modified transmission-openvpn | Tue Aug 2 12:24:06 2022 OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified transmission-openvpn | Tue Aug 2 12:24:06 2022 OPTIONS IMPORT: peer-id set transmission-openvpn | Tue Aug 2 12:24:06 2022 OPTIONS IMPORT: adjusting link_mtu to 1656 transmission-openvpn | Tue Aug 2 12:24:06 2022 OPTIONS IMPORT: data channel crypto options modified transmission-openvpn | Tue Aug 2 12:24:06 2022 Data Channel: using negotiated cipher 'AES-256-GCM' transmission-openvpn | Tue Aug 2 12:24:06 2022 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key transmission-openvpn | Tue Aug 2 12:24:06 2022 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key transmission-openvpn | Tue Aug 2 12:24:06 2022 ROUTE_GATEWAY 192.168.128.1/255.255.240.0 IFACE=eth0 HWADDR=02:42:c0:a8:80:05 transmission-openvpn | Tue Aug 2 12:24:06 2022 ERROR: Cannot open TUN/TAP dev /dev/net/tun: Operation not permitted (errno=1) transmission-openvpn | Tue Aug 2 12:24:06 2022 Exiting due to fatal error

Looking through the debug page https://haugene.github.io/docker-transmission-openvpn/debug/ I went step by step with their debugging process which ultimately failed and lead me to their FAQ page and suggested I try: docker run --privileged which fails right away. Source: https://haugene.github.io/docker-transmission-openvpn/faq/#tunsetiff_tun_operation_not_permitted

Another forum post here suggested editing the lxc config file.

[jellyfin_transcode] #jellyfin docker composed installed with transcoding arch: amd64 cores: 1 features: mount=nfs;cifs,keyctl=1,nesting=1 hostname: docker memory: 4048 net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=16:CE:53:CC:F2:4F,type=veth ostype: debian parent: clean_install rootfs: local-lvm:vm-209-disk-0,size=8G snaptime: 1659361438 swap: 512 lxc.cgroup2.devices.allow: c 226:128 rwm lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file 0 0 lxc.cgroup2.devices.allow: c 10:200 rwm lxc.mount.entry: /dev/net/tun dev/net/tun none bind,create=file

The last two entries are new but I guess that's what worked for others? Source: https://forum.proxmox.com/threads/p...rking-after-upgrade-to-7-0.92178/#post-401606

I'm beating my head up against the wall as I've already had to recreate this lxc due to the first one being unprivileged and unable to accept SMB or NFS shares. Any insight would be gratefully appreciated.
 
hi,
Rabid3east said:
Cannot open TUN/TAP dev /dev/net/tun: Operation not permitted (errno=1)
Click to expand...
can you see the /dev/net/tun device inside the LXC? check ls -aln /dev/net/tun from inside the LXC and from the PVE host, compare the uid values
 
Here are the results from inside the LXC.
root@docker ~# ls -aln /dev/net/tun
crw-rw-rw- 1 0 0 10, 200 Jul 27 07:30 /dev/net/tun
 
Being that the privileged flag caused failures right away, I tried other things. I found that, "

--device=[]Allows you to run devices inside the container without the --privileged flag.

Even have this running to allow one of my dock containers access to Intel Quick-sync. I used:
devices: - /dev/dri:/dev/dri
in that situation and it worked.

Tried something similar for this current issue.

devices: - /dev/net/tun:/dev/net/tun
and
devices: - /dev/net:/dev/net
But nothing has worked yet. Still get the same error.
 
Rabid3east said:
Did I not edit the lxc ###.config file properly to give permission to the /dev/net/tun?
Click to expand...
the lxc config looks alright for passing /dev/net/tun, and the permission inside the container seems correct (the file belongs to uid 0 = root), so your problem is probably on the docker end.

a word of advice is that using docker inside LXC container is not very advisable because of complications and security concerns that come with nesting containers.
 
What's the recommended method for running docker then? I thought this was how most installed their docker compose
 
Rabid3east said:
What's the recommended method for running docker then? I thought this was how most installed their docker compose
Click to expand...
you could make a VM and run your docker containers in there
 
I've come so far for my use case in docker in the LXC. Hate to not resolve this issue but I do see how it could create some security issues.
 
The docker / docker compose guides on youtube are littered with so many setting them up on LXC containers. I went ahead and created a Ubuntu Server VM but is there another good VM or KVM to use for my purpose that is light weight? It uses 400mb ram before running anything but the kernel.
 
Rabid3east said:
The docker / docker compose guides on youtube are littered with so many setting them up on LXC containers.
Click to expand...
Yes, YT is known for their expertise ... containers in containers ... we are not in Inciption here.

Rabid3east said:
I went ahead and created a Ubuntu Server VM but is there another good VM or KVM to use for my purpose that is light weight?
Click to expand...
Depends on what you want. Normally, we have Docker VMs that have a lot of Docker containers on them, so in the end the "empty memory footprint" does not matter. If you only want to run one Docker container, it's a lot of overhead. The kernel itself will be the biggest overhead with respect to RAM and the smallest "normal" distribution I can think of is Alpine, but if you want to run Docker, you may be better off with a Docker-only Distribution like Rancher.

I don't get why you don't use LX(C) containers (without the additional Docker) and just use OpenVPN there? My Alpine containers with OpenVPN are very small (64 MB diskspace, 32 MB RAM as container upper limits/settings, used is less). I run a couple of them side-by-side without any problem. Updating Alpine is also very easy, although not as easy as updating the Docker container, if there are regular updates.
 
in case you just want to have a container that's connected to another openvpn server, you can take a look here [0]

the linked wiki page is about setting up an openvpn server, but if you just do the part until the installation script, you should have an unprivileged LXC container capable of connecting to a VPN of your choice.
from there you can bindmount a share from your host where your transmission will save the files to.

[0]: https://pve.proxmox.com/wiki/OpenVPN_in_LXC
 
LnxBil said:
you may be better off with a Docker-only Distribution like Rancher.
Click to expand...
I may check that out. I ended with a debian minimal install which is working well until I can't get quicksync passthrough working. Ironically I had it working in the LXC lol. Setting up a jellyfin VM/LXC has been time consuming. Because I'm facing new issues in a different environment, I created a new thread for that. https://forum.proxmox.com/threads/debian-minimal-docker-compose-quicksync-passthrough.113145/ I'd be happy if either of them would work.
LnxBil said:
I don't get why you don't use LX(C) containers (without the additional Docker) and just use OpenVPN there? My Alpine containers with OpenVPN are very small (64 MB diskspace, 32 MB RAM as container upper limits/settings, used is less). I run a couple of them side-by-side without any problem. Updating Alpine is also very easy, although not as easy as updating the Docker container, if there are regular updates.
Click to expand...
What do you mean without the additional Docker? I'm setting up jellyfin openvpn radarr etc as a stack and docker just does that so well. I havn't tried Alpine containers but my use case was to put vpn on transmission or the like.
oguz said:
the linked wiki page is about setting up an openvpn server, but if you just do the part until the installation script, you should have an unprivileged LXC container capable of connecting to a VPN of your choice.
from there you can bindmount a share from your host where your transmission will save the files to.
Click to expand...
Maybe others have been able to bindmount in an unprivileged LXC but I wasn't able to. I read most said it's just not possible. That's where I originally started this docker stack journey. I needed to mount my smb or nfs shares from the NAS to the docker. Other than that unprivileged LXC's were easier to work with. I had the privileged LXC nearly at the finish line, with mounted samba shares and passthrough quicksync. Sadly, I just can't get the /dev/net/tun to passthrough in the privileged container and nobody seems to understand why.
 
Rabid3east said:
Maybe others have been able to bindmount in an unprivileged LXC but I wasn't able to. I read most said it's just not possible. That's where I originally started this docker stack journey. I needed to mount my smb or nfs shares from the NAS to the docker.
Click to expand...
it's definitely possible, see our wiki [0]

to mount your SMB/NFS shares into the unprivileged container, you can mount them on your host and then do the bindmount. in the end you just have to adjust the permissions of the folders from the host side.

Rabid3east said:
Sadly, I just can't get the /dev/net/tun to passthrough in the privileged container and nobody seems to understand why.
Click to expand...
as i mentioned before your passthrough inside the LXC container is working, the nested docker container is having issues with it.

[0]: https://pve.proxmox.com/wiki/Linux_Container#_bind_mount_points
 
oguz said:
it's definitely possible, see our wiki [0]

to mount your SMB/NFS shares into the unprivileged container, you can mount them on your host and then do the bindmount. in the end you just have to adjust the permissions of the folders from the host side.
Click to expand...
Okay, maybe that's what I misunderstood. I think I may give unprivilaged another try, sadly I deleted the lxc.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom