[SOLVED] Containers can't mount NFS shares after recent update

aaronouthier

Member
Feb 18, 2021
13
0
6
45
Hello,

My NFS connectons also went down for all of my containers after a Proxmox 6.3 update about a week ago.
Only Proxmox containers seem to be affected. NFS client says the server is rejecting the connection or something, however, the NFS server doesn't seem to log any connection attempt. Running dmesg on the client spews about 3 pages of errors mentioning NFS and "apparmor", as well as quite a bit of other gibberish.

All of my Proxmox VMs, as well as separate physical machines can access the same NFS server share just fine, with the same command-line options used in my Proxmox LXC containers. Only the LXC containers are failing to connect.
 
Last edited:
Code:
[ 6249.395083] audit: type=1400 audit(1618018762.372:126): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=6400 comm="(ionclean)" srcname="/" flags="rw, rbind"    [ 8049.402164] audit: type=1400 audit(1618020562.369:127): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=15868 comm="(ionclean)" srcname="/" flags="rw, rbind"   [ 9849.411037] audit: type=1400 audit(1618022362.373:128): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=25327 comm="(ionclean)" srcname="/" flags="rw, rbind"   [11649.417278] audit: type=1400 audit(1618024162.371:129): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=2531 comm="(ionclean)" srcname="/" flags="rw, rbind"    [13449.423896] audit: type=1400 audit(1618025962.372:130): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=12075 comm="(ionclean)" srcname="/" flags="rw, rbind"   [15249.427970] audit: type=1400 audit(1618027762.372:131): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=21528 comm="(ionclean)" srcname="/" flags="rw, rbind"   [17049.437587] audit: type=1400 audit(1618029562.371:132): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=30982 comm="(ionclean)" srcname="/" flags="rw, rbind"   [18849.444176] audit: type=1400 audit(1618031362.373:133): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=8238 comm="(ionclean)" srcname="/" flags="rw, rbind"    [20649.450960] audit: type=1400 audit(1618033162.369:134): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=17772 comm="(ionclean)" srcname="/" flags="rw, rbind"   [22449.459148] audit: type=1400 audit(1618034962.370:135): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=27193 comm="(ionclean)" srcname="/" flags="rw, rbind"   [24249.465442] audit: type=1400 audit(1618036762.372:136): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=4434 comm="(ionclean)" srcname="/" flags="rw, rbind"    [25488.607618] audit: type=1400 audit(1618038001.508:137): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=11150 comm="(ogrotate)" srcname="/" flags="rw, rbind"   [25489.553622] audit: type=1400 audit(1618038002.456:138): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=11194 comm="(pachectl)" srcname="/" flags="rw, rbind"   [25576.744452] audit: type=1400 audit(1618038089.647:139): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-101_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=11685 comm="(ogrotate)" srcname="/" flags="rw, rbind"   [26049.472304] audit: type=1400 audit(1618038562.373:140): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=14149 comm="(ionclean)" srcname="/" flags="rw, rbind"   [27849.480142] audit: type=1400 audit(1618040362.373:141): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=23578 comm="(ionclean)" srcname="/" flags="rw, rbind"   [29649.488371] audit: type=1400 audit(1618042162.373:142): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=558 comm="(ionclean)" srcname="/" flags="rw, rbind"     [31449.495527] audit: type=1400 audit(1618043962.371:143): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=10221 comm="(ionclean)" srcname="/" flags="rw, rbind"   [33249.501836] audit: type=1400 audit(1618045762.372:144): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=19702 comm="(ionclean)" srcname="/" flags="rw, rbind"   [33697.230820] perf: interrupt took too long (2551 > 2500), lowering kernel.perf_event_max_sample_rate to 78250                                                           [35049.509855] audit: type=1400 audit(1618047562.372:145): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=29117 comm="(ionclean)" srcname="/" flags="rw, rbind"   [36849.515705] audit: type=1400 audit(1618049362.370:146): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=6697 comm="(ionclean)" srcname="/" flags="rw, rbind"    [38649.522687] audit: type=1400 audit(1618051162.371:147): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=16185 comm="(ionclean)" srcname="/" flags="rw, rbind"   [40449.530260] audit: type=1400 audit(1618052962.372:148): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=25616 comm="(ionclean)" srcname="/" flags="rw, rbind"   [42249.538065] audit: type=1400 audit(1618054762.372:149): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=2896 comm="(ionclean)" srcname="/" flags="rw, rbind"    [44049.545289] audit: type=1400 audit(1618056562.369:150): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=12368 comm="(ionclean)" srcname="/" flags="rw, rbind"   [45248.143769] perf: interrupt took too long (3207 > 3188), lowering kernel.perf_event_max_sample_rate to 62250                                                           [45849.552029] audit: type=1400 audit(1618058362.371:151): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=21810 comm="(ionclean)" srcname="/" flags="rw, rbind"   [47649.556176] audit: type=1400 audit(1618060162.371:152): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=31254 comm="(ionclean)" srcname="/" flags="rw, rbind"   [49449.567002] audit: type=1400 audit(1618061962.373:153): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=8527 comm="(ionclean)" srcname="/" flags="rw, rbind"    [51249.573692] audit: type=1400 audit(1618063762.370:154): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=18273 comm="(ionclean)" srcname="/" flags="rw, rbind"   [53049.579954] audit: type=1400 audit(1618065562.371:155): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=27748 comm="(ionclean)" srcname="/" flags="rw, rbind"   [54849.587831] audit: type=1400 audit(1618067362.372:156): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=4924 comm="(ionclean)" srcname="/" flags="rw, rbind"    [56649.595076] audit: type=1400 audit(1618069162.372:157): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=14428 comm="(ionclean)" srcname="/" flags="rw, rbind"   [58449.602126] audit: type=1400 audit(1618070962.370:158): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=23842 comm="(ionclean)" srcname="/" flags="rw, rbind"   [60249.609162] audit: type=1400 audit(1618072762.370:159): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=908 comm="(ionclean)" srcname="/" flags="rw, rbind"     [62049.618265] audit: type=1400 audit(1618074562.370:160): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=10524 comm="(ionclean)" srcname="/" flags="rw, rbind"   [63849.623119] audit: type=1400 audit(1618076362.373:161): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=20101 comm="(ionclean)" srcname="/" flags="rw, rbind"   [65649.630059] audit: type=1400 audit(1618078162.370:162): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=29585 comm="(ionclean)" srcname="/" flags="rw, rbind"   [67449.637778] audit: type=1400 audit(1618079962.371:163): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=6779 comm="(ionclean)" srcname="/" flags="rw, rbind"    [69249.646765] audit: type=1400 audit(1618081762.371:164): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=16252 comm="(ionclean)" srcname="/" flags="rw, rbind"   [71049.655896] audit: type=1400 audit(1618083562.376:165): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=25679 comm="(ionclean)" srcname="/" flags="rw, rbind"   [72849.659648] audit: type=1400 audit(1618085362.370:166): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=2964 comm="(ionclean)" srcname="/" flags="rw, rbind"    [74649.668035] audit: type=1400 audit(1618087162.373:167): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=12447 comm="(ionclean)" srcname="/" flags="rw, rbind"   [76449.677932] audit: type=1400 audit(1618088962.371:168): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=21870 comm="(ionclean)" srcname="/" flags="rw, rbind"   [78249.686506] audit: type=1400 audit(1618090762.371:169): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=31313 comm="(ionclean)" srcname="/" flags="rw, rbind"   [80049.688968] audit: type=1400 audit(1618092562.369:170): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=8614 comm="(ionclean)" srcname="/" flags="rw, rbind"    [81849.693404] audit: type=1400 audit(1618094362.372:171): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=18300 comm="(ionclean)" srcname="/" flags="rw, rbind"   [83649.701831] audit: type=1400 audit(1618096162.372:172): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=27909 comm="(ionclean)" srcname="/" flags="rw, rbind"   [85449.709502] audit: type=1400 audit(1618097962.373:173): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=5311 comm="(ionclean)" srcname="/" flags="rw, rbind"    [87249.715965] audit: type=1400 audit(1618099762.370:174): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=14987 comm="(ionclean)" srcname="/" flags="rw, rbind"   [89049.724506] audit: type=1400 audit(1618101562.370:175): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-100_</var/lib/lxc>" name="/run/systemd/unit-root/" pid=24640 comm="(ionclean)" srcname="/" flags="rw, rbind"   [89764.827989] audit: type=1400 audit(1618102277.471:176): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-101_</var/lib/lxc>" name="/mnt/tv/" pid=28464 comm="mount.nfs" fstype="nfs" srcname="192.168.15.30:/mnt/tv"

All of that was generated by a single command inside a Proxmox 6.3 LXC container with the built-in Ubuntu 20.04 template:
Code:
mount -t nfs -o rw,port=2049,proto=tcp,vers=4.2 192.168.15.30:/mnt/tv /mnt/tv
 
Output of pveversion -v:
Code:
proxmox-ve: 6.3-1 (running kernel: 5.4.106-1-pve)                                    pve-manager: 6.3-6 (running version: 6.3-6/2184247e)                                 pve-kernel-5.4: 6.3-8                                                                pve-kernel-helper: 6.3-8                                                             pve-kernel-5.4.106-1-pve: 5.4.106-1                                                  pve-kernel-5.4.103-1-pve: 5.4.103-1                                                  pve-kernel-5.4.73-1-pve: 5.4.73-1                                                    ceph-fuse: 15.2.9-pve1                                                               corosync: 3.1.0-pve1                                                                 criu: 3.11-3                                                                         glusterfs-client: 5.5-3                                                              ifupdown: 0.8.35+pve1                                                                ksm-control-daemon: 1.3-1                                                            libjs-extjs: 6.0.1-10                                                                libknet1: 1.20-pve1                                                                  libproxmox-acme-perl: 1.0.8                                                          libproxmox-backup-qemu0: 1.0.3-1                                                     libpve-access-control: 6.1-3                                                         libpve-apiclient-perl: 3.1-3                                                         libpve-common-perl: 6.3-5                                                            libpve-guest-common-perl: 3.1-5                                                      libpve-http-server-perl: 3.1-1                                                       libpve-storage-perl: 6.3-8                                                           libqb0: 1.0.5-1                                                                      libspice-server1: 0.14.2-4~pve6+1                                                    lvm2: 2.03.02-pve4                                                                   lxc-pve: 4.0.6-2                                                                     lxcfs: 4.0.6-pve1                                                                    novnc-pve: 1.1.0-1                                                                   proxmox-backup-client: 1.0.13-1                                                      proxmox-mini-journalreader: 1.1-1                                                    proxmox-widget-toolkit: 2.4-9                                                        pve-cluster: 6.2-1                                                                   pve-container: 3.3-4                                                                 pve-docs: 6.3-1                                                                      pve-edk2-firmware: 2.20200531-1                                                      pve-firewall: 4.1-3                                                                  pve-firmware: 3.2-2                                                                  pve-ha-manager: 3.1-1                                                                pve-i18n: 2.3-1                                                                      pve-qemu-kvm: 5.2.0-5                                                                pve-xtermjs: 4.7.0-3                                                                 qemu-server: 6.3-10                                                                  smartmontools: 7.2-pve2                                                              spiceterm: 3.1-1                                                                     vncterm: 1.6-2                                                                       zfsutils-linux: 2.0.4-pve1
 
Update: I figured it out! Ability to mount NFS is controlled in Options->Features. Have to tick NFS mount option.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!