Search results

On your /var/lib/vz there would be enough room it is mounted on /dev/mapper/pve-data, but somehow your /dev/dm-0 block device is reading as absolutely full, 33G of 33G used. If it doesn't contain very large files, then it must be chock full of small files or the file system has become corrupted...

Your root device /dev/dm-0 is full. Maybe your log files in /var/log have exploded or maybe there is something stored in /var/lib/vz/dump? Look at the largest files with: find / -xdev -size +100M It being a LVM disk of course, if you have some extra medium you can attach you could also...

Then it seems that your root is mounted with noacl. Or are you using a different file system like ZFS on the root?

@fips It seems that your /tmp mount doesn't accept posix acl. Is it just part of your root mount, or did you set a tmpfs on that?

Ok, so you are going to lock it down to tcp and NFSv4?

Do you get any info on the commandline if you do: /usr/bin/rpcinfo -p ipofnfsserver Also Proxmox tries to a df on the mountpoint of the NFS system. Does that work?

Just looking at your backup logs, I think that it is functioning, but perhaps slower than you thought they would? Snapshots are not yet supported for the LXC vms, unless you put them on ZFS, but rsync backup should also work. It would be better to have a local tmpdir as it says, to conserve...

Good news. A fix has been made in the lxcfs project for a faulty realloc call. This will be incorporated in lxcfs package for Proxmox and will probably fix the dying of lxcfs by the realloc error.

Looking at the output of the lxc-start command, it does not seem that the problem lies in there. The warning is about a deprecation, it is not yet a blocker. But container starting is blocked if your cluster has no quorum.

Yes. Sigh... It's time there was a new installer.

I am thinking, you are using ZFS storage with a SWAP device on ZFS, like the ISO install makes for you. This has problems. https://forum.proxmox.com/threads/zfs-swap-crashes-system.25208/#post-126215

Yes, it improves. But in the end it may be better to just turn swap off.

This is easy to install because it is just point and click in the ISO install, but I have a little reservation about it. There is a known bug with the partitioner placing SWAP on a so-called ZVOL (a virtual block device on zfs). As soon as swap gets used a little, your machine will be prone to...

@SPQRInc Your RAM usage does not look bad. You can easily assign 30G more while keeping enough file cache, if these numbers are typical. Are you seeing a lot of memory usage by the systemd process on the host also, like @hregis? Could you do: cat /proc/1/status lsof -p 1

@SPQRInc About the NTP. That is very interesting. The LXC containers explicitly drop the capability to set the system time at start up. This might lead to some undefined behaviour when NTP thinks it is running as root user. The overcommit, I meant the amount of RAM you set for the containers...

@hregis Do you have systemd installed in your debian containers (Debian 8)? Have you looked at memory consumption of your containers? If I am correct you overcommited your RAM about three times (with all containers running). I also notice that systemd on the host is using a whopping 4GB...

Have to rule something out though. Can you show the configuration files for container 102? Both /etc/pve/lxc/102.conf and /var/lib/lxc/102/config. In the past Proxmox did kernel memory limiting, and that doesn't show up in the memory usage statistic.

You are not the only one. Problem is still being hunted down. LXCfs is necessary for the containers to see their uptime and memory usage combined with the ability to run systemd. https://forum.proxmox.com/threads/cannot-open-proc-stat-transport-endpoint-is-not-connected.25225/

Hello @SPQRInc The mounts look good, or at least not any weirder than usual in the containe. I tested for remount problems, as i suggested earlier, but that doesn't seem to be possible. Are the containers hitting their memory limits at the point of failure?

Hello @SPQRInc Thanks for reporting it. I think I have a fix on the problem now. The apparmor DENIED message makes me suspect that there is a readonly recursive mount of /sys just before that by systemd, which is being allowed, so that wouldn't show up in the log. Problem is with the way it is...