[SOLVED] Local full after backup unsuccessfull

[unnaDmed]

Hey guys,

Proxmox is amazing and was working pretty flawlessly in my environment. Today I checked in on the system and saw that there was a backup job throwing an error. Checking the storages I saw that my external harddrive dropped from 500 GB storage to just 100 GB storage. After a reboot the capacity was back to 500 GB again. This harddrive is mounted and used for the backups. Within the same time the local storage increased for the aprox amount of general backups of my system. I now checked with several tools to find the "missing" backup-jobs as the local storage is now at around 83%, but I can't find them neither delete them to get back to my ~10% usage.

This is my "mount":

sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=8038216k,nr_inodes=2009554,mode=755,inode64)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,noexec,relatime,size=1614408k,mode=755,inode64)
/dev/mapper/pve-root on / type ext4 (rw,relatime,errors=remount-ro)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,inode64)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k,inode64)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
efivarfs on /sys/firmware/efi/efivars type efivarfs (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2496)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
ramfs on /run/credentials/systemd-sysusers.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
ramfs on /run/credentials/systemd-tmpfiles-setup-dev.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
/dev/sdb1 on /mnt/usb-backup type ext4 (rw,relatime)
/dev/sda2 on /boot/efi type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)
ramfs on /run/credentials/systemd-sysctl.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
ramfs on /run/credentials/systemd-tmpfiles-setup.service type ramfs (ro,nosuid,nodev,noexec,relatime,mode=700)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,nosuid,nodev,noexec,relatime)
sunrpc on /run/rpc_pipefs type rpc_pipefs (rw,relatime)
lxcfs on /var/lib/lxcfs type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
/dev/fuse on /etc/pve type fuse (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other)
tracefs on /sys/kernel/debug/tracing type tracefs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=1614404k,nr_inodes=403601,mode=700,inode64)

This is from "df -h":

Filesystem            Size  Used Avail Use% Mounted on
udev                  7.7G     0  7.7G   0% /dev
tmpfs                 1.6G  1.4M  1.6G   1% /run
/dev/mapper/pve-root   94G   79G   11G  88% /
tmpfs                 7.7G   46M  7.7G   1% /dev/shm
tmpfs                 5.0M     0  5.0M   0% /run/lock
efivarfs              128K   86K   38K  70% /sys/firmware/efi/efivars
/dev/sdb1             458G   75G  360G  18% /mnt/usb-backup
/dev/sda2             511M  336K  511M   1% /boot/efi
/dev/fuse             128M   20K  128M   1% /etc/pve
tmpfs                 1.6G     0  1.6G   0% /run/user/0

This is from "du -sm /var/lib/vz/*":

1       /var/lib/vz/dump
1       /var/lib/vz/images
1       /var/lib/vz/private
1       /var/lib/vz/snippets
1       /var/lib/vz/template

This is from "/etc/pve/storage.cfg":

dir: local
        path /var/lib/vz
        content vztmpl,iso
        shared 0

lvmthin: local-lvm
        thinpool data
        vgname pve
        content images,rootdir

dir: usb-backups
        path /mnt/usb-backup
        content backup
        prune-backups keep-all=1
        shared 0

I also attached a screenshot from the storage graph of my local - here you can see the raising of the storage usage within the three last days. The backups from this time are also missing, last backup on my external harddrive was from the 18.05.2024.

Thanks for any advice - you are an amazing community!

Greets

Daniel

 

[unnaDmed]

This solved my problem:
https://forum.proxmox.com/threads/b...ve-mnt-backups-disconnects.133914/post-591010