Excessive Inode Consumption and /etc/lvm/archive

[wits-zach]

Some of you have probably seen these threads below:

• https://forum.proxmox.com/threads/apt-get-update-autoremove-failed-no-space-left.91867/
• https://forum.proxmox.com/threads/an-help-to-free-inode-usage-on-my-servers.69730/

I am experiencing a similar problem - but not due to Samba. I have very, very high inode usage due to /etc/lvm/archive creating lots of metadata backups (in the form of .vg files). I only became aware of this because of my inability to update to 5.4.124-1-pve. I kept getting the same errors everyone did with Samba.

I managed to "fix it", and my solution is found below:

cd /
du --inodes -xS | sort -n #found the directory /etc/lvm/archive had 1M+ inodes consumded

#There were so many files in /etc/lvm/archive that bash was returning too many arguments errors, so I used the below to find the string of each VM's operating system to delete on encounter
 
cd /etc/lvm/archive
find . -iname 'OS1_*' -exec rm '{}' \;
find . -iname 'OS2-*' -exec rm '{}' \;
cp /etc/lvm/lvm.conf /etc/lvm/lvm.conf.bkp
nano /etc/lvm/lvm.conf
cat lvm.conf | grep retain
        # Configuration option backup/retain_min.
        retain_min = 10
        # Configuration option backup/retain_days.
        retain_days = 2

I say "fix" because I want to know at what cost. I changed it from 30 to 2 days. They are metadata backups, so I figure will be ok (there is a lot of creation/destruction of VM's), but I just want to make sure this doesn't make me pay a price later.

 

[Stoiko Ivanov]

du --inodes -xS | sort -n #found the directory /etc/lvm/archive had 1M+ inodes consumded

seems the result of the lot of VM creation/destruction...

many operations on volume-groups (e.g. lvm creation/resize/deletion (all of which happen if you create a vm, migrate it, move a disk, clone it) create such a config-archive...
Whether you need it or not depends on your use-case - we had a few cases where having the archive entries around helped us find bugs which were quite well hidden - otoh in the years I worked as sysadmin/system designer - I did not run once into the situation of needing data from there (not really a relevant sample size - just my experience)

but having multiple million lvm-config copies sounds a bit much to keep around for ever

I hope this helps!