[SOLVED] LXC runs out of inodes. Ways to avoid this without ZFS

P

Peronia

Active Member
Proxmox Subscriber
Aug 5, 2020
12
3
43
Hi together,

I have an LXC container (EXT4). I am currently testing NixOS. The container is working fine, no problems. But by design it deals with a lot of small files.
As a consequence I ran out of inodes. Sure I can add more space to the container and solve it this way, but this seems not ideal for me.

I tried to setup the same container with BTRFS. This works nice and my inode problem is solved. But actually Proxmox doesn't support quotas for BTRFS, so each container can use the full data volume. This is something that I don't want.

Now is my question: How can I avoid the inode problem? Is there any solution?
I don't plan to use ZFS since I don't have (and plan) a second SSD and it looks to some overkill for me.
 
How could each lxc use the full btrfs data volume as you give the lxc a disk with size ? And why you want inside this disk quotas ?
Anyway xfs has dynamically inode generation and even quotas which are done by kernel so any application even pve has to live with and cannot ignore by not support as at reaching limit there's stop :)
 
"here" is empty but anyway ... Yes, you can do a lxc with xfs:
First dnf install xfsprogs or apt install xfsprogs inside the lxc
stop lxc
cd .../images/<machine-id>
mkdir e x
cp -a <vmid>.raw <vmid>.raw.xfs
mkfs.xfs -f <vmid>.raw.xfs
mount -o loop <vmid>.raw e
mount -o loop <vmid>.raw.xfs x
cp -a e/* x/.
sync
umount e
umount x
rmdir e x
rm -f <vmid>.raw
mv <vmid>.raw.xfs <vmid>.raw
start lxc
 
Last edited:
Thanks waltar, this looks promising but also a bit hacky.
I red that xfs needs a priviledged container? Do you use xfs in an container and can say how stable it is?
Yeah I messed up with the link. I corrected it now.

Okay I tried it. My problem is that I can't find the dir where the .raw files exists. I found only /dev/pve but this aren't the .raw files.
 
Last edited:
Just did a test lxc which works fine but we mostly use vm's, just have few pihole and lxc as gateways:
root@deb1271xfsunpriv:~# mount
/mnt/pve/sim6_data/images/186/vm-186-disk-0.raw on / type xfs (rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota)
 
Haha, we use nfs file storage for images, and so it's has .raw and .qcow2 filename endings. If using block storage you need dd or first migrate lxc/vm to file storage space, convert and later migrate back to block storage space as the easy way instead of struggling with blocks.
 
Okay, I solved it this way:

Prerequisites:​

  • Container-ID: eg. 105
  • LVM-Storage: eg. Volume Group pve
  • Thin Pool: eg. data

Steps:​

  1. Install xfsprogs
  2. Stop container
  3. Create new XFS-Volume: lvcreate -V 10G -T pve/data -n vm-105-disk-xfs
    -T pve/data points to the thin pool
  4. Create filesystem: mkfs.xfs /dev/pve/vm-105-disk-xfs
  5. Mount both volumes:
    mkdir /mnt/ext4 /mnt/xfs
    mount /dev/pve/vm-105-disk-0 /mnt/ext4
    mount /dev/pve/vm-105-disk-xfs /mnt/xfs
  6. Copy data (incl. permissions & symlinks): rsync -aHAX --numeric-ids /mnt/ext4/ /mnt/xfs/
  7. Unmount: umount /mnt/ext4 /mnt/xfs
  8. Either:
    Adapt container config in /etc/pve/lxc/105.conf: rootfs: local-lvm:vm-105-disk-xfs
    or:
    Rename volume: lvrename pve vm-105-disk-0 vm-105-disk-ext4
    The following applies:
    pve your volume group
    vm-105-disk-0 current name
    vm-105-disk-ext4 new name
    Update symlinks: When you use /dev/mapper, you can use these commands to ensure that everything is recognized correctly:
    vgscan
    vgchange -ay
  9. Start container
  10. Optional: Delete old volume: lvremove /dev/pve/vm-105-disk-ext4

This works fine. I checked that I now have XFS. But one question: I read about I can't use things like xfs_admin in an unpriviledged LXC. Is that true? Need I still xfsprogs in such container?
 
Last edited:
  • Like
Reactions: JoaGamo and waltar
You don't need the bundle of xfs_* tools inside a lxc but the xfs driver if using xfs inside and maybe mkfs.xfs for further disks.
 
  • Like
Reactions: Peronia
Experimenting with anythink allways give new insides and mostly gets fun and as learning never ends in life ... so go on. But have a fast reliable storage under the hood as when the xfs inside thinks is has written data which isn't in reality that could lead to damage inside lxc.
 
You must log in or register to reply here.
Top Bottom