File system in VM

[William Edwards]

I am trying to decide between using XFS or EXT4 inside KVM VMs. My goal is not to over-optimise in an early stage, but I want to make an informed file system decision and stick with that.

Situation:

• Ceph as backend storage
• SSD storage
• Writeback cache on VM disk
• No LVM inside VM
• CloudLinux 7 OS inside VM (CentOS 7 derivative)
• VM disk sizes range from 256 GB to 1 TB
• Many small files (use case: web hosting)

I am mostly taking reliability and robustness into account. I am aware both XFS and EXT4 are stable, but I can imagine one is more suitable for my use case than the other.

I have performed some testing, and all I can conclude is that EXT4 seems to reserve more space for internal use; hence my question here.

Which file system is more suitable?

 

[t.lamprecht]

I am mostly taking reliability and robustness into account. I am aware both XFS and EXT4 are stable, but I can imagine one is more suitable for my use case than the other.

For reliability and robustness I'd say that both are equal, which means both are really stable and rock solid.

Many small files (use case: web hosting)

XFS is a bit more flexible with a very high file count but less efficient with small files. Each file has a inode with the basic metadata in it. Checkout the df -ih command to see the current inode usage.
Both filesystems calculate their initial inode number availability depending on how much spaces the underlying volume provides.
But once created ext4 has a fixed maximal number of possible inodes. One can make XFS "maximal INode space percentage" grow, as long there's enough space. So XFS is a bit more flexible for many inodes.

ext4 is a bit more efficient with small files as their default metadata size is slightly smaller.

There are opinions that for:

• large files + multi threaded file access -> XFS
• smaller files + single threaded -> ext4

Honestly, it totally depends on the workload and the effects will probably only show with lots of data/files/work being done.
At the end this is only a very rough guideline.

I can conclude is that EXT4 seems to reserve more space for internal use

This is mostly the "5% for root, + inode spaces" being reserved.

You can tune the 5% for root, to much less if you want, e.g. to 1% with tune2fs -m 1 /dev/sdaX
See also https://www.redhat.com/archives/ext3-users/2009-January/msg00026.html for some rationale.

The inode space usage difference between XFS and ext4 will cancel out once you actually have a lots of file. They'll both require the space after all, ext4 is just pre allocated (IIRC) and XFS not.