ext4 filesystem and other disk optimizations

[gkovacs]

Last year I've posted a thread including some suggested disk optimizations to the next major version of Proxmox VE, and the Dietmar suggested they will be added to the 2.0 todo list:
http://forum.proxmox.com/threads/38...optimizations-for-the-next-version-of-Proxmox

From the thread the most important optimization would be the option to select the ext4 filesystem when installing PVE. Second most important would be the fdisk partition alignment for SSD boundaries, because it is not modifiable later.

I have just downloaded and installed PVE 2.0 and it still installs ext3 without asking, which is suboptimal for many reasons, even though 2.6.32 includes support for ext4.

Why do I think we need ext4? Many reasons:
- significantly faster IO:
http://www.phoronix.com/scan.php?page=article&item=ext4_benchmarks&num=3
- much faster fsck (which on ext3 can easily take 1-2 hours on a couple hundred GB PVE node, which is downtime)
http://en.wikipedia.org/wiki/File:E2fsck-uninit.svg
- much faster with large number of files and directories (would speed up vzdump and vzmigrate)
- online defragmentation
- larger maximum filesystem and file sizes
- much less fragmentation
- more safe against filesystem errors

Is it possible to have this feature included in the 2.0 installer?
I would volunteer for testing and benchmarking different filesystems.

 

[tom]

Ext4 is a new option, just read the quick install guide.

Sent from my GT-P1000 using Tapatalk

 

[gkovacs]

Tom, this is great news!

One more question: since the stable PVE 1.9 comes with the same kernel, does it support ext4 for a new installation as well?

 

[tom]

Tom, this is great news!

One more question: since the stable PVE 1.9 comes with the same kernel, does it support ext4 for a new installation as well?

No, we do not have or plan this for PVE 1.9.

We did a lot performance tests (single disk, raid volumes) with ext3 and ext4 and as a result of these tests we decided to go for ext3 as default.

 

[_ED]

I'm resurrecting this old post with some stuff

I'm setting up a new proxmox machine with PX 2.1, and if all will be ok I'll migrate the "old" 9 machines (reinstall not upgrade).

What we know is that ext3 is to slow in fsck with TB of data so we can't/want use it. 2.6.32 support in a decent way ext4.

On my machine I did some test .

• 2 Intel Xeon E5606
• ram 24GB
• 6 HD HITACHI (HUA722010CLA330) of 1TB in RAID 5 (100GB + 3.8TB) + 1 spare
• controller RAID: Adaptec 5805

some DOC:

• noatime - Don't update inode access times on the filesystem. Can help performance (see atime options).
• nodiratime - Do not update directory inode access times on the filesystem. Can help performance (see atime options).
• relatime - Update inode access times relative to modify or change time. Access time is only updated if the previous access time was earlier than the current modify or change time. (Similar to noatime, but doesn't break mutt or other applications that need to know if a file has been read since the last time it was modified.) Can help performance (see atime options).

noatime already includes nodiratime. You do not need to specify both options.

I think to use some *atime mix, relatime not block mutt and give good results in term of performance without use "esoteric" parameters like nodelalloc

Any comment or suggestion is appreciated

ext3
mount /dev/sdb1 /mnt

CPU BOGOMIPS: 34218.24
REGEX/SECOND: 856437
HD SIZE: 356.71 GB (/dev/sdb1)
BUFFERED READS: 442.54 MB/sec
AVERAGE SEEK TIME: 8.32 ms
FSYNCS/SECOND: 2325.07
DNS EXT: 71.97 ms
DNS INT: 5.77 ms (enginsoft.it)


ext4
mount /dev/sdb1 /mnt

CPU BOGOMIPS: 34218.24
REGEX/SECOND: 866316
HD SIZE: 356.71 GB (/dev/sdb1)
BUFFERED READS: 457.60 MB/sec
AVERAGE SEEK TIME: 8.47 ms
FSYNCS/SECOND: 1916.36
DNS EXT: 65.42 ms
DNS INT: 18.94 ms (enginsoft.it)


ext4
mount /dev/sdb1 -o defaults,noatime /mnt

CPU BOGOMIPS: 34218.24
REGEX/SECOND: 837154
HD SIZE: 356.71 GB (/dev/sdb1)
BUFFERED READS: 507.85 MB/sec
AVERAGE SEEK TIME: 8.34 ms
FSYNCS/SECOND: 2567.53
DNS EXT: 49.41 ms
DNS INT: 18.99 ms (enginsoft.it)


ext4
mount /dev/sdb1 -o defaults,relatime /mnt

CPU BOGOMIPS: 34218.24
REGEX/SECOND: 833172
HD SIZE: 356.71 GB (/dev/sdb1)
BUFFERED READS: 509.70 MB/sec
AVERAGE SEEK TIME: 8.13 ms
FSYNCS/SECOND: 2573.66
DNS EXT: 47.90 ms
DNS INT: 24.18 ms (enginsoft.it)


ext4
mount /dev/sdb1 -o defaults,nodelalloc /mnt
root@pxfi00:~# pveperf /mnt
CPU BOGOMIPS: 34218.24
REGEX/SECOND: 853649
HD SIZE: 356.71 GB (/dev/sdb1)
BUFFERED READS: 471.67 MB/sec
AVERAGE SEEK TIME: 8.65 ms
FSYNCS/SECOND: 2583.33
DNS EXT: 65.76 ms
DNS INT: 5.93 ms (enginsoft.it)


ext4
mount /dev/sdb1 -o defaults,nodelalloc,relatime /mnt
CPU BOGOMIPS: 34218.24
REGEX/SECOND: 839405
HD SIZE: 356.71 GB (/dev/sdb1)
BUFFERED READS: 498.40 MB/sec
AVERAGE SEEK TIME: 8.62 ms
FSYNCS/SECOND: 2082.76
DNS EXT: 68.57 ms
DNS INT: 17.32 ms (enginsoft.it)


ext4 (Proxmox Staff)
mount /dev/sdb1 -o rw,relatime,barrier=1,nodelalloc,data=ordered /mnt
CPU BOGOMIPS: 34218.24
REGEX/SECOND: 839571
HD SIZE: 356.71 GB (/dev/sdb1)
BUFFERED READS: 506.94 MB/sec
AVERAGE SEEK TIME: 8.52 ms
FSYNCS/SECOND: 2658.29
DNS EXT: 44.09 ms
DNS INT: 5.39 ms (enginsoft.it)