Prox Cluster Shared NFS Storage Locking ext4 vs xfs storage format

[Hubguru]

Hey All,

Ran into strange scenario, I started with using shared NFS storage with the storage disks formatted with XFS, was seeing NFS locking timeouts with large file transfers and VM disk creations using qcow2, 20 Gig and above. I spun up another NFS storage node and formatted storage with EXT4 instead and found creating 40Gig VM Disks was instant, the disk file on the storage node showed correct file size but the disk is sparse, meaning empty.

So my question is what is the difference with xfs and ext4 file system when using NFS, does proxmox natively use thin provisioning for ext4 and thick for xfs? Not sure why this would be different or is this not expected behavior?

Any incite will be helpful.

Thanks.

JR

 

[Alwin]

So my question is what is the difference with xfs and ext4 file system when using NFS, does proxmox natively use thin provisioning for ext4 and thick for xfs? Not sure why this would be different or is this not expected behavior?

Proxmox does not see the underlying filesystem of the NFS server.

You tested two different servers, with two different filesystems, if they do not have the exact same hardware, then it is not really comparable. Utilisation, network traffic, configuration can all lead to different results, regardless the filesystem used.

 

[Hubguru]

Proxmox does not see the underlying filesystem of the NFS server.

You tested two different servers, with two different filesystems, if they do not have the exact same hardware, then it is not really comparable. Utilisation, network traffic, configuration can all lead to different results, regardless the filesystem used.

Same server type, switches, networks, disks, only change was formatting the storage in ext4 instead of xfs. There was some obscure post I read on the forums here, can't seem to find it now, another prox user had similar results, change storage format to ext4 from xfs and eliminated the NFS locking they were seeing.

So I think there is something going on with NFS protocol not necessarily with proxmox. Maybe because xfs journaling is bigger than ext4 or something like that. Another note, performance really has not changed, I'm seeing the same network speeds and such with same file sizes, but not seeing the NFS locking timeouts. I'm going to run with ext4 format for storage going forward, just thought it was interesting and wanted to see if anyone had insight on why this is happening.

Thanks.

JR

 

[Alwin]

Hm... you could try to disable subtree_check, that sometimes may cause trouble. I never experienced lockups on XFS, maybe someone else on the forum has some insight to this.