Hi,
Which filesystem is best for live migration and HA support in a Proxmox cluster: ext4, XFS, or ZFS?
Thank you!
Which filesystem is best for live migration and HA support in a Proxmox cluster: ext4, XFS, or ZFS?
Thank you!
Which file system is the best
- Thread starter powersupport
- Start date
I think you should specify something more about your intended use, goals, hardware available (or planned to get).
What would be best to use may vary based on these things, and they may also give you advice from other parties regarding clusters.
You need to think carefully about cluster choices and design, and I recommend you get enough experience with what you'll be using before having something in production (maybe on many servers).
What would be best to use may vary based on these things, and they may also give you advice from other parties regarding clusters.
You need to think carefully about cluster choices and design, and I recommend you get enough experience with what you'll be using before having something in production (maybe on many servers).
We are planning to integrate a SAN lun to proxmox as iscsi storage, there are not much documentation available for the same, in some documentation I see using OCFS2 file system and ZFS, but OCFS2 is not officially supported by proxmox and zfs is not ideal for cluster, so we looking to other options and client likes to have xfs or ext4, but in some only forums I see both doesn't support HA and live migrations,so which is the best file system for the setup
None of these is suitable for Shared Storage usage.
Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
ZFS over iSCSI is possible for shared storage, but requires a ZFS capable remote storage appliance, as outlined here.
You are correct that "ZFS over iSCSI" is an option. However, OP asked about filesystems and "ZFS over iSCSI" is not a filesystem but rather a combination of technologies. If one was so inclined they could create EXT4+QCOW over iSCSI with the same end result.
Part of OP's inquiry relates to HA, so one would need the ZoI appliance to be HA capable itself. The resources needed to properly implement an iSCSI appliance (regardless of the file system its backed by) are high.
It is also unlikely that exposing an HP Alletra RAID LUN and overlaying it with ZFS will lead to satisfactory results.
Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
Last edited:
One of the other 4-5 threads OP has opened today
PS
I fixed Alletra spelling in my prior reply
Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
Each of the 3 mentioned filesystems is suitable for shared storage with nfs access while if doing the nfs in ha is desireable for 30 pve nodes.
In theory, yes.
In practice, if I was forced to implement a standalone HA NFS appliance with shared block backend I would not use EXT4. In all 3 cases this would have to be an Active/Passive cluster, channeling 30-node traffic to a single server. Frankly, at this point OP is better advised to buy a proper HA NFS appliance.
Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
I think we are on the same page.
It was not one of my suggestions to use either external NFS or ZFS/iSCSI, as each scheme requires external servers and is only capable of using one server at a time.
There is a reason Isilon exists and Dell/EMC bought it.
Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
Btw. I had setup an PowerScale (=Isilon) in the past once for file serving and when checking connection and pull eth cable or a power plug
surprisingly there was not missing 1 ping between nfs client and the ha-nfs service while doing so. That was quiet impressive ...
Write performance was good too too but read was really bad and not as expected, about just a quarter. And it's quiet expensive but yeah, ha isn't cheap.
surprisingly there was not missing 1 ping between nfs client and the ha-nfs service while doing so. That was quiet impressive ...
Write performance was good too too but read was really bad and not as expected, about just a quarter. And it's quiet expensive but yeah, ha isn't cheap.