RDMA Requirements for Proxmox with ZFS and Third-Party iSCSI Storage

[Rajan_Kumar.Jaiswal@sita.]

Hi All,

We are currently evaluating the network switch requirements for a new Proxmox deployment and would like to understand whether RDMA support is required from the Proxmox or storage perspective.

Our planned environment is:

• Proxmox VE
• Local ZFS storage on some clusters
• Third-party shared storage using iSCSI on other clusters
• Standard Kubernetes/VM workloads

We have a few questions regarding RDMA and switch selection:

1. Does Proxmox VE itself make use of RDMA for any of its core features (live migration, clustering, ZFS, etc.)?
2. For local ZFS storage, is there any benefit or requirement to use RDMA-enabled NICs?
3. For third-party iSCSI storage, does RDMA depend entirely on the storage vendor (iSER/RoCE/iWARP support), or does Proxmox provide any native RDMA integration?
4. Is RDMA a supported and commonly deployed configuration with Proxmox VE, or is standard Ethernet networking the recommended approach?
5. If RDMA is used with iSCSI or another storage solution, would you recommend selecting switches that support DCB features such as PFC, ETS, and DCBX?
6. Has anyone successfully deployed Proxmox with RDMA-enabled networking? If so, which NICs, storage platform, and switch models are you using?

We are currently evaluating Cisco switch options. The Catalyst C9300 does not support the complete DCB feature set required for RoCE. Before selecting our switching platform, we would like to understand whether RDMA is a practical requirement for a Proxmox deployment or only for specific storage solutions.

Any recommendations or best practices would be greatly appreciated.

 

[Francois Baillargeon]

Hey @Rajan_Kumar.Jaiswal@sita.

To the best of my knowledge and documentation, RDMA is not supported by open-iscsi at this time.

Unless you have extremely low latency requirements or super high throughput, it's not really a real improvements.

You might get the performance you want out of plain iSCSI queue tuning, jumbo frame and the other usual low hanging fruit like that.