1. R

    CEPH in PVE 7.3 can not working with RDMA/RoCE?

    Hi, PVE geekers: I build a pve cluster on three server (with ceph), with pve & ceph package version as follow: root@node01:~# pveversion pve-manager/7.3-3/c3928077 (running kernel: 5.15.74-1-pve) root@node01:~# ceph --version ceph version 16.2.13 (b81a1d7f978c8d41cf452da7af14e190542d2ee2)...
  2. M

    Mellanox - iSER/Ceph

    I'm having issues getting iSER over iWARP to work properly with Proxmox (or any other OS) and Intel has been very unhelpful about getting me to a working state. I do need the added bandwidth/latency offered by RDMA; so I'm looking into alternatives. I'm making this post to get any input from...
  3. H

    Mellanox Need help setting up Infiniband

    Hi, I have 2x ConnectX-2 Cards and need to setup infiniband, RDMA over NFS on Proxmox 5.4. I followed a few guides online but I'm unable to change protocol type from ETH to IB. What is correct way of setting the port type on Proxmox?
  4. elurex

    Ceph blustore over RDMA performance gain

    I want to share following testing with you 4 PVE Nodes cluster with 3 Ceph Bluestore Node, total of 36 OSD. OSD: st6000nm0034 block.db & block.wal device: Samsung sm961 512GB NIC: Mellanox Connectx3 VPI dual port 40 Gbps Switch: Mellanox sx6036T Network: IPoIB separated public network &...
  5. G

    ceph performance 4node all NVMe 56GBit Ethernet

    Hi Folks, Whis performanche shoud i expect for this cluster ? are my settings ok ? 4 nodes: system: Supermicro 2028U-TN24R4T+ 2 port Mellanox connect x3pro 56Gbit 4 port intel 10GigE memory: 768 GBytes CPU DUAL Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz ceph: 28 osds 24 Intel Nvme 2000GB...
  6. M

    RoCE / RDMA for Ceph and Proxmox

    Hello there ! Could you tell if it is possible to leverage RDMA / RoCE with Ceph Jewel (and for Proxmox cluster communications too ideally...) ? If yes, is there some documentation available for the configuration of the hosts and the switches ? Thanks.
  7. A

    RDMA transport for Ceph using Accelio

    Has anyone managed to implement accelio for a Ceph cluster in Proxmox? I've been toying with this idea for some time as it has the potential to have a much lower overall latency then using IPOIB. I'd be really interested in hearing about your experience and potential pitfalls
  8. A

    SCST / ib_srpt / isert-scst in releases ?

    Hello, is there any change to get SCST together with srpt and the required kernel-modules in the releases ? Current scst compiles fine against Kernel 4.4.6 and in-tree ofed / rdma packages. Having those packages / kernel modules in the release would skip the need to compile / build with each...


The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!