Proxmox VE with infiniband - SuperMicro twin server

[check-ict]

I've been using Proxmox VE for internal tests and it works great. Now I want to setup a small redundant VPS hosting cluster.

My idea is to buy a 2u twin (2 nodes) from SuperMicro. This server will be available soon and it has an internal infiniband connection.

Is it possible to use DRDB with infiniband? If this is possible, I will only need the twin server without a 10 gigabit network card.

Also, what will DRDB do with the disk performance? I want to use 8 disks in RAID 10 (7200 rpm) on each node/server (16 disks total).

 

[gillesMo]

Hello,

DRBD is TCP/IP only, but I know it is possible to do TCP/IP over Infiniband, so it must be possible.

Concerning performance, with Infiniband you'll get good network latency but certainly less than a local drive...
Be sure to get a battery backed write cache with your RAID card, this is important for I/O perfs.
For the same reason, if you can have SAS 15k tr/min drives...

And please, I'm interested to have some feedbacks if you build your cluster and make some benchmarks. Keep us informed

 

[udo]

Hello,

DRBD is TCP/IP only, but I know it is possible to do TCP/IP over Infiniband, so it must be possible.

Concerning performance, with Infiniband you'll get good network latency but certainly less than a local drive...
Be sure to get a battery backed write cache with your RAID card, this is important for I/O perfs.
For the same reason, if you can have SAS 15k tr/min drives...

And please, I'm interested to have some feedbacks if you build your cluster and make some benchmarks. Keep us informed

Hi,
you can select some other "modi" like normal tcp/ip with drbd (e.g. "address ssocks 172.16.1.12:7788" for Dolphin-Nics with a socket-connection).
With infiniband i have yet no experiences (but please give feedback). But this post show's that infiniband work: http://lists.linbit.com/pipermail/drbd-user/2010-September/014764.html

Udo