10GbaseT Switch Recommendation

[Gurn_Blanston]

Greetings,

We just had a weird event happen when shutting down our third "quorum" host, which is being replaced. We use IPoIB on some old Mellanox IB cards and a Mellanox IB switch. The result was that one of the two remaining hosts went into kernel panic and had to be powercycled. In order to have faster VM migrations, we moved the cluster network over to IB several months ago, which I understand is no longer necessary as of PVE 4.4. At any rate, this sudden and unexpected outage has resulted in enough of a human panic to justify scrapping IB for 10GBaseT. The reason we were on IB in the first place is that 10G switches are super expensive, at least the managed ones from the likes of Dell/Cisco, etc.

Are any of you out there using the cheaper Netgear switches? Such as
NETGEAR ProSAFE 8-Port 10-Gigabit Ethernet Web Managed Switch (XS708E-200NES)

This switch costs around $700 USD and we could make due with 8 ports. I have found mixed reviews regarding cheap 10G switches. Any positive or negative experiences and with what hardware?

Regarding NICs, we have had success with Intel X540T2 to connect two hosts without any switch so I imagine we will stick with those adapters.

Thanks!

 

[tschanness]

The small Netgears are not Stackable (or at least: were). Apart from that I found them really easy to use perform quiet good.

Jonas

 

[tom]

A bit more expensive, Cisco SG350XG-2F10 12p 10GBase-T Stack Switch - we use one in our testlab, works very good.

We also got an cheaper Netgear since two year, also working without problems but this one is quite noisy.

 

[Nikolay]

We bought JetStream 12-Port 10GBase-T + 10 SPF+

http://www.tp-link.com/us/products/details/cat-40_T1700X-16TS.html

It works pretty good for last half of year.

 

[Gurn_Blanston]

Thank you all for your time and attention! We splurged and went with

NETGEAR ProSAFE 8-Port 10-Gigabit Ethernet Smart Managed Switch (XS708T-100NES)

It was around $1000 USD and has the word "managed" in the description. There is no serial console so it isn't as managed as you might hope but so far it is able to keep up with our spindle based zpools. We don't have anything that can write steadily at more than 500 MB/s but it seems to handle that easily. It is a single point of failure and has only a single power supply so we don't push cluster traffic on it. It is primarily used to connect our three hosts to an old re-purposed Precision T7500 stuffed with RAM and cheap 5 TB disks for backups.

I think it is worth mentioning that all of our NFS timeouts and surginess has gone away since abandoning Infiniband. I don't know if it was due to our old eBay IB hardware or driver problems but we never did get IPoIB to run in a trustworthy fashion.

Thanks again for the advice,

GB