I recently found out (kinda lived under a Rock in that Regards 
) that the NIC I was using (Mellanox ConnectX-2 and ConnectX-3) do NOT Support ASPM and therefore the CPU will never be able to achieve high Power Saving States (anithing above C2/C3 IIRC). Same goes for my preferred HBA, LSI 9211-8i / 9107-8i / etc (including the more recent 9300-16i/9305-16i/...).
For Ethernet, I ordered a few Intel X710-DA2 (2 x 10 gbps) which should support ASPM. I will also give the Intel XXV710-DA2 (2 x 25 gbps) and Mellanox ConnectX-4 CX4121A (2 x 25 gbps) a try. No idea about the Intel XXV710-DA2 and ASPM Support besides ONE single Forum Post I found on the Internet, while the Mellanox ConnectX-4 (hopefully also this specific one) should somewhat partially support ASPM, although definitively not the highest Power Saving States like the Intel.
I was trying to assess whether to go Infiniband if nothing just for some Tests (I currently have a Voltaire 4036 which is super loud and Power Hungry but should do for some Tests). I currently have some Voltaire HCA700Ex2 which is just a Mellanox Connectx-2 HCA and surely that does NOT support ASPM. Neither do the more recent Mellanox ConnectX-5 and Mellanox ConnectX-6 from what I researched.
In the Mellanox Camp, the ConnectX-4 seems to be the only one that somewhat partially supports ASPM. I couldn't find any Info on Intel Infiniband HCA.
In some eBay search the Intel XXV710-DA2 (same as the Model I bought) came up, but strangely they state that it supports Infiniband. But it does NOT support RDMA / ROCE.
Does anybody have some Reccomendation for Infiniband HCA ? And what is your Experience with ASPM ?
Thanks
) that the NIC I was using (Mellanox ConnectX-2 and ConnectX-3) do NOT Support ASPM and therefore the CPU will never be able to achieve high Power Saving States (anithing above C2/C3 IIRC). Same goes for my preferred HBA, LSI 9211-8i / 9107-8i / etc (including the more recent 9300-16i/9305-16i/...).For Ethernet, I ordered a few Intel X710-DA2 (2 x 10 gbps) which should support ASPM. I will also give the Intel XXV710-DA2 (2 x 25 gbps) and Mellanox ConnectX-4 CX4121A (2 x 25 gbps) a try. No idea about the Intel XXV710-DA2 and ASPM Support besides ONE single Forum Post I found on the Internet, while the Mellanox ConnectX-4 (hopefully also this specific one) should somewhat partially support ASPM, although definitively not the highest Power Saving States like the Intel.
I was trying to assess whether to go Infiniband if nothing just for some Tests (I currently have a Voltaire 4036 which is super loud and Power Hungry but should do for some Tests). I currently have some Voltaire HCA700Ex2 which is just a Mellanox Connectx-2 HCA and surely that does NOT support ASPM. Neither do the more recent Mellanox ConnectX-5 and Mellanox ConnectX-6 from what I researched.
In the Mellanox Camp, the ConnectX-4 seems to be the only one that somewhat partially supports ASPM. I couldn't find any Info on Intel Infiniband HCA.
In some eBay search the Intel XXV710-DA2 (same as the Model I bought) came up, but strangely they state that it supports Infiniband. But it does NOT support RDMA / ROCE.
Does anybody have some Reccomendation for Infiniband HCA ? And what is your Experience with ASPM ?
Thanks