additional NICs - recommendations or advice?

[hotwired007]

I'm looking at adding some additional NICs into my Dell R415, PE1950s and 860s to work with my SAN, does anyone have any recommendations for intel NICs?

Ive been looking at the intel i340 and i350 PCI-e range of cards, is it worth looking at a quad instead of a dual for the price difference?

My iSCSI SAN has 2 banks of 4x 1GB NICs (Active/Active controllers (MD3200i))

Any advice/experience/recommendations?

 

[mir]

If your SAN has 4 unused NIC's then I would go for the quad card. If HA is the goal then make 2 bonds consistent of 2 nic's and configure a multipath connection to your SAN*). If HA is not desired then assemble all 4 nic's into a single bond and you would theorectically have 8000 Mb/s 4000 up and 4000 down.

*) A pedantic person would say 2 dual instead of 1 quad

 

[tom]

why do you recommend a bond AND multipath? and what bond mode do you use here to get 4000Mb/s?

 

[mir]

why do you recommend a bond AND multipath? and what bond mode do you use here to get 4000Mb/s?

2xbond mode balance-rr.

"balance-rr This mode is the only mode that will permit a single TCP/IP connection to stripe traffic across multiple interfaces. It is therefore the only mode that will allow a single TCP/IP stream to utilize more than one interface's worth of throughput. This comes at a cost, however: the striping often results in peer systems receiving packets out" (http://www.linuxfoundation.org/collaborate/workgroups/networking/bonding)

 

[tom]

this does not really answer my question: why do you recommend a bond AND multipath?

did you ever tested SAN with balance-rr and multipathing, getting 4000 MB/s?

 

[mir]

The last one first. I haven't tested balance-rr and multipathing but I have testet balance-rr with 2 nics. Overall the performance was something in-between the bandwidth of 1 and 2 nics. The reason for not gaining full performance is out-of-order packages which as throughput increases also increases.

Why I recommend bond and multipath is related to balance-rr problems with out-of-order packages which I expect is solved using to separate bonds configured in a multipath setup. All other bonding mode will not let you stripe package flow over more than 1 nic so the trade-off is: Do you expect many concurrent connections, go for 802.3ad and a bond with all nics. Do you however, expect to have a traffic flow with few but very throughput demanding connections balance-rr is a god option. Due to out-of-order packages the multipath is recommended for HA.