I have 2 Dell R730, 1cpu, 32ram, 6x ssd raid, and 3 different 10gb nics to choose from, going through a Nortel 5530 with 2x 10gb XFP, 9k frames, and switch ports VLAN'd (untagged) from all other ports.
iperf tests look great:
dd performance on the box locally is good:
But an scp from box to...