New SAN - Dell PowerVault MD3200i

[hotwired007]

We've just bought a new Storage server a Dell MD3200i - with iSCSI and 2 controllers with 4 1GB NICs - there are a few ways that i can configure this and i'm trying to figure out the best way. Has anyone used a MD3200i with Proxmox yet?

I have 1 48 port switch currently alothough i will be purchasing a second one soon. I plan on having the active ports plugged into Switch 1 and the backup ports plugged into switch 2.

I currently have 5 Proxmox host servers - each has 2 NICs - these are bonded in Active/Backup mode.

My current storage server, a ReadyNAS 2100 has 2 NICs bonded in Active/Backup mode, this is starting to struggle with some of the IO we are throwing at it and we are trying to eliminate SPoF (i know only one switch ) as it only has one PSU - we've had issues with UPS brownouts in teh past - now replaced and upgraded models. (Eventaully i'll get round to documenting my entire hardware and Proxmox setup!)

As there are 8 NICs available on the MD3200i, i want to use them so that i have 4 1GB links with Active/Backup mode.

I have looked at this - http://pve.proxmox.com/wiki/ISCSI_Multipath - is there anything else i need to bear in mind?

Secondly - i had planned on installing the MD3200i with my test cluster so that i could test the Multipath setup - i'm guessing if i gave both of the NICs in my ReadyNAS 2100 separate IPs i could do the same without too much extra configuration but this will break the current iSCSI connections? although it might be an idea to install on my test cluster first so that i can learn what goes wrong and let you guys know!

My primary cluster is in production and i really dont want to break it!

 

[hotwired007]

Ok so i've installed the MD3200i with my test cluster, 3 Poweredge 860s, its switched on and using the same IP range as the rest of my network, each controller port is configured for an individual IP address ie:

Controller 0 Port 0 - 192.168.1.82
Controller 1 Port 0 - 192.168.1.83
Controller 0 Port 1 - 192.168.1.84
Controller 1 Port 1 - 192.168.1.85
etc

ive configured the disks (6x 1TB SAS) as one RAID5 disk.

Next step is to configure proxmox to talk to it and add the multipath - i'm guessing that the multpath tools will need installing on all of my cluster nodes?

 

[tom]

yes, on all nodes. but you as mentioned on the wiki, you need to configure separate network paths, looks like you configure all controller in the same IP network.

 

[hotwired007]

why do they need separate network paths? will i need to configure an iSCSI connection for all of the IPs in Proxmox?

 

[tom]

that's the basic idea of multipath. if you configure all your path using the same network your will see no performance/redundancy improvements - test it.

just follow the wiki article, everthing is there.

if you cannot work it out, I suggest you go for a commercial support subscription (with ssh remote assistance) for direct help (I assume you don´t buy a Dell SAN for home use).

 

[hotwired007]

thanks for the advice, the issue i have is that i dont have enough ports on my server to dedicate 2 to iSCSI and as the IO per server is actually quite low the bandwidth isnt an issue at the moment, thats why the MD3200i was configured to the same IP ranges, if i set them to the defaults (outside my range) i cannot access the MD3200i. this is my first attempt at setting up a multipathed SAN and i doubt i'll get it right first time - im quite happy for it to break on my test cluster, i just need to find the best way of making it work.

although the wiki article is well written it doenst answer some of my questions:

Please configure your iSCSI storage on the GUI if you have not done that already ("Datacenter/Storage: Add iSCSI target").

Do i need to configure an iSCSI connection in Proxmox for every IP address on the MD3200i?

Install multipath tools

Hence why i asked if it has to be done on every node.

 

[xldata]

Having all interfaces in the same subnet is possible if you use ifaces, that way you can bind a connection/session to an interface and use all interfaces available. Without it all traffic will be routed over the default interface for that specific network... Don't know if proxmox is using ifaces when configuring iscsi... else you have to do this manually... There is by the way a MD specific rdac driver for multipathing you should use... Follow up the advices for using RDAC from Dell...

 

[dietmar]

Why to setup multipath when there is only one network interface?

 

[xldata]

Why to setup multipath when there is only one network interface?

There might be one for each host but there are multiple on the storage... When one of those fails you still have available paths...

 

[dietmar]

There might be one for each host but there are multiple on the storage... When one of those fails you still have available paths...

Ah, I see.