Route traffic to SAN

[titouan]

Hi,

I have recently installed a 3 node cluster with network storage for my VMs. I have juste added SFPs nic in my nodes to handle the traffic between my servers and my SAN.
The problem is that I can't seem to route the traffic intended to the SAN via my SFP cards.
How would you do it ?
Here is the configuration for my nic (bond0 is my LAN, bond1 is intented to administrate the proxmox, bond2 is for my DMZ VMs and bond3 is for my SAN):

auto lo
iface lo inet loopback


iface eth0 inet manual


iface eth1 inet manual


iface eth2 inet manual


iface eth3 inet manual


iface eth4 inet manual


iface eth5 inet manual


iface eth6 inet manual


iface eth7 inet manual


auto bond1
iface bond1 inet manual
        slaves eth2 eth3
        bond_miimon 100
        bond_mode 802.3ad
        bond_xmit_hash_policy layer2+3


auto bond0
iface bond0 inet static
        address  10.37.4.2
        netmask  255.255.128.0
        gateway  10.37.0.1
        slaves eth0 eth1
        bond_miimon 100
        bond_mode 802.3ad
        bond_xmit_hash_policy layer2+3


auto bond2
iface bond2 inet manual
        slaves eth4 eth5
        bond_miimon 100
        bond_mode 802.3ad
        bond_xmit_hash_policy layer2+3


auto bond3
iface bond3 inet manual
        slaves eth6 eth7
        bond_miimon 100
        bond_mode 802.3ad
        bond_xmit_hash_policy layer2+3


auto vmbr0
iface vmbr0 inet manual
        bridge_ports bond1
        bridge_stp off
        bridge_fd 0


auto vmbr1
iface vmbr1 inet manual
        bridge_ports bond2
        bridge_stp off
        bridge_fd 0


auto vmbr2
iface vmbr2 inet static
        address  10.37.4.8
        netmask  255.255.128.0
        bridge_ports bond3
        bridge_stp off
        bridge_fd 0

I've tried to add a route like this :

ip route add 10.37.4.10 dev vmbr2

but it didn't do anything
and I've tried an iptables route which didn't do more :

iptables -t nat -A POSTROUTING -d 10.37.4.10/17 -o vmbr2

I hope some of you can help !
Thanks for advance

 

[pirateghost]

Based on a quick look at your IPs, it appears that both of your gateways are in the same subnet. Why would you need 2 gateways within the same subnet? I see no reason to route SAN traffic over anything.

 

[titouan]

Based on a quick look at your IPs, it appears that both of your gateways are in the same subnet. Why would you need 2 gateways within the same subnet? I see no reason to route SAN traffic over anything.

Hi pirateghost, thank's for your answer

The reason the 2 card are in the same subnet is that when we put the server together we only had the 1G card to serve traffic to SAN and traffic to the LAN/Internet. So I can't change my SAN subnet (which would be really easier I reckon) because it would imply to break my LVM/iSCSI setup on which are stored all of my VMs. So I need my 10G card to be on se same subnet as my SAN because I can't change its IP address. And I can't change my entire LAN subnet for obvious reason

 

[pirateghost]

Hi pirateghost, thank's for your answer

The reason the 2 card are in the same subnet is that when we put the server together we only had the 1G card to serve traffic to SAN and traffic to the LAN/Internet. So I can't change my SAN subnet (which would be really easier I reckon) because it would imply to break my LVM/iSCSI setup on which are stored all of my VMs. So I need my 10G card to be on se same subnet as my SAN because I can't change its IP address. And I can't change my entire LAN subnet for obvious reason

But why would you need to route out if the traffic is in the same subnet?

You don't need a gateway for that card

 

[titouan]

But why would you need to route out if the traffic is in the same subnet?

You don't need a gateway for that card

I want my first 1G card to handle only the network traffic from the VMs and my 10G card to only handle the traffic between the server and the SAN (where are my VMs hard drive stored)
If I don't configure anything, everything pass through the 1G card. So how do I tell my server whn to use which card ?

 

[pirateghost]

You separate the subnets, and use direct IP to IP communication between San and server.

Your design is broken. Fix it.