1 Server, 2 NICs

[boca]

Hi all,

I'm trying to setup my server so that I have 1 bridge per network card.

So what I've done is:

NIC1 (eth0): 10.3.3.200/24 - Gateway 10.3.3.1 - vmbr0
NIC2 (eth1): 10.3.3.210/24 - vmbr1

This works fine, and I can ping both 10.3.3.200 and .210 without any issues.

What makes me confused is that when I run a ping command against 10.3.3.210 and then unplug the network card from that interface, it still works? Which indicates that all traffic is still flowing through eth0 (10.3.3.200)?

My question is; How can I setup Proxmox to exclusively use eth0 for 10.3.3.200 and eth1 for 10.3.3.210? or is this not possible?

Thanks!

 

[dietmar]

Do not assign an network address to vmbr1 (not necessary).

 

[vanDivX]

A while back I asked the same question. e100 replied that

"It likely only works because the IPs you assigned to A and B are in the same subnet and the kernel has forwarding enabled.
If they were different subnets I doubt it would work."

 

[boca]

Do not assign an network address to vmbr1 (not necessary).

Hi Dietmar,

I've tried your approach but it doesn't seem to work? Do you have any other suggestions I could try?




Either way, thank you!

 

[dietmar]

I've tried your approach but it doesn't seem to work?

You rebooted the node after that change?

 

[boca]

You rebooted the node after that change?

Yes. I've rebooted the server after applying those changes. I've also tried this on another (R610 Dell) server with the same results.

 

[snowman66]

Which indicates that all traffic is still flowing through eth0 (10.3.3.200)

This is ok because both IP's are in the same subnet/network. See output of command: route -n or ip route.

You need to assign IP 10.3.3.210 inside VM or maybe you need bonding?

http://pve.proxmox.com/wiki/Network_Model

 

[boca]

The VMs have an IP address assigned. I've tried Ubuntu and Windows, with both becoming unreachable if they're under vmbr1. And if i change them over to vmbr0 it works correctly. Very strange :/

 

[snowman66]

This doesn't work because there are no routes on eth1/vmbr1. In this case networking will only work between VMs on vmbr1 and without connectivity to/from the outside networks.

 

[boca]

Thanks snowman66, you're correct. route -n show only connectivity to vmbr0. I've tried adding some entries to vmbr1 but they don't work :/ Looking through the link (network models) it doesn't seem to relate to my scenario so I'm not sure how to go about getting vmbr1 VMs accessible from the outside

 

[boca]

And this is what I've added to routing table

 

[snowman66]

I'm not sure how to go about getting vmbr1 VMs accessible from the outside

Easy way:
1. Assign additional IP address on your router (or firewall): 10.3.4.1/24.
2. Connect eth1/vmbr1 to router (or firewall).
3. Boot VM and set the IP address: 10.3.4.100 and gateway 10.3.4.1
4. Enable port forwarding on your router (or firewall) for IP 10.3.4.100

 

[udo]

Hi,
Dietmar wrotes times ago: don't use an IP on the pve-host for vmbr1! It's makes no sense to use two addresses in the same subnet.
Simply use an interface-file like this:

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
    address 0.0.0.0
    netmask 0.0.0.0

auto eth1
iface eth1 inet static
    address 0.0.0.0
    netmask 0.0.0.0

auto vmbr0
iface vmbr0 inet static
        address  10.3.3.200
        netmask  255.255.255.0
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0
    gateway 10.3.3.1

auto vmbr1
iface vmbr1 inet manual
        bridge_ports eth1
        bridge_stp off
        bridge_fd 0

If you connect an VM to vmbr1 it's connected with the same natwork, where eth1 is connected - the host has nothing to do with that! Of course, you must select bridged network.

Udo

 

[udo]

Hi all,

I'm trying to setup my server so that I have 1 bridge per network card.

So what I've done is:

NIC1 (eth0): 10.3.3.200/24 - Gateway 10.3.3.1 - vmbr0
NIC2 (eth1): 10.3.3.210/24 - vmbr1

This works fine, and I can ping both 10.3.3.200 and .210 without any issues.

What makes me confused is that when I run a ping command against 10.3.3.210 and then unplug the network card from that interface, it still works? Which indicates that all traffic is still flowing through eth0 (10.3.3.200)?

My question is; How can I setup Proxmox to exclusively use eth0 for 10.3.3.200 and eth1 for 10.3.3.210? or is this not possible?

Thanks!

Hi,
one question: this setup makes no sense for me - why you want to seperate the traffic between pve-host and VMs in the same network?
Does the pve-host create an lot of traffic? In this case why? Perhaps of using iSCSI? Than it's a lot better to seperate the iSCSI-Network to host-only (without bridge).
Or Backup? But during backup-time is normaly not so much traffic on the VMs, or?!

Udo

 

[boca]

Easy way:
1. Assign additional IP address on your router (or firewall): 10.3.4.1/24.
2. Connect eth1/vmbr1 to router (or firewall).
3. Boot VM and set the IP address: 10.3.4.100 and gateway 10.3.4.1
4. Enable port forwarding on your router (or firewall) for IP 10.3.4.100

Hmmm, that seems messy :\

Hi,
Dietmar wrotes times ago: don't use an IP on the pve-host for vmbr1! It's makes no sense to use two addresses in the same subnet.
Simply use an interface-file like this:

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
    address 0.0.0.0
    netmask 0.0.0.0

auto eth1
iface eth1 inet static
    address 0.0.0.0
    netmask 0.0.0.0

auto vmbr0
iface vmbr0 inet static
        address  10.3.3.200
        netmask  255.255.255.0
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0
    gateway 10.3.3.1

auto vmbr1
iface vmbr1 inet manual
        bridge_ports eth1
        bridge_stp off
        bridge_fd 0

If you connect an VM to vmbr1 it's connected with the same natwork, where eth1 is connected - the host has nothing to do with that! Of course, you must select bridged network.

Udo

That is pretty much how my config looks like, but still no luck...

auto eth0
iface eth0 inet manual

auto eth1
iface eth1 inet manual

auto vmbr0
iface vmbr0 inet static
        address  10.3.3.235
        netmask  255.255.255.0
        gateway  10.3.3.1
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0

auto vmbr1
iface vmbr1 inet manual
        bridge_ports eth1
        bridge_stp off
        bridge_fd 0

Hi,
one question: this setup makes no sense for me - why you want to seperate the traffic between pve-host and VMs in the same network?
Does the pve-host create an lot of traffic? In this case why? Perhaps of using iSCSI? Than it's a lot better to seperate the iSCSI-Network to host-only (without bridge).
Or Backup? But during backup-time is normaly not so much traffic on the VMs, or?!

Udo

What I want to acheive is very simple.

VM A and VM B = vmbr0 = eth0
VM C and VM D = vmbr1 = eth1

I want to utilize multiple network cards on the same network

 

[boca]

Easy way:
1. Assign additional IP address on your router (or firewall): 10.3.4.1/24.
2. Connect eth1/vmbr1 to router (or firewall).
3. Boot VM and set the IP address: 10.3.4.100 and gateway 10.3.4.1
4. Enable port forwarding on your router (or firewall) for IP 10.3.4.100

Hmmm, that seems messy :\

Hi,
Dietmar wrotes times ago: don't use an IP on the pve-host for vmbr1! It's makes no sense to use two addresses in the same subnet.
Simply use an interface-file like this:

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
    address 0.0.0.0
    netmask 0.0.0.0

auto eth1
iface eth1 inet static
    address 0.0.0.0
    netmask 0.0.0.0

auto vmbr0
iface vmbr0 inet static
        address  10.3.3.200
        netmask  255.255.255.0
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0
    gateway 10.3.3.1

auto vmbr1
iface vmbr1 inet manual
        bridge_ports eth1
        bridge_stp off
        bridge_fd 0

If you connect an VM to vmbr1 it's connected with the same natwork, where eth1 is connected - the host has nothing to do with that! Of course, you must select bridged network.

Udo

That is pretty much how my config looks like, but still no luck...

auto eth0
iface eth0 inet manual

auto eth1
iface eth1 inet manual

auto vmbr0
iface vmbr0 inet static
        address  10.3.3.235
        netmask  255.255.255.0
        gateway  10.3.3.1
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0

auto vmbr1
iface vmbr1 inet manual
        bridge_ports eth1
        bridge_stp off
        bridge_fd 0

Hi,
one question: this setup makes no sense for me - why you want to seperate the traffic between pve-host and VMs in the same network?
Does the pve-host create an lot of traffic? In this case why? Perhaps of using iSCSI? Than it's a lot better to seperate the iSCSI-Network to host-only (without bridge).
Or Backup? But during backup-time is normaly not so much traffic on the VMs, or?!

Udo

What I want to achive is very simple.

VM A and VM B = vmbr0 = eth0
VM C and VM D = vmbr1 = eth1

I want to utilize both of my network cards on the same network (no bonding).

 

[udo]

...

What I want to achive is very simple.

VM A and VM B = vmbr0 = eth0
VM C and VM D = vmbr1 = eth1

I want to utilize both of my network cards on the same network (no bonding).

Hi,
strange - must work.
How looks your VM-config that connect to vmbr1?
What do you see on the host with tcpdump if you ping from this VM and to this VM from/to outside (same network, but on a different switchport)?
command are:

tcpdump -i vmbr1 host ip.of.vm.vmbr1

Udo

 

[snowman66]

Hmmm, that seems messy :\

I tought you needed vmbr0 for managment and vmbr1 for VMs...:/

What I want to achive is very simple.

VM A and VM B = vmbr0 = eth0
VM C and VM D = vmbr1 = eth1

I want to utilize both of my network cards on the same network (no bonding).

This scenario does work somehow, but only in direction vmbr1 -> vmbr0. Maybe ARP related problem because there is IP on vmbr0. If you add new eth3 and vmbr2 without IP i guess you should be ok.