activating vmbr0 renders host unreachable

js67 · Feb 24, 2011

Hi,
I'm feeling kind of stupid, nothing works...

Background:
I have a dedicated server at Server4You in Germany. Installed Debian Lenny image provided by the S4Y and installed Proxmox by means of apt-get.
I do have two public IPs but from different subnets. The main ip was bound to the interface eth0 the second I would like to use for a virtual machine.

Main IP:

Code:

IP     188.138.xx.71
Gateway     188.138.xx.1
Broadcast     188.138.xx.255
Netmask     255.255.255.0

Further IP:

Code:

IP     188.138.xx.201
Gateway     188.138.xx.193
Broadcast     188.138.xx.255
Netmask     255.255.255.192

The provider binds all traffic to the MAC address of the main interface (eth0)

So I thought I need a routed setup. I checked the Internet etc. but as soon as I configure vmbr0 the server will be unreachable

I tried various setups but the result is always the same. Server unreachable. So I have to boot into recovery mode and reset the settings.

The server is still reachable if I configure just eth0 in this configuration:
/etc/network/interfaces

Code:

# network interface settings
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
        address  188.138.xx.71
        netmask  255.255.255.255
        gateway  188.138.xx.1
        broadcast  188.138.xx.255
        network 188.138.xx.0
        pointopoint 188.138.xx.1
        up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp

My current vmbr0 configuration, which does not work looks like:

Code:

# network interface settings
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
        address  188.138.xx.71
        netmask  255.255.255.255
        gateway  188.138.xx.1
        broadcast  188.138.xx.255
        network 188.138.xx.0
        pointopoint 188.138.xx.1
        up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp

auto vmbr0
iface vmbr0 inet static
        address 188.138.xx.201
        netmask 255.255.255.192
#       broadcast 188.138.xx.255
        bridge_ports none
        bridge_stp off
        bridge_fd 0

As far as I understand the gateway for the secondary IP has to be set in the vm only as the host can not have two default gateways, right?

Can anybody tell me why the host will turn out unreachable as soon as I activate vmbr0?

Thanks
Jörg

udo · Feb 25, 2011

Hi,
you wan't to use your second ip inside the VM - so so should not use it at vmbr0.

Under normal circumstances should your vmbr0-config looks like that:

Code:

# network interface settings
auto lo
iface lo inet loopback

auto eth0
iface eth0 static
        address  0.0.0.0
        netmask 0.0.0.0

auto vmbr0
iface vmbr0 inet static
        address  188.138.xx.71
        netmask  255.255.255.255
        gateway  188.138.xx.1
        broadcast  188.138.xx.255
        network 188.138.xx.0
        pointopoint 188.138.xx.1
        up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp

is the netmask right???

Udo

js67 · Feb 25, 2011

Hi Udo,
the netmask is like this because I configured the interface as point2point. I do have the real netmask as well. But this does not change a thing.

I thought, that not using eth0 and just vmbr0 will lead to vmbr0 getting its own MAC address. But the MAC address always has to be the one of eth0 otherwise the switch will drop the traffic.
How will this work if I have more than one additional IP, meaning vmbr1 etc.

Thanks
Jörg

udo · Feb 25, 2011

js67 said:
Hi Udo,
the netmask is like this because I configured the interface as point2point. I do have the real netmask as well. But this does not change a thing.

I thought, that not using eth0 and just vmbr0 will lead to vmbr0 getting its own MAC address. But the MAC address always has to be the one of eth0 otherwise the switch will drop the traffic.
How will this work if I have more than one additional IP, meaning vmbr1 etc.

Thanks
Jörg

Hi,
with using vmbr0 (as bridge for eth0) you have the same mac-address as eth0:

Code:

proxmox1:~# ifconfig[B] eth0[/B]
eth0      Link encap:Ethernet  Hardware Adresse [B]00:0f:53:07:3a:9d[/B]  
          inet6-Adresse: fe80::f:5300:107:3a9d/64 Gültigkeitsbereich:Verbindung
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metrik:1
          RX packets:9496238230 errors:4295309538 dropped:0 overruns:0 frame:4295309524
          TX packets:8255222780 errors:0 dropped:0 overruns:0 carrier:0
          Kollisionen:0 Sendewarteschlangenlänge:1000 
          RX bytes:2149431546997 (1.9 TiB)  TX bytes:10737934656523 (9.7 TiB)
          Interrupt:33 

proxmox1:~# ifconfig [B]vmbr0[/B]
vmbr0     Link encap:Ethernet  Hardware Adresse [B]00:0f:53:07:3a:9d[/B]  
          inet Adresse:172.20.2.11  Bcast:172.20.2.255  Maske:255.255.255.0
          inet6-Adresse: fe80::20f:53ff:fe07:3a9d/64 Gültigkeitsbereich:Verbindung
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metrik:1
          RX packets:2502689063 errors:0 dropped:0 overruns:0 frame:0
          TX packets:917749374 errors:0 dropped:0 overruns:0 carrier:0
          Kollisionen:0 Sendewarteschlangenlänge:0 
          RX bytes:257029866221 (239.3 GiB)  TX bytes:7495220778196 (6.8 TiB)

Additional ip addresses must not end at the host - you can use it in the vm which bridged to vmbr0. But with all from one mac address this can be different (i had no experiences with that). But there are some post in this forum.
On hetzner is an discription how it works for there configuration: http://wiki.hetzner.de/index.php/Proxmox_VE#Netzwerkkonfiguration_KVM

Udo

js67 · Feb 25, 2011

Hi Udo,
thanks a lot. I will try it again..

The Hetzner setup I already tried and it does not work :-(

I prefer to have my servers at Hetzner, but my current Budget is limited and I do not know how long I will need the server. Therefor I wanted to save the setup fee. Maybe I should switch to Hetzner. Their service is much better...but for now I will need to get this up and running. Will check later and post an update.

Jörg

js67 · Feb 27, 2011

Hi,
still no solution. I tried several setups, Hetzner as suggested, routed etc.

As soon as I activate vmbr0 I do not have access to the machine. As I do not have direct access to it I can not analyse anything :-(

Does anybody have some ideas how I can further analyse the problem?

Thanks
Jörg

js67 · Mar 1, 2011

I just digged deeper into the issue.
After rebooting a recovery image and mounting the hardisk the
dmesg log lists

Code:

 ADDRCONF(NETDEV_UP): eth0: link is not ready

syslog

Code:

kernel: ADDRCONF(NETDEV_UP): eth0: link is not ready
 kernel: Bridge firewalling registered
 kernel: tg3: eth0: Link is down.
 kernel: tg3 0000:02:00.0: PME# enabled
 kernel: tg3.c:v3.102 (September 1, 2009)
 kernel: tg3 0000:02:00.0: PCI->APIC IRQ transform: INT A

 kernel: tg3 0000:02:00.0: setting latency timer to 64
 kernel: tg3 0000:02:00.0: PME# disabled
 kernel: tg3 mdio bus: probed
 kernel: eth0: Tigon3 [partno(BCM57780) rev 57780001] (PCI Express) MAC address 00:19:99:9b:a2:b4
 kernel: eth0: attached PHY driver [Broadcom BCM57780] (mii_bus:phy_addr=200:01)
 kernel: eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] TSOcap[1]
 zulu423 kernel: eth0: dma_rwctrl[76180000] dma_mask[64-bit]

Any ideas why eth0 runs without vmbr0 configured and fails if vmbr0 is present?

Regards
Jörg

udo · Mar 2, 2011

js67 said:

I just digged deeper into the issue.
After rebooting a recovery image and mounting the hardisk the
dmesg log lists

Code:

 ADDRCONF(NETDEV_UP): eth0: link is not ready

syslog

Code:

kernel: ADDRCONF(NETDEV_UP): eth0: link is not ready
 kernel: Bridge firewalling registered
 kernel: tg3: eth0: Link is down.
 kernel: tg3 0000:02:00.0: PME# enabled
 kernel: tg3.c:v3.102 (September 1, 2009)
 kernel: tg3 0000:02:00.0: PCI->APIC IRQ transform: INT A

 kernel: tg3 0000:02:00.0: setting latency timer to 64
 kernel: tg3 0000:02:00.0: PME# disabled
 kernel: tg3 mdio bus: probed
 kernel: eth0: Tigon3 [partno(BCM57780) rev 57780001] (PCI Express) MAC address 00:19:99:9b:a2:b4
 kernel: eth0: attached PHY driver [Broadcom BCM57780] (mii_bus:phy_addr=200:01)
 kernel: eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] TSOcap[1]
 zulu423 kernel: eth0: dma_rwctrl[76180000] dma_mask[64-bit]

Any ideas why eth0 runs without vmbr0 configured and fails if vmbr0 is present?

Regards

Jörg

Hi,
hmm - no idea... but have you tried to give eth0 the ip address (188.138.xx.71) and the same for vmbr0? Perhaps you can start then with vmbr0 and search than more.

Udo

js67 · Mar 2, 2011

Hi Udo,
it is as I already mentioned as long as I do not have vmbr0 configured at all it works. Once vmbr0 is configured in any way (tried "millions" of combinations) it seems to deactivate eth0 (physical device).
Are there any drivers/packages which have to be present for vmbr0? Where do I find the "scripts" configurations which deal with vmbr0?

Thanks
Jörg

1100101 · Mar 16, 2011

Hi,
did u find a solution for this problem? Have the same prob, activating vmbr0 renders the host unreachable...

J.L. Loinaz · Apr 8, 2011

Hi,

We consulted Server4You support and this is his reply:

because of our port security, only one MAC address is allowed per switch port. Therefore you can't use virtualization software in bridged mode. If you want to use virtualization, you have to configure it to work using NAT.

you can see more info in: http://pve.proxmox.com/wiki/Network_Model

in section Masquerading (NAT).

J.L. Loinaz · Apr 10, 2011

The problem occurs when the bridge automatically start, if you config the bridge manually working properly.

manually change / etc / network / interfaces

example:

# network interface settings

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
85.25.x.x address
netmask 255.255.255.0
85.25.x.x gateway

vmbr0 iface inet static
address 192.168.1.10
netmask 255.255.255.0
none bridge_ports
bridge_stp off
bridge_fd 0

restart the machine, then:

brctl addrb vmbr0

wmbr0 ifconf up

vmbr0 ifconfig 192.168.1.1 netmask 255.255.255.0

echo 1> / proc/sys/net/ipv4/conf/eth0/proxy_arp

echo 1> / proc/sys/net/ipv4/ip_forward

iptables-t nat-A POSTROUTING-s .168.1.0/24 '192 '-o eth0-j MASQUERADE

I hope that helps you, greetings.

Fleg · Mar 12, 2012

I had the same kind of problem and noticed that it was related to the kernel version.
This occur on HP proliant servers

Search

Search

activating vmbr0 renders host unreachable

js67

New Member

udo

Distinguished Member

js67

New Member

udo

Distinguished Member

js67

New Member

js67

New Member

js67

New Member

udo

Distinguished Member

js67

New Member

1100101

New Member

J.L. Loinaz

Member

J.L. Loinaz

Member

Fleg

New Member