Public and Private bridge - systems can't ping other systems on them.

K

kimnzl

New Member
Dec 19, 2010
6
0
1
Hi,
I am looking to have 2 bridges.
Host:
vmbr0 is connected to eth0 for general networking and it is set statically to 10.1.1.3/24 with a default gateway of 10.1.1.2.
vmbr1 is for host only networking. It is planned to be used for iSCSI to get raw drives passed through to an OpenSolaris host. It is set statically to 192.168.1.1/24

I have the OpenSolaris host running within kvm. With 2 e1000 cards connected to vmbr0 and vmbr1 set statically to 10.1.1.4/24 and 192.168.1.2/24 respectively (matching the above).

I have not been able to get either system to be able to ping the other on any of the 4 addresses.

I can ping the host 10.1.1.1 and 10.1.1.2 from both the Host and the OpenSolaris vm so that network is mostly working just the vm can't ping the host and vice versa.

Does anyone have any idea why it does not work?

As below I have tried using promisc mode on the cards to no effect.

host:~# cat /etc/network/interfaces
# network interface settings
auto lo
iface lo inet loopback

iface eth0 inet manual
# pre-up ifconfig eth0 0.0.0.0 promisc up
# pre-down ifconfig eth0 down

iface eth1 inet manual
# pre-up ifconfig eth1 0.0.0.0 promisc up
# pre-down ifconfig eth1 down

auto vmbr0
iface vmbr0 inet static
address 10.1.1.3
netmask 255.255.255.0
gateway 10.1.1.2
broadcast 10.1.1.255
bridge_ports eth0
bridge_stp off
bridge_fd 0

auto vmbr1
iface vmbr1 inet static
address 192.168.1.1
netmask 255.255.255.0
broadcast 192.168.1.255
bridge_ports none
bridge_stp off
bridge_fd 0
 
kimnzl said:
Hi,
I am looking to have 2 bridges.
Host:
vmbr0 is connected to eth0 for general networking and it is set statically to 10.1.1.3/24 with a default gateway of 10.1.1.2.
vmbr1 is for host only networking. It is planned to be used for iSCSI to get raw drives passed through to an OpenSolaris host. It is set statically to 192.168.1.1/24

I have the OpenSolaris host running within kvm. With 2 e1000 cards connected to vmbr0 and vmbr1 set statically to 10.1.1.4/24 and 192.168.1.2/24 respectively (matching the above).

I have not been able to get either system to be able to ping the other on any of the 4 addresses.

I can ping the host 10.1.1.1 and 10.1.1.2 from both the Host and the OpenSolaris vm so that network is mostly working just the vm can't ping the host and vice versa.

Does anyone have any idea why it does not work?

As below I have tried using promisc mode on the cards to no effect.

host:~# cat /etc/network/interfaces
# network interface settings
auto lo
iface lo inet loopback

iface eth0 inet manual
# pre-up ifconfig eth0 0.0.0.0 promisc up
# pre-down ifconfig eth0 down

iface eth1 inet manual
# pre-up ifconfig eth1 0.0.0.0 promisc up
# pre-down ifconfig eth1 down

auto vmbr0
iface vmbr0 inet static
address 10.1.1.3
netmask 255.255.255.0
gateway 10.1.1.2
broadcast 10.1.1.255
bridge_ports eth0
bridge_stp off
bridge_fd 0

auto vmbr1
iface vmbr1 inet static
address 192.168.1.1
netmask 255.255.255.0
broadcast 192.168.1.255
bridge_ports none
bridge_stp off
bridge_fd 0
Click to expand...

I have similar problem:
http://forum.proxmox.com/threads/53...rking-between-KVM-guests-after-upgrade-to-1.7

It seems to be rather bug, than a feature, since there's no logical explanation for such behavior. Especialy, it just works, with previous versions (try PM VE 1.5, to verify), or other VE's (vmware). Something is causing malformed datagrams, and it looks completely random. Image attached, for ssh.

ssh_pm.png
 
To add to the confusion I have another host with hugely different hardware where the Public bridge (routed) setup works correctly.
One difference is that the other system is AMD based, it has sky2 and tg3 cards. The host I am having issues with has Xeon 53xx CPU and dual e1000 nics.

Both are running 1.7 with the 2.6.32-4-pve kernel.

host:~# pveversion -v
pve-manager: 1.7-10 (pve-manager/1.7/5323)
running kernel: 2.6.32-4-pve
proxmox-ve-2.6.32: 1.7-28
pve-kernel-2.6.32-4-pve: 2.6.32-28
qemu-server: 1.1-25
pve-firmware: 1.0-9
libpve-storage-perl: 1.0-16
vncterm: 0.9-2
vzctl: 3.0.24-1pve4
vzdump: 1.2-9
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1
pve-qemu-kvm: 0.13.0-2
ksm-control-daemon: 1.0-4

amdhost:~# pveversion -v
pve-manager: 1.7-10 (pve-manager/1.7/5323)
running kernel: 2.6.32-4-pve
proxmox-ve-2.6.32: 1.7-28
pve-kernel-2.6.32-4-pve: 2.6.32-28
qemu-server: 1.1-25
pve-firmware: 1.0-9
libpve-storage-perl: 1.0-16
vncterm: 0.9-2
vzctl: 3.0.24-1pve4
vzdump: 1.2-9
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1
pve-qemu-kvm: 0.13.0-2
ksm-control-daemon: 1.0-4

amdhost /etc/network/interfaces
# network interface settings
auto lo
iface lo inet loopback

iface eth0 inet manual

auto vmbr1
iface vmbr1 inet static
address 192.168.100.2
netmask 255.255.255.248
bridge_ports eth0
bridge_stp off
bridge_fd 0

iface eth1 inet manual

auto vmbr0
iface vmbr0 inet static
address 10.1.1.1
netmask 255.255.255.0
gateway 10.1.1.2
bridge_ports eth1
bridge_stp off
bridge_fd 0
 
Last edited:
My systems are Intel Xeon / Broadcom NetXtreme II based. To add to disaster, default kernel 2.6.32 doesn't even support raid controller (well documented bug): 24:00.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS 9240
so you can't install it on large chunk of IBM servers. Looks like upstream bug, and once again, Debian community "shines" at its best. :)
 
kimnzl said:
Hi,
I am looking to have 2 bridges.
Host:
vmbr0 is connected to eth0 for general networking and it is set statically to 10.1.1.3/24 with a default gateway of 10.1.1.2.
vmbr1 is for host only networking. It is planned to be used for iSCSI to get raw drives passed through to an OpenSolaris host. It is set statically to 192.168.1.1/24

I have the OpenSolaris host running within kvm. With 2 e1000 cards connected to vmbr0 and vmbr1 set statically to 10.1.1.4/24 and 192.168.1.2/24 respectively (matching the above).

I have not been able to get either system to be able to ping the other on any of the 4 addresses.

I can ping the host 10.1.1.1 and 10.1.1.2 from both the Host and the OpenSolaris vm so that network is mostly working just the vm can't ping the host and vice versa.

Does anyone have any idea why it does not work?

As below I have tried using promisc mode on the cards to no effect.

host:~# cat /etc/network/interfaces
# network interface settings
auto lo
iface lo inet loopback

iface eth0 inet manual
# pre-up ifconfig eth0 0.0.0.0 promisc up
# pre-down ifconfig eth0 down

iface eth1 inet manual
# pre-up ifconfig eth1 0.0.0.0 promisc up
# pre-down ifconfig eth1 down

auto vmbr0
iface vmbr0 inet static
address 10.1.1.3
netmask 255.255.255.0
gateway 10.1.1.2
broadcast 10.1.1.255
bridge_ports eth0
bridge_stp off
bridge_fd 0

auto vmbr1
iface vmbr1 inet static
address 192.168.1.1
netmask 255.255.255.0
broadcast 192.168.1.255
bridge_ports none
bridge_stp off
bridge_fd 0
Click to expand...
Hi,
try for vmbr1 a dummy device:
like
Code: 
iface vmbr0 inet static
        address  10.1.1.3
        netmask  255.255.255.0
        gateway  10.1.1.2
        broadcast 10.1.1.255
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0
        post-up /usr/local/scripts/generate_vmbr1.sh
and /usr/local/scripts/generate_vmbr1.sh:
Code: 
modprobe -o dummy0 dummy
ifconfig dummy0 up
brctl addbr vmbr1
brctl addif vmbr1 dummy0
If the host need an ip (normaly not) you must assign one to the bridge.

Udo
 
ziggy said:
My systems are Intel Xeon / Broadcom NetXtreme II based. To add to disaster, default kernel 2.6.32 doesn't even support raid controller (well documented bug): 24:00.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS 9240
so you can't install it on large chunk of IBM servers. Looks like upstream bug, and once again, Debian community "shines" at its best. :)
Click to expand...

I cannot verify it in the moment but just to note, 99 % of these cases its the fault of the hardware vendors, not the the Debian community. For a user its not important who failed but you can go for hardware vendors supporting Debian quite well.

And back to topic, if its well documented post the bug report as a reference.

looks like this: http://lists.debian.org/debian-kernel/2010/11/msg00449.html
 
Last edited:
Hi Udo,
Trying the bridge using a dummy0 did not work either. Pings / communication will still not go through. The better question is why the public bridge does not work... It works properly on the amdhost machine. I have take the private bridge out of the config leaving only the public one. I still can't seem to get any ping response. As a note both ends do not have firewalls. This is beginning to seem like a bug to me. What further steps could I take to debug the problem…?
 
I cannot verify it in the moment but just to note, 99 % of these cases its the fault of the hardware vendors, not the the Debian community.
Click to expand...
Oh, just remember the famous ssh bug. Problem with Debian cummunity is, well, a lot of guys recompiling those packages really don't know what the hell are they doing.

Of course, bugs happen, but this is really mainstream, enterprise class hardware and, unfortunately, after upgrade, just doesn't work anymore. Works even with old Centos 2.6.18 kernel.

To bad, becouse your product is unique on the market, and I wouldn't mind paying for it, only if underlying system could be little more reliable. :)

I'll keep trying with other kernels, but so far it only works with 2.6.18, but it has problem with Symbios Logic MegaRAID SAS 9240 controller (horrible guest perfomance). This is corrected with 2.6.35 kernel. but than, there's problem with network datagram corruption. :)
 
I currently have a reasonably nasty workaround. I have a dual port PCI-X GB NIC with the cable going from one port to the other. This is creating a physical bridge. The internal eth2 is bridged to vmbr2 and eth3 is bridged to vmbr3. The VM has access to vmbr3. It is not the best solution but with the NIC and an mtu of 9000, I have connectivity between the VM and the Host.
However I believe that there is a bug here somewhere. If anyone can think of anything please let me know.
 
hi all

i've had a similar issue with one virtual machine running a windows 2003 server which has been migrated and moved to different hardware and finally it has been virtualized first as a vmware image and now as a kvm image on a proxmox cluster with drbd + lvm as shared storage.. to make matters worse, this is our primary domain controller ;)

strangely i could ping some of the ohter vm's running on the same host but some others i could not ping like described in other posts..

one thing that we all have in common: we are all using the e1000 on our virtual machines.. this made me curious.. i've replaced that e1000 with a new RTL8139 (with a new mac address as well) and it now works flawlessly with all other guests and the host as well..

so it seems there is a bug in the e1000 emulator or at least connected to it causing this issue..

now that i knew what to google for i found it:
https://bugzilla.redhat.com/show_bug.cgi?id=510840

windows 2003 server 32bit with standard e1000 driver does not work. it is recommended to either use another driver or another interface with those hosts. the bug is closed as "won't fix" so don't wait for it to eventually work :)

hope this helps.

cheers
pascal
 
just a little additional information for your amusement.. i just talked to our guys from the hardware department (i work for a system integrator) and they told me that they experienced similar issues with the standard e1000 drivers even on physical machines.. so it seems kvm is just VERY precise in emulating those cards lol ..

cheers
pascal
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back