Strange network problem behind the bridge

[Emil Makariev]

Hi guys,
I've got a strange network issue from 3-4 days.
I'm using Proxmox 4.4.21-1-pve with 2 VM's. All the machines have 1 public and 1 private address. Few days ago, the public ip's only on the VM's just stopped. If I setup the IP's as aliases directly on the proxmox host, they work! Behind the bridge, there is no ping to the gateway. I think the ISP cannot locate the mac addresses on the VM's. I'm 100% sure, the bridge is working and is properly configured. I tried all the available virtual hardware.
Any ideas how to solve this issue?

Thanks,
Emo

 

[manu]

a) Are you able from a VM to ping the other VM and ping the host ?
b) Could it be that your ISP / hoster restricts the gateway to forward packets ?

You can also have on the Proxmox host at the ICMP traffic with

tcpdump -nni vmbr0 icmp

here is a correct output of the ICMP traffic between a VM with IP adress 192.168.16.75 and a gateway with IP adress 192.168.16.1
10:30:11.074297 IP 192.168.16.75 > 192.168.16.1: ICMP echo request, id 7161, seq 1, length 64
10:30:11.074442 IP 192.168.16.1 > 192.168.16.75: ICMP echo reply, id 7161, seq 1, length 64

 

[Emil Makariev]

a) Are you able from a VM to ping the other VM and ping the host ?

I can ping all the machines from the second NIC in 10.10.10.0/24 network.
In the public network, they use the gateway, because are in different subnet. But I setup a random IP from 93.155.145.x on VM01 and there is a ping between VM01 and Proxmox host in the public network.

b) Could it be that your ISP / hoster restricts the gateway to forward packets ?

All I know is, they have restriction only by mac. But they turn off this protection for my addresses. The guys are not very responsive and they said, there is no problem on their side, because If I setup the IP's directly on the proxmox host or connect my laptop to the switch (see the picture below), I have ping and internet ( I tested with all pulic ip addresses).

You can also have on the Proxmox host at the ICMP traffic with

tcpdump -nni vmbr0 icmp

here is a correct output of the ICMP traffic between a VM with IP adress 192.168.16.75 and a gateway with IP adress 192.168.16.1
10:30:11.074297 IP 192.168.16.75 > 192.168.16.1: ICMP echo request, id 7161, seq 1, length 64
10:30:11.074442 IP 192.168.16.1 > 192.168.16.75: ICMP echo reply, id 7161, seq 1, length 64

I don't receive the ICMP packages, when the route is going through their gateway.
I think the isp gateway, do not route the traffic, because they cannot see the mac on the VM.
Any ideas why?

This is the schema of my network:

This is my /etc/network/interfaces:

auto lo
iface lo inet loopback

iface eth1 inet manual

iface eth0 inet manual

auto vmbr0
iface vmbr0 inet static
address 93.155.145.x
netmask 255.255.255.0
gateway 93.155.145.1
bridge_ports eth1
bridge_stp off
bridge_fd 0

auto vmbr1
iface vmbr1 inet static
address 10.10.10.1
netmask 255.255.255.0
bridge_ports eth0
bridge_stp off
bridge_fd 0

post-up echo 1 > /proc/sys/net/ipv4/ip_forward

Thanks a lot for your help!

 

[manu]

Since you're using here a bridging setup you should not need the
post-up echo 1 > /proc/sys/net/ipv4/ip_forward

I aussume all packets in the 10.10.10.1 network should stay in vmbr1 since all your vms have a public IP adress in the vmbr0 network, so no need to anything here.

Try to remove this line and see if that helps.

 

[Emil Makariev]

Tried, but the situation is the same

 

[manu]

Can you post the configuration of one of the problematic VM ? Are you sure the NIC of each VM having the a public address is attached to the vmbr bridge which is connected to the gateway ? ( I suppose vmbr0 here)

Also you could try to copy the mac adress of one VM and uses that on you laptop to check if you still get through the router. It would allow you to test if your ISP is doing mac adress filtering.

 

[Emil Makariev]

Here is the configuration of the VM:

bootdisk: sata0
cores: 8
memory: 32768
name: webdev
net0: virtio=36:66:30:63:37:37,bridge=vmbr0
net1: virtio=BA:68:14:9F:A5:46,bridge=vmbr1
numa: 0
ostype: l26
sata0: local:100/vm-100-disk-2.qcow2,cache=writeback,size=128G
smbios1: uuid=b9d64add-619e-4dfa-af78-d25e0aadc493
sockets: 1

Here is the /etc/network/interfaces file from the same VM:

# The primary network interface
auto eth0
iface eth0 inet static
address 93.155.147.x
netmask 255.255.255.0
network 93.155.147.0
broadcast 93.155.147.255
gateway 93.155.147.1
# dns-* options are implemented by the resolvconf package, if installed
dns-nameservers 93.155.147.1

auto eth1
iface eth1 inet static
address 10.10.10.2
netmask 255.255.255.0

I forgot to mention, I worked with this configuration 6-8 months without any problems and the only changes I've made are updates. The last proxmox update was few days before the problem. The guys from the ISP said, there is no changes on their end (which I'm not pretty sure is a truth), but in this situation, I don't know how to convince them otherwise.

Also you could try to copy the mac adress of one VM and uses that on you laptop to check if you still get through the router. It would allow you to test if your ISP is doing mac adress filtering.

I tried, didn't help. Now the mac filtration from the ISP is off.

 

[manu]

> I tried, didn't help. Now the mac filtration from the ISP is off.

I mean here did you set your laptop to have the mac address of your VM ? ( und shudtting down the VM and clearing the arp cache)
Because apart from that I have I have no clue.