[SOLVED] [UNITYMEDIA][ROUTER] No WAN traffic returns to VM bridge network adapter

TomFreudenberg

New Member
Feb 4, 2020
17
0
1
Hamburg, Germany
TLDR;

Sending packets from VM to WAN like 8.8.8.8 can be listen to on PROXMOX Server but isn't be receiveable by VM - 100% packet loss for VM

_____

Hi,

first of all I like to say "Hello" while being new here.
I am using Proxmox for the first time and we are trying to do some testings.
Since years we are using VMware for virtualisation and cluster.

My setup is as follows:

A.) Hardware with 4 CPU and 2 NETWORK interface
B.) INTERFACE 0 [80:WW:XX:YY:ZZ:60] = enp3s0 => vmbr0 == WAN
C.) INTERFACE 1 [other MAC] = enp5s0 => vmbr1 == LAN

We get the WAN IP v4 and v6 from our ISP via DHCP.
I am currently using "Unitymedia" and have a ConnectBox attached in "ModemMode"
When using the hardware with my standard Debian Router configuration, everything works fine.

/etc/network/interfaces on PROXMOX Server

Code:
auto lo
iface lo inet loopback

# WAN WIRE CONNECTED
iface enp3s0 inet manual

# LAN WIRE CONNECTED
iface enp5s0 inet manual

# WAN
auto vmbr0
iface vmbr0 inet manual
        bridge-ports enp3s0
        bridge-stp off
        bridge-fd 0

# LAN
auto vmbr1
iface vmbr1 inet static
        address  10.255.255.10
        netmask  255.255.255.0
        bridge-ports enp5s0
        bridge-stp off
        bridge-fd 0
All Firewalls, ebtables, options etc. on PROXMOX are disabled.
pve-firewall status => disabled/running


I have create a VM and used the Ubuntu-Server-1804-ISO to install the system.
Just standard installation for testing - nothing set other than default

The network adapters in VM are configured as

Network Device (net0) => virtio (80:WW:XX:YY:ZZ:60, bridge=vmbr0)
Network Device (net1) => virtio (generated MAC, bridge=vmbr1)

Attention: to get a working DHCP Request on the WAN Interface I had to use the SAME MAC Address an the VirtIO Adapter as from the regular hardware interface. I am not sure if this is correct?

____

What happens:

When starting the Ubuntu VM the system will boot correctly and receive its WAN IP from ISP by DHCP on net0 (ens18).
On net1 (ens19) we have configured a local 10.x.x.x/24 network.

/etc/netplan/50-cloud-init.yaml

Code:
network:
    ethernets:
        ens18:
            dhcp4: true
        ens19:
            addresses:
            - 10.255.255.9/24
            nameservers: {}
    version: 2
____

When pinging a system outside the WAN Interface and logging via tcpdump on PROXMOX Console I see correct outgoing and incoming packets:

on PROXMOX console:

tcpdump -n -i vmbr0 | grep -i icmp | egrep '(1\.2\.3\.4|8\.8\.8\.8)'

for ping 8.8.8.8 from Ubuntu VM

Code:
17:18:04.854175 IP 44.55.66.16 > 8.8.8.8: ICMP echo request, id 1439, seq 1, length 64
17:18:04.876269 IP 8.8.8.8 > 44.55.66.16: ICMP echo reply, id 1439, seq 1, length 64
17:18:05.862521 IP 44.55.66.16 > 8.8.8.8: ICMP echo request, id 1439, seq 2, length 64
17:18:05.885419 IP 8.8.8.8 > 44.55.66.16: ICMP echo reply, id 1439, seq 2, length 64
17:18:06.886534 IP 44.55.66.16 > 8.8.8.8: ICMP echo request, id 1439, seq 3, length 64
17:18:06.901289 IP 8.8.8.8 > 44.55.66.16: ICMP echo reply, id 1439, seq 3, length 64
17:18:07.910529 IP 44.55.66.16 > 8.8.8.8: ICMP echo request, id 1439, seq 4, length 64
17:18:07.924144 IP 8.8.8.8 > 44.55.66.16: ICMP echo reply, id 1439, seq 4, length 64
I got the outgoing and incoming packets on the PROXMOX console (BUT NOT INSIDE UBUNTU VM).

That means, Ubuntu ping will show 100% LOSS because none of these packets will come into the VM.


_____

I did not really found more stuff on that and hope that someone may guide me to a solution.

Thanks in advance
Tom


P.S.: I also tried things like:

Code:
pve-firewall stop
echo 0 > /proc/sys/net/bridge/bridge-nf-call-iptables
echo 1 > /proc/sys/net/ipv4/ip_forward
But always same result - no traffic from WAN enters the VM
 
Last edited:

TomFreudenberg

New Member
Feb 4, 2020
17
0
1
Hamburg, Germany
Maybe that is important in addition:

When I stop the VM and change the entry of vmbr0 into dhcp on PROXMOX Server like

Code:
iface vmbr0 inet dhcp
        bridge-ports enp3s0
        bridge-stp off
        bridge-fd 0
and run in console:

Code:
ifdown vmbr0
ifup vmbr0
I get the same DHCP address from WAN ISP and

CAN SUCCESSFULLY PING to 8.8.8.8

but this is on PROXMOX console not IN VM
 

spirit

Famous Member
Apr 2, 2010
3,931
212
83
www.odiso.com
>>Attention: to get a working DHCP Request on the WAN Interface I had to use the SAME MAC Address an the VirtIO Adapter as from the regular >>hardware interface. I am not sure if this is correct?

you have setup same mac address in your vm nic than enp3s0 ?

if yes, i don't think it can work, because you'll have same mac twice in the vmbr0, and it'll don't know where to forward traffic.


>>We get the WAN IP v4 and v6 from our ISP via DHCP.

when you said ISP, do you talk about your ISP at home with a small internet box ? or something like a datacenter ?
is it the ISP which really manage the DHCP ? Do you have multiple ip addresses coming from the dhcp, or it's only 1 ? (a small box configured in bridged mode for example)
 

TomFreudenberg

New Member
Feb 4, 2020
17
0
1
Hamburg, Germany
Hi @spirit

thanks for replying.

>> is it the ISP which really manage the DHCP ?

yes

>> Do you have multiple ip addresses coming from the dhcp, or it's only 1 ?

>> >> We get the WAN IP v4 and v6 from our ISP via DHCP.
>> >> I am currently using "Unitymedia" and have a ConnectBox attached in "ModemMode"

We will always just get 1 IP Address via DHCP and - yes - it is a small box configured in "bridge mode"

----

>> you have setup same mac address in your vm nic than enp3s0 ?

yes

>> if yes, i don't think it can work, because you'll have same mac twice in the vmbr0, and it'll don't know where to forward traffic.

I read a lot about that and found a number of documentation where people had written about running pfSense on proxmox. Those had entered same MAC address in their configuration. So I thought that this might be a kind of "standard" configuration.

----

As far as I understand, in that case the WAN NIC is only available in ONE VM and not used (assigned) in the proxmox host. So the traffic is always only for that single adapter.

Running:

Code:
ip address
on proxmox will just show the internal ip address from LAN
 

spirit

Famous Member
Apr 2, 2010
3,931
212
83
www.odiso.com
do you have tried to configure the macaddress only in the vm, and not on the host.

as vmbr is also a bridge, it's like double bridge (the bridge from your box, and the bridge from proxmox).
I don't think mac address need to be set on promox nic.
 

TomFreudenberg

New Member
Feb 4, 2020
17
0
1
Hamburg, Germany
>> do you have tried to configure the macaddress only in the vm, and not on the host.

I don't know what you mean but I have not configured the MAC on the host somewhere.

Its just the NIC (hardware) which has this MAC from vendor
 

TomFreudenberg

New Member
Feb 4, 2020
17
0
1
Hamburg, Germany
As @spirit mentioned above having the same MAC ADDRESS on NIC interface and TAP interface won't work.

This issue in general came up while using german ISP UnityMedia.

If you use their service and run the "ConnectBox" in "BridgeMode" (Modem only) they will store the MAC ADDRESS from the FIRST ROUTER you connect. This MAC ADDRESS seems to be not releasable in any way. I had a long talk to a service guy who assured that this is nothing they can configure somewhere.

The solution for that issue is just to remember and use always that MAC address.

This can be done by overwriting "hwaddress ether 01:02:03:04:05:06" inside /etc/network/interfaces for the hardware NIC on iface vmbr0. Make sue also to overwrite the address for the physical device. While that will not be "up" the "hwaddress" parameter does not work. I solved this by rule:

pre-up ip link set dev enp3s0 address 01:02:03:04:05:06

On the proxmox host console check that "bridge fdb show" will show the new MAC address on physical and virtual network device.

Now you may use the "UnityMedia learned" MAC address as a setting to network adapter in your VM configuration.

Good luck
Tom
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!