Slow network speed for bridged interfaces inside VM

[bmit]

Hello togehter,

i have a problem with a bridged interface and slow network speed inside the vm.

General config:

• 3 Hosts at Hetzner
• All Hosts 10G Uplink
• Same problem on all hosts
• IPs are changed
• All to Hetzner Config regarding Proxmox: Hetzner Proxmox Netowrk Config Guide
• VMs are Debian 12
• Problem: Slow network speed if using the bridged Interface at vm (good speed at host)

Network Config VMHost Proxmox:

• enp2so: Public IP Host (1.0.0.1)
• vmbr0: routed bridge with public IP (1.0.0.2)
• vmbr4001: bridged network with vlan4001 on enp2s0 using Hetzner vBridge with Public IP (2.0.0.2/29, GW 2.0.0.1) - MTU 1400
• enp2s0vlan4002 and vmbr1001: some internal stuff
• All Virtual Interfaces are VirtIO

Spoiler: Network config VMHost Proxmox

auto lo
iface lo inet loopback

iface lo inet6 loopback

auto enp2s0
iface enp2s0 inet static
address 1.0.0.1/32
gateway 10.0.0.129

iface enp2s0 inet6 static
address 2a01::2/64
gateway fe80::1

auto vmbr0
iface vmbr0 inet static
address 10.0.0.1/32
bridge-ports none
bridge-stp off
bridge-fd 0
up ip route add 10.0.0.2/32 dev vmbr0

iface vmbr0 inet6 static
address 2a01::3/64
up ip -6 route add 2a01::/64 dev vmbr0

auto enp2s0.4002
iface enp2s0.4002 inet static
address 172.16.0.2/24
mtu 1400

auto enp2s0.4001
iface enp2s0.4001 inet manual

auto vmbr4001
iface vmbr4001 inet static
address 2.0.0.2/29
bridge-ports enp2s0.4001
bridge-stp off
bridge-fd 0
mtu 1400

auto vmbr1001
iface vmbr1001 inet static
bridge-ports none
brodge-stp off
bridge-fd 0

VM 1 Network Config:

• This VM is using the vmbr0 routed Interface
• 1.0.0.2

Spoiler: Network Config VM 1

auto lo
iface lo inet loopback

auto ens18
iface ens18 inet static
address 1.0.0.2/32
gateway 1.0.0.1
dns-nameservers 9.9.9.9

VM 2 Network Config:

• This VM is using the vmbr4002 bridged vlan interface (Hetzner vSwitch)
• 2.0.0.3

Spoiler: Network Config VM 2] auto lo iface lo inet loopback auto ens18 iface ens18 inet static address 2.0.0.3/29 gateway 2.0.0.2 dns-nameservers 9.9.9.9 [/SPOILER] [U]VM 3 Network Config:[/U] [LIST] [*]This VM is using the vmbr4002 bridged vlan interface (Hetzner vSwitch) [*]2.0.0.4 [/LIST] [SPOILER="Network Config VM 3] auto lo iface lo inet loopback auto ens18 iface ens18 inet static address 2.0.0.4/29 gateway 2.0.0.2 dns-nameservers 9.9.9.9 [/SPOILER] [HR][/HR] [B]Speed Tests:[/B] [U]Host to public iperf Server:[/U] [LIST] [*]Speed: good as expected (3/5,5G) [*]Prepare: ip route add 62.176.246.73/32 via 2.0.0.1 [*]Command: iperf3 -c a209.speedtest.wobcom.de -4 --bidir -P 30 -i 10 --bind-dev vmbr4001 [/LIST] [SPOILER="Speedtest Host to Public Server

[SUM][TX-C] 0.00-10.00 sec 3.52 GBytes 3.03 Gbits/sec 7414 sender
[SUM][TX-C] 0.00-10.02 sec 3.44 GBytes 2.95 Gbits/sec receiver
[SUM][RX-C] 0.00-10.00 sec 5.82 GBytes 5.00 Gbits/sec 15769 sender
[SUM][RX-C] 0.00-10.02 sec 5.72 GBytes 4.91 Gbits/sec receiver

VM 1 (routed interface) to public iperf Server:

• Speed: ok - now asynchroniously with lower Download than Upload (4/7G)
• Command: iperf3 -c a209.speedtest.wobcom.de -4 --bidir -P 30 -i 10

Spoiler: Speedtest VM 1 to Public Server

[SUM][TX-C] 0.00-10.00 sec 4.42 GBytes 3.79 Gbits/sec 12879 sender
[SUM][TX-C] 0.00-10.02 sec 4.33 GBytes 3.71 Gbits/sec receiver
[SUM][RX-C] 0.00-10.00 sec 7.79 GBytes 6.69 Gbits/sec 4283 sender
[SUM][RX-C] 0.00-10.02 sec 7.69 GBytes 6.59 Gbits/sec receiver

VM 2 (bridged interface) to public iperf Server:

• Speed: bad - now asynchroniously with insane low Download (4G/10M)
• Command: iperf3 -c a209.speedtest.wobcom.de -4 --bidir -P 30 -i 10
• Same with VM 3 to public

Spoiler: Speedtest VM 2 to Public Server

[SUM][TX-C] 0.00-10.00 sec 4.43 GBytes 3.80 Gbits/sec 7032 sender
[SUM][TX-C] 0.00-10.02 sec 4.38 GBytes 3.76 Gbits/sec receiver
[SUM][RX-C] 0.00-10.00 sec 37.1 MBytes 31.1 Mbits/sec 1639 sender
[SUM][RX-C] 0.00-10.02 sec 2.46 MBytes 2.06 Mbits/sec receiver

VM 2 (bridged interface) to VM 3 (bridged interface) -- same VM Host:

• Speed: very bad (100K/100K)
• Command VM 3: iperf3 -s
• Command: iperf3 -c 2.0.0.4 -4 --bidir

Spoiler: Speedtest VM 2 to VM 3

[ ID][Role] Interval Transfer Bitrate Retr
[ 5][TX-C] 0.00-10.00 sec 107 KBytes 88.0 Kbits/sec 5 sender
[ 5][TX-C] 0.00-10.00 sec 0.00 Bytes 0.00 bits/sec receiver
[ 7][RX-C] 0.00-10.00 sec 738 KBytes 605 Kbits/sec 65 sender
[ 7][RX-C] 0.00-10.00 sec 137 KBytes 112 Kbits/sec receiver

VM 2 (bridged interface) to Host:

• Speed: very good as expected (14/14G)
• Command Host: iperf3 -s
• Command VM 2: iperf3 -c 2.0.0.2 -4 --bidir -P 30 -i 10
• Same with VM 3

Spoiler: Speedtest VM 2 to Host

[ ID][Role] Interval Transfer Bitrate Retr
[SUM][RX-C] 0.00-10.00 sec 15.7 GBytes 13.5 Gbits/sec 12907 sender
[SUM][RX-C] 0.00-10.00 sec 15.6 GBytes 13.4 Gbits/sec receiver
[SUM][TX-C] 0.00-10.00 sec 15.9 GBytes 13.7 Gbits/sec 0 sender
[SUM][TX-C] 0.00-10.00 sec 15.9 GBytes 13.6 Gbits/sec receiver

---

Consumtion:

• Good Speed from Host to Public iPerf Servers
• OK Speed from Routed Interface VM to Public iPerf Server
• Good Speed between VM (bridged interface) and Host
• No / Very Bad Speed from VM (bridged interface) to Internet

To be honest: I have no idea at this point.

Thank you all in advance for your help

 

[bmit]

EDIT: If i create vmbr0 as bridged interfaces instead of routed one (so VM 1) i have also very good speed as expected (8/8G). So it seems like it#s not a problem with routed interfaces, only with routed interfaces in combination with Hetzner vSwitch - but vSwitch can not be the problem cause from host on it works (see above). Hetzner support checked that also and told me all is good from their side.