network speed inside VM's slow

[manilx]

Hi

I have the following network configuration:

# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

auto enp1s0
iface enp1s0 inet manual

auto enp2s0
iface enp2s0 inet manual

auto enp3s0
iface enp3s0 inet manual

auto enp4s0
iface enp4s0 inet manual

auto enp5s0
iface enp5s0 inet manual

auto enp6s0
iface enp6s0 inet manual

auto bond1
iface bond1 inet manual
    bond-slaves enp2s0 enp3s0
    bond-miimon 100
    bond-mode active-backup
    bond-primary enp2s0

auto vmbr0
iface vmbr0 inet static
    address 192.168.2.5/24
    gateway 192.168.2.1
    bridge-ports enp1s0
    bridge-stp off
    bridge-fd 0
#management interface

auto vmbr1
iface vmbr1 inet manual
    bridge-ports bond1
    bridge-stp off
    bridge-fd 0
#interface for vm's

auto vmbr2
iface vmbr2 inet manual
    bridge-ports enp5s0
    bridge-stp off
    bridge-fd 0
#WAN Untangle

auto vmbr3
iface vmbr3 inet manual
    bridge-ports enp6s0
    bridge-stp off
    bridge-fd 0
#LAN Untangle

I have OPNsense running inside a VM using vmbr2&3


Using my Mac conected to the same switch as the proxmox server, using the same network and OPNsense firewall I get
Throughput of almost the theoretical maximum of my IP line (1000down/200up): 900down and 200up.
Great.

Now using speedtest inside other VM's (Windows Server or Debian) I get a throughput of 350down and 190up.
Not good.

VM's use the vmbr1 with bond1. I have it now as active-backup but had it LACP before. Same throughput.
I have no clue whatsoever why I get slower downloads. Copying to a NAS I get the maximum of 100MB/s so it's only internet speed....

Any help much appreciated.

 

[Dunuin]

Do you use virtio or E1000 virtual NICs?

 

[manilx]

Do you use virtio or E1000 virtual NICs?

Virtio on all VM's and network cards

 

[manilx]

Network throughput from VM's to shares "outside" of proxmox devices are at full 100mb/s speed. The same between VM's is less than half.
So internet download follows this pattern as firewall is a VM.

So why is this so and how to fix/improve this??

 

[manilx]

I have tested the following:
created a new bridge with no network cards or IP.
Added the bridge to 2 Windows VM's and gave them an IP.
Attached a share to from onw VM to the other via this new network.
Tested copying and it is fast (~350mb/s).

So the question is how can I
1. attach this bridge to the firewall vm (easy) and have the other VM's using this for internet access instead of my standard internal network but keeping all other non internet connections going via the normal bridge/network.

I'm over my head on this.....

 

[manilx]

Anybody? Really appreciate help if there is a solution.

 

[shrdlicka]

Hi,

from what I read, you are testing two different things. The one connected to the internet tests the speed from the VM to the internet.

The second test between two VMs on the same host doesn't use the "normal" network at all. This is just dependent on your machine's processing speed and can go ludicrous fast.

Can you do a test between the Proxmox host and another machine in the LAN with iperf3 and see how fast this is?

 

[manilx]

Hi

Actually I wasn’t testing 2 different things per se…

The internet connection from one (e.g. Windows) VM is via a firewall running on another VM. So the traffic now is going out of one VM into the switch and then into another one. Here I get slow speed.
The internet connection from a PC via switch to the firewall VM is fast. The only diffence here is PC and VM going to the internet (via firewall VM).
In my eyes there should be the same speed!

The test I did by not going to the switch from one VM to another was really just a test.

I need to get the VM->switch->firewall_VM->internet faster or the same speed as from a PC/Mac.

 

[shrdlicka]

I'm a bit confused by the word "switch" do you mean the bridge interface or a physical device?

I need to get the VM->switch->firewall_VM->internet faster or the same speed as from a PC/Mac.

does this mean:
1.) VM (on HOST A) -> switch > firewall VM (on HOST B) -> internet
2.) VM (on HOST A) -> switch > firewall VM (also on HOST A) -> internet

 

[manilx]

the switch is the hardware switch the proxmox is connected to and the PC/Mac/NAS.

so
VM (on proxmox) -> switch -> firewall (VM on proxmox) -> switch -> internet. 300mb/s
Mac/PC -> switch -> firewall (VM) -> switch -> internet 900+ mb/s

 

[Tismij]

FWIW, had slow internet on host (2/3 of speed) and really slow internet in VM (1/3 of speed).
After trying a lot finally fixed it by upgrading kernel to kernel 6.1, my mini-server has an AMD 5600G and it seems you need a newer kernel to use it properly.
Now excellent speeds again