Slow Linux bridge and OVS bridge

T

tplecko

New Member
Proxmox Subscriber
Feb 28, 2024
10
1
3
I have 3 Dell servers with 2 Xeon Gold 6438Y (each) + BCM57508 NetXtreme-E in each, connected via Mikrotik DACs to CRS520-4XS.
Proxmox 8.2 installed, and iperf from host to host tops at 60Gbps (not great, but it's fast).
I have 2 VMs with 2 nics (Virtio). One connected to vmbr0 (with one 100G port as a slave), and to vmbr1 (without any slaves)
iperf from some physicyl machine to VM is the same 60Gbps, as above mentioned.
However, iperf from VM to VM (via any bridge) is slow as hell. I can't get past 16Gbps.

Does anyone have any idea why this would be the case?
 
Did you test iperf between one of the Dell Server with the VM located on another Dell Server? how about the result you get?
 
When testing between host-1 and VM located on host-2, i get 60Gbps - 70Gbps.
When testing between VM located on host-1 and VM located on host-2, max speed is 35Gbps

I have tried changing MTU to 9000, but got the same results
 
I may not provide useful help, but can you provide following information for both PVE hosts?
/etc/network/interfaces
ip a
VM.conf
 
Identical servers, identical configs on all


Bash: 
root@pve-04:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback

iface eno8303 inet manual

iface eno12399np0 inet manual

iface eno12409np1 inet manual

iface eno8403 inet manual

iface eno12419np2 inet manual

iface eno12429np3 inet manual

auto ens6f0np0
iface ens6f0np0 inet manual
        ovs_type OVSPort
        ovs_bridge vmbr1

iface ens6f1np1 inet manual

auto vmbr0
iface vmbr0 inet static
        address 10.100.100.104/24
        gateway 10.100.100.1
        bridge-ports eno8303
        bridge-stp off
        bridge-fd 0

auto vmbr200
iface vmbr200 inet static
        address 10.100.102.104/24
        bridge-ports ens6f1np1
        bridge-stp off
        bridge-fd 0

auto vmbr1
iface vmbr1 inet static
        address 10.100.101.104/24
        ovs_type OVSBridge
        ovs_ports ens6f0np0
#100G Port 1

auto vmbr2
iface vmbr2 inet manual
        ovs_type OVSBridge
#Internal network

source /etc/network/interfaces.d/*
 
Ok, I really have no idea with this situation, may I ask do you think to use linux bridge instead OpenvSwitch and do the same testing again?
 
You must log in or register to reply here.
Top Bottom