Open vSwitch (OVS) w/ DPDK on Proxmox 8

[rtorres]

Hi everyone,

I’ve been trying with setting up Open vSwitch w/ DPDK on my Proxmox server but I’m struggling to move the management port from eno1 (temporary) back to enp4s0 (LAN interface) OVS w/ DPDK.

My Setup:​

• eno1: Temporary Proxmox management port. Normally an unused port. [1GbE Intel NIC] - 192.168.100.10
• enp4s0: LAN interface (intended Proxmox management port). [2.5GbE Intel I225 NIC] - 172.22.1.10/24
• enp3s0: WAN interface. [2.5GbE Intel I226 NIC] - ISP provided IP

Right now, I’ve got a pfSense VM running without any issues:

• LAN is on ovsbr0 (dpdk-p1), tied to enp4s0.
• WAN is on ovsbr1 (dpdk-p0), tied to enp3s0.

The problem arises when I try to move the Proxmox management interface from eno1 to ovsbr0.

When I set up Proxmox, it initially had the IP address 172.22.1.10. I temporarily changed it to 192.168.100.10 on vmbr2 as temporary backup to configure OVS+DPDK for LAN and WAN.

Normally, it would remain at 172.22.1.10:8006 on the LAN interface (ovsbr0 - dpdk-p1), but for some reason, I couldn’t connect to Proxmox at 172.22.1.10:8006 after plugging into the LAN port. Interestingly, I could still access pfSense at 172.22.1.11, but not Proxmox.

I was advised to assign ovsbr0 the IP address 172.22.1.10. It showed as down in ip link show, so I brought it up, assigned the IP, and added a route to 172.22.1.11, where pfSense resides. However, I still couldn’t ping pfSense and kept getting a "Network Unreachable" error. Enabling promiscuous mode on ovsbr0 didn’t resolve the issue either.

root@empve:~# ip addr show ovsbr0
6: ovsbr0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 1000
    link/ether 1c:fd:08:7d:6a:dc brd ff:ff:ff:ff:ff:ff
    inet 172.22.1.10/24 scope global ovsbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::1efd:8ff:fe7d:6adc/64 scope link
       valid_lft forever preferred_lft forever

I was also suggested to add the network configuration to /etc/network/interfaces, but I still couldn’t connect to Proxmox at 172.22.1.10:8006.

# Loopback interface
auto lo
iface lo inet loopback

# LAN interface on ovsbr0
auto ovsbr0
iface ovsbr0 inet static
        address 172.22.1.10/24
        gateway 172.22.1.11
        ovs_type OVSBridge
        ovs_ports dpdk-p1

# WAN interface on ovsbr1
auto ovsbr1
iface ovsbr1 inet manual
        ovs_type OVSBridge
        ovs_ports dpdk-p0

Here's the output of ovs-vsctl show, where pfSense is able to connect to both WAN and LAN.

root@empve:~# ovs-vsctl show
b3327cb8-5b76-465a-8be2-3713bdcf6701
    Bridge ovsbr0
        datapath_type: netdev
        Port dpdk-p0
            Interface dpdk-p0
                type: dpdk
                options: {dpdk-devargs="0000:03:00.0"}
        Port vhost-user-1
            Interface vhost-user-1
                type: dpdkvhostuserclient
                options: {vhost-server-path="/var/run/vhostuserclient/vhost-user-client-1"}
        Port ovsbr0
            Interface ovsbr0
                type: internal
    Bridge ovsbr1
        datapath_type: netdev
        Port ovsbr1
            Interface ovsbr1
                type: internal
        Port vhost-user-2
            Interface vhost-user-2
                type: dpdkvhostuserclient
                options: {vhost-server-path="/var/run/vhostuserclient/vhost-user-client-2"}
        Port dpdk-p1
            Interface dpdk-p1
                type: dpdk
                options: {dpdk-devargs="0000:04:00.0"}
ovs_version: "3.1.0"

What could I be doing wrong? Proxmox CLI still shows the 172.22.1.10:8006 when I hooked a monitor up.

I tried following this page but it doesn't mention anything about the Proxmox Management Port:

Y

Thread '[Tutorial] Run Open vSwitch (OVS) + DPDK on PVE 7.0'

Sep 29, 2021

Hello everyone,

I've successfully configured Open vSwitch + DPDK on PVE 7.0.
In my setup, a VM with 2-core 1.8 GHz can send 64 byte packets to wire via a Open vSwitch bridge at about 5Mpps via 10Gb Ethernet.
I write this to share you my steps. Hopefully this post can help with PVE development so that some day Open vSwitch + DPDK will be officially supported by Proxmox.

My environment:

- Intel(R) Xeon(R) CPU E5-2630L v4 @ 1.80GHz
- 2 x 32GB DDR4 ECC registered RAM
- Proxmox VE 7.0-11
- Network adapter: Intel Ethernet Converged X520-DA2 10Gigabit Ethernet Card
- Guest has 2 CPU core...

• yux
• Replies: 15
• Forum: Proxmox VE: Networking and Firewall

• 

I am purely doing this as a hobby, I like learning new things by getting myself into a hole LOL. I don't have any vlans, very simple non-fancy network.

Thank you for the assistance!!

 

[eminent]

You have to make an ovs port, attach it to the bridge and assign the port the IP address.

 

[rtorres]

You have to make an ovs port, attach it to the bridge and assign the port the IP address.

I appreciate you responding to my question! I was about to lose hope

When you say create an ovs port, proxmox was originally on ovsbr0.


How would I assign the IP address to ovsbr0?

Would it be something like this:

ip addr add 172.22.1.10/24 dev ovsbr0
ip link set dev ovsbr0 up
ip route add default via 172.22.1.11 dev ovsbr0

For some reason when I ran those commands I still couldn't connect to the Proxmox GUI.

Unless there is a service I need to restart. Pardon my ignorance

 

[eminent]

I appreciate you responding to my question! I was about to lose hope

When you say create an ovs port, proxmox was originally on ovsbr0.


How would I assign the IP address to ovsbr0?

Would it be something like this:

ip addr add 172.22.1.10/24 dev ovsbr0
ip link set dev ovsbr0 up
ip route add default via 172.22.1.11 dev ovsbr0

For some reason when I ran those commands I still couldn't connect to the Proxmox GUI.

Unless there is a service I need to restart. Pardon my ignorance

I did mine from the gui. You could edit it from /etc/network/interfaces to change it back to the old. Then create it in networks.

 

[rtorres]

I did mine from the gui. You could edit it from /etc/network/interfaces to change it back to the old. Then create it in networks.

Oh I think you are talking about regular ovs, ovs with dpdk doesn't show in GUI because it bypasses the Proxmox kernel completely.

Maybe someone else can explain it better but that is what I got when reading up on DPDK.

 

[eminent]

Oh I think you are talking about regular ovs, ovs with dpdk doesn't show in GUI because it bypasses the Proxmox kernel completely.

Maybe someone else can explain it better but that is what I got when reading up on DPDK.

Oh my bad, you are right.