proxmox 7.0 sdn beta test

Mike Lupe said:
Some feature request for the UI:

- Do not allow Vnet to be deleted if in use, analog to not being able to delete Zones, if in use by a Vnet.
Click to expand...

Technically, it's not so easy, I need to check that. (If they are a lof of vms/ct, I need to parse each vm/ct config to known if the vnet is used)
 
spirit said:
Technically, it's not so easy, I need to check that. (If they are a lof of vms/ct, I need to parse each vm/ct config to known if the vnet is used)
Click to expand...

Ok, I see. Maybe an additional hint on the "Vnet remove" confirmation popup could already help.
 
pieteras.meyer said:
Do I need to install the python script? ifupdown2/addons/openvswitch.py
Click to expand...
no no, it's already included if ifupdown2: 2.0.1-1+pve8

What I see is if I add or make changes and use the apply configuration in the Webgui all running VM loose network connection, and need to shutdown and restart the vm to restore network,
Click to expand...
that's strange, seem to be a bug

if possible to test, can you send me:

- /etc/network/interfaces content ? and /etc/network/interfaces.new with the changes.
- ifupdown2 debug log : "cp /etc/network/interfaces.new /etc/network/interfaces; ifreload -a -d"
- and result of "ovs-vsctl show" before/after reload
 
Mike Lupe said:
@spirit @t.lamprecht Unfortunately I haven't got a test-cluster right now to test SDN "over" (or in parallel to) already configured OVS VLANs. Don't ask why I got on with OVS for VLANs - at that time I thought it was a good idea ;)
Click to expand...

Is this still a good idea? Do you have any extra features or performance gain if you use SDN+OVS versus SDN alone? For example for having DPDK support for Mellanox cards with openvswitch-switch-dpdk ? or any hardware offloading?

In priciple just SDN sounds better because you have only one thing that you need to care about.
 
Mike Lupe said:
Ok, I see. Maybe an additional hint on the "Vnet remove" confirmation popup could already help.
Click to expand...
The main problem, is config locking. If somebody configure a vnet in a vm nic, at the same time you remove this vnet in sdn.
as we have 1config by vm, it's more difficult to implement. (to have it working at 100%, no race).
But it's not difficult to have a best effort check on vnet removal
 
Rares said:
Is this still a good idea? Do you have any extra features or performance gain if you use SDN+OVS versus SDN alone? For example for having DPDK support for Mellanox cards with openvswitch-switch-dpdk ? or any hardware offloading?

In priciple just SDN sounds better because you have only one thing that you need to care about.
Click to expand...

"SDN over/in parallel to OVS" mainly because I've already got a bunch of VLANs configured with OVS, and switching them all over to SDN means work, especially before being sure everything works as expected. But honestly, now I become a bit insecure about OVS: @spirit , I thought OVS is needed and a prerequisite for SDN, isn't it...?
 
spirit said:
The main problem, is config locking. If somebody configure a vnet in a vm nic, at the same time you remove this vnet in sdn.
as we have 1config by vm, it's more difficult to implement. (to have it working at 100%, no race).
But it's not difficult to have a best effort check on vnet removal
Click to expand...

I see. Are you using the mysql database? Maybe a helper table with zone/vnet/VMs assignments could do the job, so you don't have to "scan" the interfaces. But there being a same possible race-condition case, this would mean checking the helper table again, when pressing the "Add" button in the "Add network Device" dialogue of a VM, so a just removed Vnet would be recognized.

Just some thoughts.... thanks again for the good job.
 
Hi Spirit,

Please see below as requested, Just for you information I also have the the below patch loaded for "dot1q-tunnel" from David Herselman so im not sure if this is causing issues,

https://forum.proxmox.com/threads/proxmox-5-0-and-ovs-with-dot1q-tunnel.34090/

spirit said:
if possible to test, can you send me:

- /etc/network/interfaces content ? and /etc/network/interfaces.new with the changes.
- ifupdown2 debug log : "cp /etc/network/interfaces.new /etc/network/interfaces; ifreload -a -d"
- and result of "ovs-vsctl show" before/after reload
Click to expand...

before changes
Code: 
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback
        pre-up ifconfig eth0 mtu 9000
        pre-up ifconfig eth1 mtu 9000

iface eth0 inet manual
#eth0 - 1_eth-0-38 - 1G

iface eth1 inet manual
#eth1 - 2_eth-0-38 - 1G

auto vlan1
iface vlan1 inet static
        address 10.254.1.49/24
        gateway 10.254.1.1
        ovs_type OVSIntPort
        ovs_bridge vmbr0
        ovs_mtu 9000

auto vlan18
iface vlan18 inet static
        address 10.255.18.9/28
        ovs_type OVSIntPort
        ovs_bridge vmbr0
        ovs_mtu 9000
        ovs_options tag=18
#storage network

auto vlan20
iface vlan20 inet static
        address 10.255.20.9/24
        ovs_type OVSIntPort
        ovs_bridge vmbr0
        ovs_mtu 9000
        ovs_options tag=20
# storage iSCSI 1

auto vlan21
iface vlan21 inet static
        address 10.255.21.9/24
        ovs_type OVSIntPort
        ovs_bridge vmbr0
        ovs_mtu 9000
        ovs_options tag=21
# storage iSCSI 2

auto bond0
iface bond0 inet manual
        ovs_bonds eth0 eth1
        ovs_type OVSBond
        ovs_bridge vmbr0
        ovs_mtu 9000
        ovs_options lacp=active bond_mode=balance-tcp other_config:lacp-time=fast
# bond0_eth0/eth1

auto vmbr0
iface vmbr0 inet manual
        ovs_type OVSBridge
        ovs_ports bond0 vlan1 vlan18 vlan20 vlan21
        ovs_mtu 9000
#guest network 1G

ovs-vsctl show before reload

Code: 
ovs-vsctl show
78077929-12e5-4097-a203-e83decbe3c1a
    Bridge "vmbr0"
        Port "tap2003i2"
            tag: 5
            Interface "tap2003i2"
        Port "vlan18"
            tag: 18
            Interface "vlan18"
                type: internal
        Port "vlan20"
            tag: 20
            Interface "vlan20"
                type: internal
        Port "tap1006i2"
            tag: 2
            Interface "tap1006i2"
        Port "tap2003i0"
            Interface "tap2003i0"
        Port "tap1006i0"
            tag: 4
            Interface "tap1006i0"
        Port "vlan1"
            Interface "vlan1"
                type: internal
        Port "vlan21"
            tag: 21
            Interface "vlan21"
                type: internal
        Port "bond0"
            Interface "eth1"
            Interface "eth0"
        Port "vmbr0"
            Interface "vmbr0"
                type: internal
        Port "tap1006i1"
            tag: 5
            Interface "tap1006i1"
        Port "tap2003i1"
            tag: 4
            Interface "tap2003i1"
    ovs_version: "2.12.0"

interface.new diif

Code: 
--- /etc/network/interfaces    2020-05-14 18:27:38.082063705 +0200
+++ /etc/network/interfaces.new    2020-05-15 16:33:31.512050401 +0200
@@ -55,19 +55,28 @@
     ovs_options tag=21
 # storage iSCSI 2
 
+auto vlan23
+iface vlan23 inet static
+    address 10.255.23.9/24
+    ovs_type OVSIntPort
+    ovs_bridge vmbr0
+    ovs_mtu 9000
+    ovs_options tag=23
+#storage iSCSI 3
+
 auto bond0
 iface bond0 inet manual
     ovs_bonds eth0 eth1
     ovs_type OVSBond
     ovs_bridge vmbr0
     ovs_mtu 9000
-    ovs_options lacp=active bond_mode=balance-tcp other_config:lacp-time=fast
+    ovs_options other_config:lacp-time=fast bond_mode=balance-tcp lacp=active
 # bond0_eth0/eth1
 
 auto vmbr0
 iface vmbr0 inet manual
     ovs_type OVSBridge
-    ovs_ports bond0 vlan1 vlan18 vlan20 vlan21
+    ovs_ports bond0 vlan1 vlan18 vlan20 vlan21 vlan23
     ovs_mtu 9000
 #guest network 1G

- ifupdown2 debug log : "cp /etc/network/interfaces.new /etc/network/interfaces; ifreload -a -d"

see attached file

and then "ovs-vsctl show" after reload

Code: 
 ovs-vsctl show
78077929-12e5-4097-a203-e83decbe3c1a
    Bridge "vmbr0"
        Port "vlan18"
            tag: 18
            Interface "vlan18"
                type: internal
        Port "vlan20"
            tag: 20
            Interface "vlan20"
                type: internal
        Port "vlan1"
            Interface "vlan1"
                type: internal
        Port "vlan21"
            tag: 21
            Interface "vlan21"
                type: internal
        Port "bond0"
            Interface "eth1"
            Interface "eth0"
        Port "vmbr0"
            Interface "vmbr0"
                type: internal
        Port "vlan23"
            tag: 23
            Interface "vlan23"
                type: internal
    ovs_version: "2.12.0"
 

Attachments

@pieteras.meyer

Thanks!
I find the problem, it seem than tap interfaces are removed because they are not defined in the bridge

"
executing /usr/bin/ovs-vsctl -- --may-exist add-br vmbr0 -- --if-exists del-port vmbr0 tap1006i0 -- --if-exists del-port vmbr0 tap1006i1 -- --if-exists del-port vmbr0 tap1006i2 -- --if-exists del-port vmbr0 tap2003i0 -
"

We have a patch for linux bridge (as it's specific to proxmox). I think it's not implemented for ovs.

I'm going to fix it next week, it should be easy.

Thanks again for helping !
 
spirit said:
@pieteras.meyer
Hi,
here a patched version ifupdown2 with the fix for the tap interface removal

http://odisoweb1.odiso.net/ifupdown2_2.0.1-1+pve8_all.deb

could you test it ?

(just download it, dpkg -i ifupdown2_2.0.1-1+pve8_all.deb)

I have send the patch to dev mailing list, it should be available soon in official repos
Click to expand...

@spirit This patch seems to partially work, almost there..

If a CT or VM has "firewall" activated in its network settings, on "applying" SDN it loses network again and must be rebooted OR the firewall flag has to be set to disabled (without reapplying SDN, and then no reboot is required).
 
Mike Lupe said:
@spirit This patch seems to partially work, almost there..

If a CT or VM has "firewall" activated in its network settings, on "applying" SDN it loses network again and must be rebooted OR the firewall flag has to be set to disabled (without reapplying SDN, and then no reboot is required).
Click to expand...
thanks for the report! I have fixed it, and uploaded the deb again. (same link, same version, just reinstall over to test)
 
spirit said:
thanks for the report! I have fixed it, and uploaded the deb again. (same link, same version, just reinstall over to test)
Click to expand...

Getting closer:
For CTs with enabled FW flag it works now, but still not for VMs. So after "Apply" to SDN, then on the VM-nic the Firewall flag has to be disabled for the VM to regain network. Then the FW flag can be enabled again and network keeps running.
 
Mike Lupe said:
Getting closer:
For CTs with enabled FW flag it works now, but still not for VMs. So after "Apply" to SDN, then on the VM-nic the Firewall flag has to be disabled for the VM to regain network. Then the FW flag can be enabled again and network keeps running.
Click to expand...
That's strange, it should be the same for vm && ct.
previously, I was looking for "fwprX" interface in ovs, but it was "fwlnX".
I have done test with vm+firewall, it's work fine for me.

can you send output of "ovs-vsctl show" before/after reload ? (I'll redo test on my side)
 
spirit said:
can you send output of "ovs-vsctl show" before/after reload ? (I'll redo test on my side)
Click to expand...

@spirit There is no change in the output of "ovs-vsctl show" before and after simply because I only "Apply" SDN without any changes made to config of Zones, Vnets or VM nic VLAN.

This is the situation:

VM-01:
- no VLAN configured, NIC attached directly to vmrb0
- FW flag on VM nic active

Steps to reproduce:
- # (external) ping VM-01: ok
- PVE: "Apply" SDN without any changes (of course no change in output of "ovs-vsctl show")
- # ping VM-01: no response

- VM-01 NIC: disable firewall (no change in output of "ovs-vsctl show")
- # ping VM-01: ok

- VM-01 NIC: enable firewall (no change in output of "ovs-vsctl show")
- # ping VM-01: ok

- PVE: "Apply" SDN without any changes (of course no change in output of "ovs-vsctl show")
- # ping VM-01: no response

[...]
 
Mike Lupe said:
@spirit There is no change in the output of "ovs-vsctl show" before and after simply because I only "Apply" SDN without any changes made to config of Zones, Vnets or VM nic VLAN.

This is the situation:

VM-01:
- no VLAN configured, NIC attached directly to vmrb0
- FW flag on VM nic active

Steps to reproduce:
- # (external) ping VM-01: ok
- PVE: "Apply" SDN without any changes (of course no change in output of "ovs-vsctl show")
- # ping VM-01: no response

- VM-01 NIC: disable firewall (no change in output of "ovs-vsctl show")
- # ping VM-01: ok

- VM-01 NIC: enable firewall (no change in output of "ovs-vsctl show")
- # ping VM-01: ok

- PVE: "Apply" SDN without any changes (of course no change in output of "ovs-vsctl show")
- # ping VM-01: no response

[...]
Click to expand...
thanks this is helpfull. I'll try to reproduce. Maybe it's an mac-address-table flush problem or something like that.
strange that it's working for ct and not vm...
 
Hi Spirit

I have also loaded the patch, and simple network change like adding empty bridge and then the reload seem to work fine, I dont make use of firewall so can confirm the bug mike is seeing,

I do however see the below error after adding a QinQ SDN and vnet, as soon as I try to start a VM with this vnet as interface then I get the below error in the Gui,

Code: 
()
ovs vmbr0 is not supported by qinq at /usr/share/perl5/PVE/Network/SDN/Zones/QinQPlugin.pm line 94.
kvm: network script /var/lib/qemu-server/pve-bridge failed with status 512
TASK ERROR: start failed: QEMU exited with code 1
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom