proxmox 6.2 sdn beta test

spirit

Famous Member
Apr 2, 2010
4,434
319
103
www.odiso.com
Hi there,

I'm a bit courious about relying on ifupdown2 when seeing what happened upstream to ifupdown2 (https://bugs.launchpad.net/ubuntu/+source/ifupdown2/+bug/1872590)

Ubuntu use netplan by default, so this don't make any sense for them to keep ifupdown1 or ifupdown2 or any think related to /etc/network/interfaces. (and netplan is just a wrapper on top of systemd-networkd)


Ifupdown2 is developped by cumulus linux, used in their switch. and main ifupdown2 developper is also the maintainer of debian package
and version 3 has been migrated to testing recently.
https://tracker.debian.org/news/1177218/ifupdown2-300-1-migrated-to-testing/

So no worry here, it'll be available for a long time.

(and it's far more advanced than netplan/systemd-networkd)
 
  • Like
Reactions: fhloston

t.lamprecht

Proxmox Staff Member
Staff member
Jul 28, 2015
3,322
609
133
South Tyrol/Italy
shop.maurer-it.com
I'm a bit courious about relying on ifupdown2 when seeing what happened upstream to ifupdown2 (https://bugs.launchpad.net/ubuntu/+source/ifupdown2/+bug/1872590)

Note that, while yes, an older version was removed from Debian Testing due to it not being ported to python3, the recent ifupdown2 3.0 release was then introduced to testing/unstable Debian repositories again:
https://tracker.debian.org/news/1177218/ifupdown2-300-1-migrated-to-testing/

This one is now updated to python 3 and will stay.

Removing packages temporarily from Debian testing or unstable is a pretty common workflow in Debian, so one needs to always check the context here - which is not always obvious, like it was the case for the Ubuntu Tracker you linked.

Anyway, as spirit said: ifupdown2 has its Debian maintainer also as developer, which is really nice.
Further, we package ifupdown2 ourself to have more recent versions (with all the good bug and feature fixes @spirit does included).
 
  • Like
Reactions: fhloston

cyruspy

Member
Jul 2, 2013
22
0
21
I've tried this between two test nodes (not part of a cluster) and VXLAN, the traffic seems to pass through, but ping tests reports duplicated answers:

1602113012233.png

Any hints?
 

cyruspy

Member
Jul 2, 2013
22
0
21
Given IDs are global, wouldn't make more sense to generate automatically an UUID for the VNET?, "inviting" the user to populate Alias parameter with what makes sense descriptively to the user?.

Also, given VNIs are global, should de interface autogenerate one as proposal?, and the user would be able to change it if needed (to match for example VNIs on an external node?)
 

spirit

Famous Member
Apr 2, 2010
4,434
319
103
www.odiso.com
Given IDs are global, wouldn't make more sense to generate automatically an UUID for the VNET?, "inviting" the user to populate Alias parameter with what makes sense descriptively to the user?.

Also, given VNIs are global, should de interface autogenerate one as proposal?, and the user would be able to change it if needed (to match for example VNIs on an external node?)
Hi, I can't use uuid where, because of limitation of linux interfaces name lenght. (14caracters), and I'm currently limiting vnet to 8 caracters, because I'm also use some internal mapping between vnet && other vxlan,vlan interface.
(the vnet is a bridge with same name)

That's why I have added an alias, where you can use a longer name, more descriptive. (I don't displaying yet in vm/ct nic bridge selection, I'll coming soon)

So, I don't known how to generate an unique id here.
Maybe I could add an option, to define an user custom pattern ? (like : vnet(\d+)), and auto increment counter for example.
I'm thinking to add some kind of batch vnet add wizared, where you can said add 1000 vnets with id from (1-1000).
But I can't used directly vni for the name, because vni can be really long (16millions).
 

spirit

Famous Member
Apr 2, 2010
4,434
319
103
www.odiso.com
I've tried this between two test nodes (not part of a cluster) and VXLAN, the traffic seems to pass through, but ping tests reports duplicated answers:

View attachment 20310

Any hints?
mmm, this is strange, I never see this.
Can you send your /etc/pve/sdn/*.cfg && /etc/network/interfaces && /etc/network/interfaces.d/sdn of both nodes ?
 

cyruspy

Member
Jul 2, 2013
22
0
21
Hi, I can't use uuid where, because of limitation of linux interfaces name lenght. (14caracters), and I'm currently limiting vnet to 8 caracters, because I'm also use some internal mapping between vnet && other vxlan,vlan interface.
(the vnet is a bridge with same name)

That's why I have added an alias, where you can use a longer name, more descriptive. (I don't displaying yet in vm/ct nic bridge selection, I'll coming soon)

So, I don't known how to generate an unique id here.
Maybe I could add an option, to define an user custom pattern ? (like : vnet(\d+)), and auto increment counter for example.
I'm thinking to add some kind of batch vnet add wizared, where you can said add 1000 vnets with id from (1-1000).
But I can't used directly vni for the name, because vni can be really long (16millions).
Well, a pseudo UUID would work also ;). my suggestion would be just to autogenerate the ID, I don't see a use case to define the pattern by the user.

I'm just thinking about how to make it less possible to clash ("XX VNI is already used" or "VNET ID already used"). ID shouldn't be used as descriptive (asked to the user), that's what you have Alias for.

VNI generation would be nice, but I do see a use case where the user would need to match it with what he/she has on the other side (external to your control?)
 

cyruspy

Member
Jul 2, 2013
22
0
21
mmm, this is strange, I never see this.
Can you send your /etc/pve/sdn/*.cfg && /etc/network/interfaces && /etc/network/interfaces.d/sdn of both nodes ?

Attaching files...
 

Attachments

  • sdn-troubleshooting-bigiron.tar.gz
    970 bytes · Views: 2
  • sdn-troubleshooting-miniiron.tar.gz
    829 bytes · Views: 3

spirit

Famous Member
Apr 2, 2010
4,434
319
103
www.odiso.com
Well, a pseudo UUID would work also ;). my suggestion would be just to autogenerate the ID, I don't see a use case to define the pattern by the user.

I'm just thinking about how to make it less possible to clash ("XX VNI is already used" or "VNET ID already used"). ID shouldn't be used as descriptive (asked to the user), that's what you have Alias for.
Well, I could generate a simple "vnet(\d+)" default value in the form for the vnetid. (with an autoincrement, so I easily find available vnet).


VNI generation would be nice, but I do see a use case where the user would need to match it with what he/she has on the other side (external to your control?)
I think I could just pre-fill the form too with next vni available. (User can still change it), but for vlan for example, user need to be able to define it to match is physical vlans.
I think it's be more confusing.

I need to think about this more deeply, thanks for your comments !
 

spirit

Famous Member
Apr 2, 2010
4,434
319
103
www.odiso.com
Attaching files...
sdn config seem fine,

I have question about this on bigiron node

Code:
auto vmbr1
iface vmbr1 inet static
    address 10.2.0.223/24
    gateway 10.2.0.1
    bridge-ports enp6s5f0 enp6s5f1 vmbr0.100
    bridge-stp off
    bridge-fd 0

Why do you have vmbr0.100 in vmbr1 ?

also why do you have 2 physical interfaces (enp6s5f0, enp6s5f1) in this bridge ?
(I think are plugged on same physical switch, you'll have a loop. you should create a bond interface in this case)
 

cyruspy

Member
Jul 2, 2013
22
0
21
sdn config seem fine,

I have question about this on bigiron node

Code:
auto vmbr1
iface vmbr1 inet static
    address 10.2.0.223/24
    gateway 10.2.0.1
    bridge-ports enp6s5f0 enp6s5f1 vmbr0.100
    bridge-stp off
    bridge-fd 0

Why do you have vmbr0.100 in vmbr1 ?

also why do you have 2 physical interfaces (enp6s5f0, enp6s5f1) in this bridge ?
(I think are plugged on same physical switch, you'll have a loop. you should create a bond interface in this case)
Hello!,

bigiron is in little distant from the switch. I use additional ports in that node to connect other clients as a cascaded switch.

Those physical ports are connected to a PDU and a UPS for monitoring purposes. Those ports are in access, while they need to participate in the management VLAN (VLAN 100 in the port connected to the switch)

That is why
 

spirit

Famous Member
Apr 2, 2010
4,434
319
103
www.odiso.com
Hello!,

bigiron is in little distant from the switch. I use additional ports in that node to connect other clients as a cascaded switch.

Those physical ports are connected to a PDU and a UPS for monitoring purposes. Those ports are in access, while they need to participate in the management VLAN (VLAN 100 in the port connected to the switch)

That is why
mmm, ok, I think I understand.
Why not simply do , instead your conf:

Code:
auto vmbr0
iface vmbr0 inet manual
    bridge-ports enp5s4f0
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094

auto vmbr0.100
iface vmbr0.100 inet manual
    vlan-id 100


auto vmbr1
iface vmbr1 inet static
    address 10.2.0.223/24
    gateway 10.2.0.1
    bridge-ports enp6s5f0 enp6s5f1 vmbr0.100
    bridge-stp off
    bridge-fd 0
#Downlink para gest

this (tagging vlan on physical interfaces)

Code:
auto vmbr0
iface vmbr0 inet manual
    bridge-ports enp5s4f0  enp6s5f0.100 enp6s5f1.100
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094

auto vmbr0.100
iface vmbr0.100 inet static
    address 10.2.0.223/24
    gateway 10.2.0.1

or using vlan-aware feature (with ifupdown2 package)

Code:
auto enp6s5f0
iface enp6s5f0
    bridge-access 100

auto enp6s5f1
iface enp6s5f1
    bridge-access 100

auto vmbr0
iface vmbr0 inet manual
    bridge-ports enp5s4f0  enp6s5f0 enp6s5f1
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094

auto vmbr0.100
iface vmbr0.100 inet static
    address 10.2.0.223/24
    gateway 10.2.0.1
 

alyarb

New Member
Feb 11, 2020
17
1
3
33
Is anyone using this in production, even for the simple VLAN use case?

I can't seem to create a zone and net that doesn't get the warning icon. The pve gui bugs out forcing a full screen refresh, and VMs will not start that are attached to my net:


bridge 'testNet' does not exist
kvm: network script /var/lib/qemu-server/pve-bridge failed with status 512
TASK ERROR: start failed: QEMU exited with code 1
 

spirit

Famous Member
Apr 2, 2010
4,434
319
103
www.odiso.com
Is anyone using this in production, even for the simple VLAN use case?

I can't seem to create a zone and net that doesn't get the warning icon. The pve gui bugs out forcing a full screen refresh, and VMs will not start that are attached to my net:


bridge 'testNet' does not exist
kvm: network script /var/lib/qemu-server/pve-bridge failed with status 512
TASK ERROR: start failed: QEMU exited with code 1
Hi,
that's sound like the config is not correctly apply, and bridge/vnet "testNet" is not created.
are you sure to have used the apply button on the main sdn section part ?

if yes, can you provide /etc/pve/sdn/*.cfg , /etc/nework/interfaces , /etc/network/interfaces.d/sdn config files ?
 

alyarb

New Member
Feb 11, 2020
17
1
3
33
yikes, I don't know how I could have missed that. sorry. thanks for the great module.

when people outgrow the simple VLAN and have to go for more encapsulation, how do they handle reducing the mtu on thousands of nics? is there any real performance hit by lowering the mtu for the other modes?
 

spirit

Famous Member
Apr 2, 2010
4,434
319
103
www.odiso.com
yikes, I don't know how I could have missed that. sorry. thanks for the great module.

when people outgrow the simple VLAN and have to go for more encapsulation, how do they handle reducing the mtu on thousands of nics? is there any real performance hit by lowering the mtu for the other modes?
I'm personnaly using bigger mtu on physical interfaces (I can because It's my own network).

If you can't , 2 possibity: reduce mtu inside the guests os , or on qemu, it's possible to add ",mtu=...." to nic option (virtio nic only). Not yet available in gui yet.

I'm currently looking to auto add this option when mtu is defined on vnet, should be available soon.
 

alyarb

New Member
Feb 11, 2020
17
1
3
33
VLAN mode is working well in terms of creating/applying the config.

So I create a VXLAN zone called "SDN" with MTU 8950 and all my hosts' vmbr0 addresses in the peer list.

Then I create a net called testNet, tag 9999 and the rest on auto. When I hit apply, the "pending" zone turns to error.

Where can I see the error message?

Hotplug changing the NIC's bridge does not work. Tested on Windows and pfSense VM. Qemu-ga is working, but should not be required.

Hotplug error:

Parameter verification failed. (400)

net3: hotplug problem - can't add ovs port 'tap9902i3' - command '/usr/bin/ovs-vsctl -- add-port testNet tap9902i3' failed: open3: exec of /usr/bin/ovs-vsctl -- add-port testNet tap9902i3 failed: No such file or directory at /usr/share/perl5/PVE/Tools.pm line 444.
 

Attachments

  • interfaces.txt
    3 KB · Views: 5
  • sdn.txt
    680 bytes · Views: 7
  • vnets.txt
    53 bytes · Views: 3
  • zones.txt
    289 bytes · Views: 3
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!