[HELP] Clustered SDN on single Nic nodes with managed switch

[javierspn]

Hi Team!

I have a proxmox cluster with two nodes. Each node is connected to a single trunk port on a managed switch.

This is the interfaces file for each node:

auto lo
iface lo inet loopback

iface enp0s31f6 inet manual

auto vmbr0
iface vmbr0 inet static
        bridge-ports enp0s31f6
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4092

auto vmbr0.110
        iface vmbr0.110 inet static
        address 172.26.1.10/24
        gateway 172.26.1.2


source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

iface enp0s31f6 inet manual

auto vmbr0
iface vmbr0 inet static
        bridge-ports enp0s31f6
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4092

auto vmbr0.110
        iface vmbr0.110 inet static
        address 172.26.1.11/24
        gateway 172.26.1.2


source /etc/network/interfaces.d/*

Each trunk port has the following defined VLANs:

100,110, 150, 160, 200, 250

Any VM or CT with the proper VLAN tag defined within its virtual network card works properly.

I am trying to create a VLAN SDN with a VNet with the "30" tag:



No matter the combination of settings I use, vms within the same SDN Zone, Vnet and subnet (if I create the latter) on different proxmox nodes are not able to se each other.

What am I doing wrong?

 

[shanreich]

Can you post your SDN configuration?

cat /etc/pve/sdn/*
cat /etc/network/interfaces.d/*

And the configuration of a guest:

qm config <vmid>

+ the network configuration of the guest

 

[javierspn]

Sure!

cat /etc/pve/sdn/*

subnet: fwpfs-10.10.10.0-24
        vnet fwpfnet

vnet: fwpfnet
        zone fwpfs
        tag 20

vlan: fwpfs
        bridge vmbr0
        ipam pve

cat /etc/network/interfaces.d/*

#version:14

auto fwpfnet
iface fwpfnet
        bridge_ports vmbr0.20
        bridge_stp off
        bridge_fd 0

node 1 config:

agent: 1
balloon: 0
boot: order=ide2;scsi0;net0
cores: 4
cpu: x86-64-v2-AES
ide2: none,media=cdrom
machine: q35
memory: 4096
meta: creation-qemu=8.1.5,ctime=1716917844
name: alcor
net0: virtio=BC:24:11:30:2A:6E,bridge=vmbr0,firewall=1,tag=200
net1: virtio=BC:24:11:FF:9C:3D,bridge=vmbr0,firewall=1,tag=100
net2: virtio=BC:24:11:03:DC:F3,bridge=vmbr0,firewall=1,tag=150
net3: virtio=BC:24:11:5F:01:0A,bridge=vmbr0,firewall=1,tag=250
net4: virtio=BC:24:11:C5:96:AD,bridge=vmbr0,firewall=1,tag=110
net5: virtio=BC:24:11:45:1E:D7,bridge=fwpfnet,firewall=1
numa: 0
ostype: l26
scsi0: zfs-nvme:vm-116-disk-0,iothread=1,size=64G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=c8f2bb41-c29d-409a-9494-ce0885e5db7f
sockets: 1
vmgenid: a1c5885e-ed65-4cb2-b09c-fb29c316fbde

node 2:

agent: 1
balloon: 0
boot: order=ide2;scsi0;net0
cores: 4
cpu: x86-64-v2-AES
ide2: none,media=cdrom
machine: q35
memory: 4096
meta: creation-qemu=8.1.5,ctime=1716917844
name: mizar
net0: virtio=BC:24:11:CF:09:6A,bridge=vmbr0,firewall=1,tag=200
net1: virtio=BC:24:11:E3:50:78,bridge=vmbr0,firewall=1,tag=100
net2: virtio=BC:24:11:9B:AA:45,bridge=vmbr0,firewall=1,tag=150
net3: virtio=BC:24:11:6B:75:FE,bridge=vmbr0,firewall=1,tag=250
net4: virtio=BC:24:11:55:8D:B4,bridge=vmbr0,firewall=1,tag=110
net5: virtio=BC:24:11:5E:6D:D6,bridge=fwpfnet,firewall=1
numa: 0
ostype: l26
scsi0: zfs-nvme:vm-117-disk-0,iothread=1,size=64G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=853be676-aaf3-454d-87d1-23a4f7008a38
sockets: 1
vmgenid: f7c31869-c5c8-41ef-9310-c766803effe5

Thanks a lot!

 

[spirit]

Each trunk port has the following defined VLANs:

100,110, 150, 160, 200, 250

do you mean your physical switch port ? Because you need to allow vlan30 too.

 

[javierspn]

Each trunk port has the following defined VLANs:

100,110, 150, 160, 200, 250

do you mean your physical switch port ? Because you need to allow vlan30 too.

Maybe that is the issue, and I suspected as much. VLAN 30 is not enabled.

I mean, by my understanding, I could create virtual VLANs within a Proxmox cluster, using SDN, but it makes total sense.

I suppose if I want something like that I shall use VxVLANs.

Thanks!

 

[spirit]

Maybe that is the issue, and I suspected as much. VLAN 30 is not enabled.

I mean, by my understanding, I could create virtual VLANs within a Proxmox cluster, using SDN, but it makes total sense.

I suppose if I want something like that I shall use VxVLANs.

Thanks!

Yes, you need to use vxlan in this case.

(or qinq (vlan in vlan) but I think that your hardware switch need to support it)