proxmox 6.2 sdn beta test

frybin

New Member
Jul 18, 2021
9
1
3
22
ok, got it. do you use proxmox firewall on theses nodes ? (I'm not sure from where is coming the tcp reset). The routing seem to be ok.
I don't use the proxmox firewall and have it turned off on the Datacenter and Node Level I think.
 

aderumier

Active Member
May 14, 2013
206
18
38
I don't use the proxmox firewall and have it turned off on the Datacenter and Node Level I think.
Maybe try on the exit node : sysctl -w net.ipv4.conf.all.rp_filter=0

I ll be back from holiday next week, and i ll do more tests
 

frybin

New Member
Jul 18, 2021
9
1
3
22
Maybe try on the exit node : sysctl -w net.ipv4.conf.all.rp_filter=0

I ll be back from holiday next week, and i ll do more tests
Running
Code:
sysctl -w net.ipv4.conf.all.rp_filter=0
on the exit node did not work.
 

spirit

Famous Member
Apr 2, 2010
4,875
423
103
www.odiso.com
Running
Code:
sysctl -w net.ipv4.conf.all.rp_filter=0
on the exit node did not work.
Hi,
I'm back from holiday.

can you try

sysctl -w net.ipv4.tcp_l3mdev_accept=1

on the exit-node, then restart ssh or pveproxy.
Then you should be able to join the exitnode ip from the vm.

(I don't known about other nodes (non exitnodes) of this cluster, do you have problem too ? because it should be routed like yours others clusters nodes.)
 

tisc0

Member
Jul 17, 2017
3
0
21
45
Hello,
Not sure if I'm supposed to push my specific problem here or create a new topic ?

Let's go, I guess you'll tell me or move it if it's not appropriate.

Last week, I was playing successfully with 2 clusters and SDN vxlan, with vnet non-vlan-aware, and subnets (let's assume I've been reading properly but maybe not perfectly the documentation ? Multiple times, though, and it's quite short).

Today, in another one, freshly and automatically installed by Scaleway (proxmox 6.4-13), and using what they call RPNv2 (supposed to be a VXLAN able to transport whatever we need in it), I get errors while trying to create vNIC in containers or VMs :

Screenshot from 2021-08-02 17-33-47.png
Clic ok, we're back in the config window. Clic ok again :

Screenshot from 2021-08-02 17-33-59.png

Here is the config in the /etc/network/interfaces of the 2 nodes in that cluster :


Bash:
auto lo
iface lo inet loopback

iface ens3f0 inet manual

iface ens3f1 inet manual
        mtu 9000

# WAN IP
auto vmbr0
iface vmbr0 inet static
        address xx.xx.xx.xx/24
        gateway xx.xx.xx.xx
        bridge-ports ens3f0
        bridge-stp off
        bridge-fd 0


# Preparing LAN interface
auto vmbr1
iface vmbr1 inet manual
        bridge-ports ens3f1
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
        mtu 8900

# Attaching a VLAN on vmbr1 - I could attach many, all given by service provider Scaleway
# This is the network used to create the cluster
auto vmbr1.2017
iface vmbr1.2017 inet static
        address 10.20.17.2/24
        mtu 8800

## I also tried with this very straight forward config, but same errors occured:
#auto ens3f1.2017
#iface ens3f1.2017 inet static
#       address 10.20.17.1/24


source /etc/network/interfaces.d/*

On the other node, it's similar, with 10.20.17.1/24 for the LAN (and its own public IP).
This network has bee used to create the cluster and enroll the nodes :

Bash:
root@mynode1:~# pvecm status
Cluster information
-------------------
Name:             ClusterV2
Config Version:   2
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Mon Aug  2 18:10:43 2021
Quorum provider:  corosync_votequorum
Nodes:            2
Node ID:          0x00000001
Ring ID:          1.43
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   2
Highest expected: 2
Total votes:      2
Quorum:           2 
Flags:            Quorate

Membership information
----------------------
    Nodeid      Votes Name
0x00000001          1 10.20.17.1 (local)
0x00000002          1 10.20.17.2


I don't get what I did wrong. Only a vnet VLAN-AWARE is working (and then not possible anymore to define subnets).

Thanks for any help, sorry if I didn't give you some crucial material to hemp your understanding, will push whatever you need.
 

spirit

Famous Member
Apr 2, 2010
4,875
423
103
www.odiso.com
@tisc0

can you send /etc/pve/sdn/*.cfg files ?

when you configure non-vlanware vnet (this should be the default anyway, until you want to propage vlan on top of vxlan), do you set any vlan tag in the vm nic options ? (this should be forbid)
 

tisc0

Member
Jul 17, 2017
3
0
21
45
Hi @spirit !
Thank you, it works. Sorry for that non-sense of mine, I indeed put a VLAN ID in the VM NIC options, and it's actually not forbidden.
Could you help too about the right value of MTU. Our service provider VLAN accept 9000, should I reduce it in the zone params or somewhere else ?
Thanks again
 

spirit

Famous Member
Apr 2, 2010
4,875
423
103
www.odiso.com
Hi @spirit !
Thank you, it works. Sorry for that non-sense of mine, I indeed put a VLAN ID in the VM NIC options, and it's actually not forbidden.
ok. gui still need support for this, I'll try to send patch soon. (and at least, send a correct error message)

Could you help too about the right value of MTU. Our service provider VLAN accept 9000, should I reduce it in the zone params or somewhere else ?
Thanks again
if you use vxlan, you need to lower 50bytes, so 8850 max. you can setup it in the zone, but it should also be done inside the guest. (default is 1500 in guest anyway)
 
  • Like
Reactions: tisc0

frybin

New Member
Jul 18, 2021
9
1
3
22
Hi,
I'm back from holiday.

can you try

sysctl -w net.ipv4.tcp_l3mdev_accept=1

on the exit-node, then restart ssh or pveproxy.
Then you should be able to join the exitnode ip from the vm.

(I don't known about other nodes (non exitnodes) of this cluster, do you have problem too ? because it should be routed like yours others clusters nodes.)
HI @spirit It ended up working, thanks for the help. I don't have other nodes added to this cluster since I am still testing new features out.
 
  • Like
Reactions: spirit

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!