[SOLVED] VLAN to KVM guest using OVS

M

MimCom

Active Member
Apr 22, 2011
204
3
38
Southwest NM
I've used OVS in PVE with VLANs previously, but the tagging was done by the guests.

I'm trying to convert an existing node form linuxbridge to OVS and seem to have missed something basic.
/etc/network/interfaces file looks like this:
Code: 
#auto lo
iface lo inet loopback

auto vmbr0

allow-ovs vmbr0
iface vmbr0 inet manual
    ovs_type OVSBridge
    ovs_ports eth0 vlan88 untagged

auto eth0
allow-vmbr0 eth0
allow-vmbr0 untagged
iface untagged inet static
    address  192.168.33.20
    netmask  255.255.255.0
    gateway  192.168.33.1
    ovs_type OVSIntPort
    ovs_bridge vmbr0

allow-vmbr0 vlan88
iface vlan88 inet static
    address  10.0.10.20
    netmask  255.255.255.0
    ovs_type OVSIntPort
    ovs_bridge vmbr0
    ovs_options tag=88
I am unable to ping either 192.168.33.1 (untagged router address) or 10.0.10.1 (VLAN 88 router address).

LXC guests (in vmbr0 from previous config) can ping the node at 192.168.33.20 but not the gateway.
 
Last edited:
Left out the OVSPort:
Code: 
iface eth0 inet manual
     ovs_bridge vmbr0
     ovs_type OVSPort

Now the untagged guests are working as expected, but the KVM guest I put into VLAN 88 can not ping the router. Network device is set bridge=vmbr0 tag=88, but the guest OS is not configured for tagging.
 
Last edited:
Helpfile says

tag=<integer> (1 - 4094)
VLAN tag to apply to packets on this interface.

I'm reading this as "tagged outside the guest and untagged inside." Is this a correct interpretation?
 
MimCom said:
Helpfile says

tag=<integer> (1 - 4094)
VLAN tag to apply to packets on this interface.

I'm reading this as "tagged outside the guest and untagged inside." Is this a correct interpretation?
Click to expand...


Yes that's correct.
 
MimCom said:
allow-vmbr0 vlan88
iface vlan88 inet static
address 10.0.10.20
netmask 255.255.255.0
ovs_type OVSIntPort
ovs_bridge vmbr0
ovs_options tag=88

[/CODE]
I am unable to ping either 192.168.33.1 (untagged router address) or 10.0.10.1 (VLAN 88 router address).

LXC guests (in vmbr0 from previous config) can ping the node at 192.168.33.20 but not the gateway.
Click to expand...
Try to change the above line:
ovs_options tag=88
To:
ovs_options tag=88 vlan_mode=native-untagged
 
Just for practice, I made the change in the GUI:
ovs-intport-config.png
Which produced this in /etc/network/interfaces:
Code: 
allow-vmbr0 vlan88
iface vlan88 inet static
        address  10.0.10.20
        netmask  255.255.255.0
        ovs_type OVSIntPort
        ovs_bridge vmbr0
        ovs_options tag=88 vlan_mode=native-untagged

Yet:
Code: 
root@pve1:~# ping 10.0.10.1
PING 10.0.10.1 (10.0.10.1) 56(84) bytes of data.
From 10.0.10.20 icmp_seq=1 Destination Host Unreachable
From 10.0.10.20 icmp_seq=2 Destination Host Unreachable
From 10.0.10.20 icmp_seq=3 Destination Host Unreachable
From 10.0.10.20 icmp_seq=4 Destination Host Unreachable
From 10.0.10.20 icmp_seq=5 Destination Host Unreachable
From 10.0.10.20 icmp_seq=6 Destination Host Unreachable
^C
--- 10.0.10.1 ping statistics ---
7 packets transmitted, 0 received, +6 errors, 100% packet loss, time 6122ms
 
MimCom said:
Just for practice, I made the change in the GUI:
View attachment 6295
Which produced this in /etc/network/interfaces:
Code: 
allow-vmbr0 vlan88
iface vlan88 inet static
        address  10.0.10.20
        netmask  255.255.255.0
        ovs_type OVSIntPort
        ovs_bridge vmbr0
        ovs_options tag=88 vlan_mode=native-untagged

Yet:
Code: 
root@pve1:~# ping 10.0.10.1
PING 10.0.10.1 (10.0.10.1) 56(84) bytes of data.
From 10.0.10.20 icmp_seq=1 Destination Host Unreachable
From 10.0.10.20 icmp_seq=2 Destination Host Unreachable
From 10.0.10.20 icmp_seq=3 Destination Host Unreachable
From 10.0.10.20 icmp_seq=4 Destination Host Unreachable
From 10.0.10.20 icmp_seq=5 Destination Host Unreachable
From 10.0.10.20 icmp_seq=6 Destination Host Unreachable
^C
--- 10.0.10.1 ping statistics ---
7 packets transmitted, 0 received, +6 errors, 100% packet loss, time 6122ms
Click to expand...
What does ip addr show?
Is this package installed? openvswitch-switch
Did you reboot the server after making the network changes?
 
Code: 
root@pve1:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master ovs-system state UP group default qlen 1000
    link/ether 00:1e:8c:f5:29:7d brd ff:ff:ff:ff:ff:ff
    inet6 fe80::21e:8cff:fef5:297d/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 00:1e:8c:f5:29:7e brd ff:ff:ff:ff:ff:ff
4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 8a:64:ec:be:e1:4a brd ff:ff:ff:ff:ff:ff
5: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 00:1e:8c:f5:29:7d brd ff:ff:ff:ff:ff:ff
    inet6 fe80::21e:8cff:fef5:297d/64 scope link
       valid_lft forever preferred_lft forever
6: vlan88: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 4a:74:5f:a8:04:03 brd ff:ff:ff:ff:ff:ff
    inet 10.0.10.20/24 brd 10.0.10.255 scope global vlan88
       valid_lft forever preferred_lft forever
    inet6 fe80::4874:5fff:fea8:403/64 scope link
       valid_lft forever preferred_lft forever
7: untagged: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 1e:d6:92:ac:9c:0a brd ff:ff:ff:ff:ff:ff
    inet 192.168.33.20/24 brd 192.168.33.255 scope global untagged
       valid_lft forever preferred_lft forever
    inet6 fe80::1cd6:92ff:feac:9c0a/64 scope link
       valid_lft forever preferred_lft forever
9: veth100i0@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP group default qlen 1000
    link/ether fe:82:8d:ac:85:ea brd ff:ff:ff:ff:ff:ff link-netnsid 0
11: veth101i0@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP group default qlen 1000
    link/ether fe:79:56:47:11:f5 brd ff:ff:ff:ff:ff:ff link-netnsid 1
12: tap103i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master ovs-system state UNKNOWN group default qlen 1000
    link/ether 9a:08:31:d7:cf:2c brd ff:ff:ff:ff:ff:ff

Code: 
root@pve1:~# dpkg -l|grep openv
ii  openvswitch-common                   2.7.0-2                        amd64        Open vSwitch common components
ii  openvswitch-switch                   2.7.0-2                        amd64        Open vSwitch switch implementations
root@pve1:~#

Yes, the server was rebooted after the changes.
 
And the relevant portion of 'ip addr' on the router (forked from VyOS):
Code: 
42: eth2.88@eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 04:18:d6:a0:8d:44 brd ff:ff:ff:ff:ff:ff
    inet 10.0.10.1/24 brd 10.0.10.255 scope global eth2.88
       valid_lft forever preferred_lft forever
    inet6 fe80::618:d6ff:fea0:8d44/64 scope link
       valid_lft forever preferred_lft forever
 
MimCom said:
Code: 
root@pve1:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master ovs-system state UP group default qlen 1000
    link/ether 00:1e:8c:f5:29:7d brd ff:ff:ff:ff:ff:ff
    inet6 fe80::21e:8cff:fef5:297d/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 00:1e:8c:f5:29:7e brd ff:ff:ff:ff:ff:ff
4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 8a:64:ec:be:e1:4a brd ff:ff:ff:ff:ff:ff
5: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 00:1e:8c:f5:29:7d brd ff:ff:ff:ff:ff:ff
    inet6 fe80::21e:8cff:fef5:297d/64 scope link
       valid_lft forever preferred_lft forever
6: vlan88: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 4a:74:5f:a8:04:03 brd ff:ff:ff:ff:ff:ff
    inet 10.0.10.20/24 brd 10.0.10.255 scope global vlan88
       valid_lft forever preferred_lft forever
    inet6 fe80::4874:5fff:fea8:403/64 scope link
       valid_lft forever preferred_lft forever
7: untagged: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 1e:d6:92:ac:9c:0a brd ff:ff:ff:ff:ff:ff
    inet 192.168.33.20/24 brd 192.168.33.255 scope global untagged
       valid_lft forever preferred_lft forever
    inet6 fe80::1cd6:92ff:feac:9c0a/64 scope link
       valid_lft forever preferred_lft forever
9: veth100i0@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP group default qlen 1000
    link/ether fe:82:8d:ac:85:ea brd ff:ff:ff:ff:ff:ff link-netnsid 0
11: veth101i0@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP group default qlen 1000
    link/ether fe:79:56:47:11:f5 brd ff:ff:ff:ff:ff:ff link-netnsid 1
12: tap103i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master ovs-system state UNKNOWN group default qlen 1000
    link/ether 9a:08:31:d7:cf:2c brd ff:ff:ff:ff:ff:ff

Code: 
root@pve1:~# dpkg -l|grep openv
ii  openvswitch-common                   2.7.0-2                        amd64        Open vSwitch common components
ii  openvswitch-switch                   2.7.0-2                        amd64        Open vSwitch switch implementations
root@pve1:~#

Yes, the server was rebooted after the changes.
Click to expand...
Looks ok. Is this module loaded into the kernel - 8021q ? (lsmod | grep 8021q)
 
  • Like
Reactions: MimCom
Bingo!
Code: 
root@pve1:~# lsmod | grep 8021q

root@pve1:~# modprobe 8021q

root@pve1:~# lsmod | grep 8021q
8021q                  32768  0
garp                   16384  1 8021q
mrp                    20480  1 8021q

Thank you for the help.

Perhaps PVE should load this module when/if a VLAN is defined in the GUI?
 
Yet no ping still:
Code: 
root@pve1:~# ping 10.0.10.1
PING 10.0.10.1 (10.0.10.1) 56(84) bytes of data.
From 10.0.10.20 icmp_seq=1 Destination Host Unreachable
From 10.0.10.20 icmp_seq=2 Destination Host Unreachable
From 10.0.10.20 icmp_seq=3 Destination Host Unreachable
From 10.0.10.20 icmp_seq=4 Destination Host Unreachable
From 10.0.10.20 icmp_seq=5 Destination Host Unreachable
From 10.0.10.20 icmp_seq=6 Destination Host Unreachable
^C
--- 10.0.10.1 ping statistics ---
7 packets transmitted, 0 received, +6 errors, 100% packet loss, time 6135ms
 
MimCom said:
Yet no ping still:
Code: 
root@pve1:~# ping 10.0.10.1
PING 10.0.10.1 (10.0.10.1) 56(84) bytes of data.
From 10.0.10.20 icmp_seq=1 Destination Host Unreachable
From 10.0.10.20 icmp_seq=2 Destination Host Unreachable
From 10.0.10.20 icmp_seq=3 Destination Host Unreachable
From 10.0.10.20 icmp_seq=4 Destination Host Unreachable
From 10.0.10.20 icmp_seq=5 Destination Host Unreachable
From 10.0.10.20 icmp_seq=6 Destination Host Unreachable
^C
--- 10.0.10.1 ping statistics ---
7 packets transmitted, 0 received, +6 errors, 100% packet loss, time 6135ms
Click to expand...
Add 8021q to /etc/modules and reboot. I think OVS might fail to configure vlan when the module is missing when initial configuring the network.
 
That was next on my list to try. All three modules are loaded, but still no connectivity.

Do kernel modules always load before startup scripts are run?
 
You are certain that your router (VyOS) is not blocking ping?
I would create a VM on 10.0.10.0/24 and test whether pings succeeds here.
 
There is currently no firewall ruleset on that interface and no reference to that /24 anywhere else in the router config.

I can delete eth2.88, assign 10.0.10.1/24 as a secondary address on eth2, remove the VLAN tag from the VM network options, start the VM, and it will ping the router interface.
 
Link doesn't clearly indicate which post, but if it's the one two posts up (about creating a VM on 10.0.10.0/24) perhaps I wasn't clear -- here's what I tried yesterday:

Added vlan_mode=native-untagged to VM OVS options.
Rebooted node, started VM106 -> no success connecting from existing VM106.

Loaded kernel module 8021q -> no success connecting from existing VM106

Destroyed VM106
Created new VM106, started it and opened a console.
Booted Ubuntu 16.04 minimal install, performed manual network config as 10.0.10.26 -> hangs trying to connect to a mirror.

Stopped VM 106.
Added 8021q to /etc/modules
Rebooted node
Started VM106.
Booted Ubuntu 16.04 minimal install, performed manual network config as 10.0.10.26 -> hangs trying to connect to a mirror.

Stopped VM 106.
Set VM106 hardware network device to no VLAN.
Deleted vlan88 OVS IntPort from node.
Rebooted node.
Deleted eth2.88 from router, added 10.0.10.1/24 as a secondary IP on eth2
Started VM106.
Booted Ubuntu 16.04 minimal install (which received a DHCP address from router in 192.168.33.0/24) and completed OS install.
Reboot VM106, edit /etc/network/interfaces to change from DHCP to static 10.0.10.26/24 with gateway of 10.0.10.1
Reboot VM106 and it has connectivity to the router and the world.

I should note that when vlan88 OVS IntPort was present on the node, the VM was able to ping 10.0.10.20.
 
VLAN-aware switch was blocking tagged VLAN packets. Added 88/tagged to both router and switch ports and we have ignition!

Thank you for the help on getting 8021q loaded.

I do believe the 8021q module should be configured for loading whenever a VLAN is added to a linux bridge or an OVS bridge using the PVE GUI. Another option would be to set a dependency on (or at least a recommendation for) 8021q in openvswitch.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back