Proxmox on a vlan

U

unkn0wnDnS

New Member
Mar 23, 2023
13
2
3
Hi all,

I been able to setup proxmox on a Cisco UCS.
I'm using Shared LOM ports, and on an access switch it's working fine.

Now I want to move the server to a core switch (cisco 4500x) instead.
The configuration is still the same, just like my access switch (cisco C2960x):
Code: 
int Te1/1/3
switchport mode trunk
switchport trunk native vlan 15
(the same counts for the port-channel of my switches, vlan15 is 100% available)

It should work perfectly fine.. However for some reason when I check the port, link and protocol is up.
But the mac addresses shows traffic on vlan 1, instead of 15 like it did on the cisco switch.

There is no mac address filtering or such. It's just weird.

Anyways, I now want to try and make the proxmox aware of trunk, and use a static IP within vlan 15: 10.217.180.245/23 - gateway: 10.217.180.1

I want to be able to connect to proxmox via ssh, webgui etc on 10.217.180.245

and later at some point when it works fine, i want to be able to set VM hosts on different VLAN's.

I tried a lot, and everytime when I get to the point of ie. vmbr0.15 systemctl complains that it's not supported.

When I try to install vlan: apt install vlan
it wants to delete proxmox-ve and ve-manager
X_X

I'm completely lost here, hopefully someone can help me?

my /etc/network/interfaces is completely empty now.
Code: 
auto lo
iface lo inet loopback

iface enp6s0 inet manual.  #this is the interface of the SFP

auto vmbr0
iface vmbr0 inet static
    address 10.217.180.245/23
    gateway 10.217.180.1
    bridge-ports enp6s0
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes

This works fine on the access switch and joins vlan 15 fine, but on the core switch it does not and the mac address table shows it got into vlan 1.
 
Last edited:
You could try something like this
Code: 
auto lo
iface lo inet loopback

auto enp6s0
iface enp6s0 inet manual  #this is the interface of the SFP

auto vlan0.15
iface vlan0.15 inet manual

auto vmbr0v15
iface vmbr0v15 inet static
    address 10.217.180.245/23
    gateway 10.217.180.1
    bridge-ports vlan0.15
    bridge-stp off
    bridge-fd 0
   
auto vmbr0
iface vmbr0 inet manual
    bridge-ports enp6s0
    bridge-stp off
    bridge-fd 0
    bridge-vids 2-4094
    bridge-vlan-aware yes

edit: vlan0.15, but I'm not really sure if this works. We're usually working with bonds to create channels of 2/3 NICs...
 
Last edited:
ceelight said:
You could try something like this
Code: 
auto lo
iface lo inet loopback

auto enp6s0
iface enp6s0 inet manual  #this is the interface of the SFP

auto vlan0.15
iface vlan0.15 inet manual

auto vmbr0v15
iface vmbr0v15 inet static
    address 10.217.180.245/23
    gateway 10.217.180.1
    bridge-ports vlan0.15
    bridge-stp off
    bridge-fd 0
 
auto vmbr0
iface vmbr0 inet manual
    bridge-ports enp6s0
    bridge-stp off
    bridge-fd 0
    bridge-vids 2-4094
    bridge-vlan-aware yes

edit: vlan0.15, but I'm not really sure if this works. We're usually working with bonds to create channels of 2/3 NICs...
Click to expand...
I have an additional SFP+ which I could use.
However, clients are only 1gbit, would it make sense to do LACP on 2x 10gbit?

Could you maybe share config for a bond for 2 trunked ports? If the vlan15 wont work, I'll try bonding the interfaces enp6s0 and enp7s0 with a trunked portchannel
 
Last edited:
If you are using a native vlan on the Cisco trunk port, then you shouldn't have to do any special config for Proxmox other than making the vmbr0 VLAN aware. Any untagged traffic coming into the trunk link will be VLAN15. This is exactly how I have mine set up. Below is my interface on my core switch:

Code: 
interface GigabitEthernet1/0/44
 description hw-vhost04-srv
 switchport trunk native vlan 110
 switchport trunk allowed vlan 25,100-104,107,108,110,112,113,115-118,121,254
 switchport mode trunk
 udld port

And here is my vmbr0 interface from Proxmox:
1738024934264.png

Then when I have a VM I want on another VLAN, here is the network hardware for that VM:
1738025157825.png
*Full disclosure, I simulated the above network device since I don't currently have any VMs on a different VLAN, but this is exactly how I set it up when I do.

EDIT: Here is a LXC that I have on another VLAN, and it works as expected as well.
1738025540539.png

Edit#2: I do nothing in the VM itself (IE: I don't configure the Windows OS to be on a VLAN). I simply configure it with DHCP or Static as I desire and let Proxmox handle to VLAN side of things. I hope that makes sense.
 

Attachments

  • 1738025076341.png
    1738025076341.png
    28.8 KB · Views: 3
Last edited:
ceelight said:
You could try something like this
Code: 
auto lo
iface lo inet loopback

auto enp6s0
iface enp6s0 inet manual  #this is the interface of the SFP

auto vlan0.15
iface vlan0.15 inet manual

auto vmbr0v15
iface vmbr0v15 inet static
    address 10.217.180.245/23
    gateway 10.217.180.1
    bridge-ports vlan0.15
    bridge-stp off
    bridge-fd 0
  
auto vmbr0
iface vmbr0 inet manual
    bridge-ports enp6s0
    bridge-stp off
    bridge-fd 0
    bridge-vids 2-4094
    bridge-vlan-aware yes

edit: vlan0.15, but I'm not really sure if this works. We're usually working with bonds to create channels of 2/3 NICs...
Click to expand...
Hey there, so I just tried as you said, but vlan0 is not a raw interface. So I changed it to enp6s0.15 instead.

Now the problem I see when I restart networking:
Code: 
error: netlink: enp6s0.15: cannot create vlan enpts0.15 15: operation failed with 'Operation not supported' (95)

This made me think that I'm missing out on the 'vlan' package, however running
Code: 
apt install vlan
It wants to delete proxmox-ve and ve-manager... Not sure how to get around it, but I did read a closed topic which said 'vlan is now in ve-manager'.
 
muzicman0 said:
If you are using a native vlan on the Cisco trunk port, then you shouldn't have to do any special config for Proxmox other than making the vmbr0 VLAN aware. Any untagged traffic coming into the trunk link will be VLAN15. This is exactly how I have mine set up. Below is my interface on my core switch:

Code: 
interface GigabitEthernet1/0/44
 description hw-vhost04-srv
 switchport trunk native vlan 110
 switchport trunk allowed vlan 25,100-104,107,108,110,112,113,115-118,121,254
 switchport mode trunk
 udld port

And here is my vmbr0 interface from Proxmox:
View attachment 81488

Then when I have a VM I want on another VLAN, here is the network hardware for that VM:
View attachment 81490
*Full disclosure, I simulated the above network device since I don't currently have any VMs on a different VLAN, but this is exactly how I set it up when I do.

EDIT: Here is a LXC that I have on another VLAN, and it works as expected as well.
View attachment 81491

Edit#2: I do nothing in the VM itself (IE: I don't configure the Windows OS to be on a VLAN). I simply configure it with DHCP or Static as I desire and let Proxmox handle to VLAN side of things. I hope that makes sense.
Click to expand...

You are absolutely correct about the native vlan.
The 'standard' config as I showed in the first post works perfectly fine a C2960x switch but on the 4500x for some reason the MAC address gets into vlan1. My external contact for our network is flabbergasted as well and told me to try and have the server tagged on vlan15 by itself already.
Issue might be because we're using the LOM port which is shared to the Operating System, and somehow to iOS of the 4500x core switch interprets the traffic differently compared to the c2960x access switch.
 
Even when I add a Linux VLAN through PVE's webgui, it fails because:
ifreload -a
error: netlink: vlan15: cannot create vlan vlan15 15: operation failed with 'Operation not supported' (95)
 
You must log in or register to reply here.
Top Bottom