[SOLVED] Communication over VLANs

[Bugbear]

Hi,

I'm currently splitting up my network in multiple subnets and VLANs but stumbled on something with Proxmox, I don't quite understand:


What do we create these subinterfaces (vmbr0.100, vmbr0.200) for, as in the final configuration of VMs/CTs we won't select any of those, but rather the parent interface and specify the VID manually.
And secondly, in general, when there are multiple interfaces configured, which one does PVE use for its own communication?

Maybe I'm overthinking things ... but currently I'm kind of stuck with these thoughts ^^
Any clarification is well appreciated!

 

[pille99]

define: own communication ?
if you mean the proxmox services itself like HA, Ceph - its configured on the networks and services
if you mean the VMs - no, communication at all, you need to do a trick with SDN (which is a joke itself - a "enterprise" grade software which doesnt have implemented the most basic funktion you expect from an hypervisior software which potentially runs on multiple server per cluster)

 

[Bugbear]

define: own communication ?
if you mean the proxmox services itself like HA, Ceph - its configured on the networks and services

Good question ... let's say apt on Proxmox is fetching the newest repositories; Which interface will the traffic go on?

 

[pille99]

management

 

[Bugbear]

What do you mean by that? Which of those (sub)interfaces will Proxmox use and is this configurable?

 

[Dunuin]

Good question ... let's say apt on Proxmox is fetching the newest repositories; Which interface will the traffic go on?

You will get SSH and webUI on all interfaces with an IP (so vmbr0.100 and vmbr0.200). For internet access (so apt too) it will use the interface with a gateway set (so vmbr0.100).

 

[Bugbear]

Thanks @Dunuin! This makes perfectly sense ... As you (usually) should have only one default gateway.
Sometimes you can't see the forest for the trees

 

[Bugbear]

Still I'm not sure why you should create those VLAN sub-interfaces (vmbr0.100, vmbr0.200).
What is the benefit of creating such a sub-interface inside of Proxmox when you also set the VID inside the VM settings?

 

[Dunuin]

Those VLAN sub-interfaces (vmbr0.100, vmbr0.200) are just for the host, not for the guests.
If you want your PVE management interface and gateway to be in a specific vlan, you have to do it that way. Otherwise PVE's webUI/SSH and internet access will not use any VLAN at all and PVE will try to send/receive untagged packets only.

 

[Bugbear]

Ah, okaaay ... So to make sure I understood it correctly: Creating multiple VLAN sub-interfaces (vmbr0.100 & vmbr0.200) wouldn't make lots of sense as you (normaly) can only have one gateway for the PVE management, right?

EDIT: I just realized myself that the above sentence is nonsense. In practical terms you could want to have a default gateway with sub-interface VLAN100 for PVE Management and setup a second sub-interface VLAN200 (without gateway!) e.g. for direct connection to a NAS/SAN.
Thank you for the explanations @Dunuin!

 

[Dunuin]

Ah, okaaay ... So to make sure I understood it correctly: Creating multiple VLAN sub-interfaces (vmbr0.100 & vmbr0.200) wouldn't make lots of sense as you (normaly) can only have one gateway for the PVE management, right?

You can only have one gateway, but as many management interfaces as you like, so your webUI/SSH is accessible from multiple subnets/VLANs.
Also keep in mind that the IPs you assign for PVE are used for more than just management. Let us say you install a SMB server directly on your PVE host and you got guests in multiple VLANs that need to access those SMB shares. Then you need to create such a VLAN sub-interface for each vlan and give each a IP so the guests got a IP in their vlan to access those NDS/SMB shares. Or let us say you set up some monitoring tools like a Zabbix server VM, then that VM also needs a IP of the PVe host in the same VLAN to query metrics from the PVE host.

EDIT: I just realized myself that the above sentence is nonsense. In practical terms you could want to have a default gateway with sub-interface VLAN100 for PVE Management and setup a second sub-interface VLAN200 (without gateway!) e.g. for direct connection to a NAS/SAN.
Thank you for the explanations @Dunuin!

Jup.