Changing primary network interface

lukaszp

New Member
Oct 18, 2022
1
0
1
Canada
Hi!

I am not certain if this even possible to do. I have a PVE Cluster with 3 nodes as a homelab. I am now moving it to a 10Gbps connectivity using a Mikrotik switch and three 2x 10Gbps cards.

I was able to switch Ceph cluster network which is on its own dedicated port, as well as VLANs used by my guests to the new NICs, but I cannot switch the PVE Admin / Ceph client network. For reference:

* VLAN10 - 10.0.0.0/24 - Admin, Ceph client network
* VLAN20 - 10.163.20.0/24 - Trusted systems
* VLAN21 - 10.163.21.0/24 - Untrusted systems
* VLAN100 - ISP
* VLAN101 - Ceph cluster

I've tried changing `bridge-ports` for `vmbr10` to `enp3s0f0` however the node is not able to communicate on that VLAN when this is done. Rebooting does not help. I've tried doing this through Web UI and via SSH. All other connections work. I also confirmed I'm receiving VLAN10 traffic so the switch is configured correctly.

Is there something else I am missing, or can this simply not be done and I need to wipe each node individually and re-install?

Here is my `interfaces` file from one of the nodes:

Code:
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eno1 inet manual
#office-sw 0/15

iface eno2 inet manual
#office-sw 0/16

iface enp3s0f0 inet manual
#pve-switch-1 sfp+1

iface enp3s0f1 inet manual
#pve-switch-1 sfp+4

auto vmbr10
iface vmbr10 inet static
    address 10.0.0.41/24
    gateway 10.0.0.1
    bridge-ports eno1.10
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094
#bridge-ports enp3s0f0.10
#VLAN 10 Admin

auto vmbr100
iface vmbr100 inet manual
    bridge-ports eno1.100
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094
#VLAN 100 ISP

auto vmbr101
iface vmbr101 inet static
    address 10.0.101.41/24
    bridge-ports enp3s0f1.101
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094
#VLAN 101 Ceph Cluster

auto vmbr20
iface vmbr20 inet static
    address 10.163.20.41/24
    bridge-ports enp3s0f0.20
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094
#VLAN 20 Trusted

auto vmbr21
iface vmbr21 inet static
    address 10.163.21.41/24
    bridge-ports enp3s0f0.21
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094
#VLAN 21 Untrusted
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!