Upgraded from 7 to 8 and Proxmox nodes now connect to NFS server from different source ethernet

[thjoker]

I'm testing rollout of proxmox 7 to 8 and have upgraded one of the nodes in a cluster.

When connecting to an NFS share it connects using the corosync NIC instead of the primary NIC when it previously used the primary.

This is the network config file:
cat /etc/network/interfaces auto lo iface lo inet loopback iface eno3 inet manual iface eno4 inet manual iface eno1 inet manual iface eno2 inet manual auto vmbr0 iface vmbr0 inet manual bridge-ports eno1 bridge-stp off bridge-fd 0 bridge-vlan-aware yes bridge-vids 2-4094 auto vmbr0.12 iface vmbr0.12 inet static address 10.0.2.17/24 gateway 10.0.2.1 auto vmbr1 iface vmbr1 inet static address 10.0.3.17/24 gateway 10.0.3.1 bridge-ports eno2 bridge-stp off bridge-fd 0 auto vmbr2 iface vmbr2 inet static address 10.0.4.17/24 gateway 10.0.4.1 bridge-ports eno3 bridge-stp off bridge-fd 0 auto vmbr3 iface vmbr3 inet manual bridge-ports eno4 bridge-stp off bridge-fd 0

It used to connect from 10.0.2.17 as was required, it now connects from 10.0.4.17 which is wrong as that NIC and switch are for corosync only data. All the other nodes are configured in the same way and all 7 other ones which are on version 7.4-16 work correctly.


Very strange....

 

[thjoker]

Just did the same with another node and it also switched to the 10.0.4

 

[Spoonman2002]

Just a quick question....how did you add all those different gateways, because when I try that Proxmox says "gateway already configured on vmbr0".
Or is that a cluster "thing", and I have no cluster in my homelab..

 

[thjoker]

They’re configured in the interfaces file, they’re all on individually configured VLANs as per their switch port aside from vmbr0.12 which is on a VLAN aware switch port.

 

[Spoonman2002]

Never knew you could add these gateways via the config file.
But why does the webgui complain about adding more than one gateway.....?
Thanks for this information!

 

[thjoker]

Technically I only need one, the other networks don’t need internet. But it worked and I didn’t change the interface file for years

 

[Spoonman2002]

is this your problem:
https://pve.proxmox.com/wiki/Upgrade_from_7_to_8#Network_Interface_Name_Change

 

[thjoker]

Removed the extraneous gateways from the interfacses config and hey presto, reverts back to the primary interface.

Thank you everyone for your help ;-)

cat /etc/network/interfaces
auto lo
iface lo inet loopback

iface eno3 inet manual

iface eno4 inet manual

iface eno1 inet manual

iface eno2 inet manual

auto vmbr0
iface vmbr0 inet manual
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

auto vmbr0.12
iface vmbr0.12 inet static
        address 10.0.2.17/24
        gateway 10.0.2.1

auto vmbr1
iface vmbr1 inet static
        address 10.0.3.17/24
        bridge-ports eno2
        bridge-stp off
        bridge-fd 0

auto vmbr2
iface vmbr2 inet static
        address 10.0.4.17/24
        bridge-ports eno3
        bridge-stp off
        bridge-fd 0

auto vmbr3
iface vmbr3 inet manual
        bridge-ports eno4
        bridge-stp off
        bridge-fd 0

 

[thjoker]

That broke the clustering on the non-updated ones after issuing service networking restart so I had to re-enter the 10.0.3. and the 10.0.4. statics after reboot...

May need to update them to version 8 before proceeding if anyone else has the same problem, the version 8 ones kept the static addresses

 

[Spoonman2002]

So........multiple gateways will mess up PVE network?
One gateway is oke.
Got to change my /etc/network/interfaces back again...

 

[thjoker]

It worked ok on v7 but v8 it caused issues with