PVE 6.4 to 7 no network only for host

plokko · Oct 18, 2021

I have upgraded one of my servers from PVE 6.4 to 7; i tried the same procedure on other servers before doing it online and i had no problems but now i cannot access the PVE server (GUI, SSH or ping) but i can still reach all the VMs.

I can still have access to the machine by scheduling a KVM access but i cannot mantain it so i have to work fast on it.

To be clear ALL the VM do work and are reachable via internet; only the PVE host is unreachable and cannot connect to the internet (ping does not work).
The PVE host is connected via a single ethernet card (enp35s0, i checked, the name did not change) and it has a dedicated ip just for the PVE (it uses the default ip, the VM are MAC binded)

i checked the configuratio and everything seems in order

I tried adding hwaddress with the MAC taken from ip -c link but it did not help, i even tried with a different MAC address;
I already tried to restart network or reboot without avail.

Please do note that i have limited and scheduled time on the machine, i must plan ahead of time.

I don't know what may be the cause

NOTE: i do not have ifupdown2 installed and i cannot install/update any packages as there is no connection to the machine

Thanks in advance to anyone that tries to help me.

Moayad · Oct 20, 2021

Hello,

Please post the full network configuration for the host and the output of ip a.

plokko said:
I tried adding hwaddress with the MAC taken from ip -c link but it did not help, i even tried with a different MAC address;

I do not see the hwaddress? where you configured?

EDIT:
Also please post the output of pveversion -v

plokko · Oct 22, 2021

Moayad said:
Hello,

Please post the full network configuration for the host and the output of ip a.

Moayad said:
I do not see the hwaddress? where you configured?

This was a screenshot i saved before i added hwaddress (i added a line after gateway) and it worked; i do not have continuos access to the machine so it was the only screenshot available

Moayad said:
EDIT:
Also please post the output of pveversion -v

plokko · Oct 22, 2021

Ok, figured out myself:
i had to set the HWADDR the same as the physical ethernet card or the provider would block any request;
this worked differently for the secondary ips and i did not have to set it before so i didn't thought about that.

Moayad · Oct 22, 2021

Hi,

Thank you for the outputs!

regarding the hwaddress did you restart the host after you added that line to your network configuration?

otherwise, if you have physical access to your host I recommend you install the ifupdown2 [0]

[0] http://download.proxmox.com/debian/...n/binary-amd64/ifupdown2_3.1.0-1+pmx3_all.deb

joshin · Oct 22, 2021

@plokko Any chance you're at OVH?

I had to do the same thing on an upgraded server there, and this may have to do with their Failover IP weird networking requirements.

plokko · Oct 22, 2021

Moayad said:
Hi,

Thank you for the outputs!

regarding the hwaddress did you restart the host after you added that line to your network configuration?

otherwise, if you have physical access to your host I recommend you install the ifupdown2 [0]

[0] http://download.proxmox.com/debian/pve/dists/bullseye/pve-no-subscription/binary-amd64/ifupdown2_3.1.0-1+pmx3_all.deb

yes, i did reboot each time but i did not think about using the same MAC address of the ethernet card.
It's strange as i do have other Proxmox installation and they work without problems, don't know if my provider checks the MAC at first boot or whatever but now that i know it it's simple (i did not try it before because i was worried it could lead to a MAC confict)

joshin said:
@plokko Any chance you're at OVH?

I had to do the same thing on an upgraded server there, and this may have to do with their Failover IP weird networking requirements.

No, but i think it may be the same problem:
the bridge after the update will get a new MAC address and the service provider thinks it's a new VM on the machine and not the primary service (i bought additional IPs for other VMs and i had to set the MAC of the VM card to a specific one to assign the correct IP, for the primary IP i do not have this option).
I don't know if the problem will present itself on a clean installation or only after the update.

joshin · Oct 22, 2021

plokko said:
yes, i did reboot each time but i did not think about using the same MAC address of the ethernet card.
It's strange as i do have other Proxmox installation and they work without problems, don't know if my provider checks the MAC at first boot or whatever but now that i know it it's simple (i did not try it before because i was worried it could lead to a MAC confict)

No, but i think it may be the same problem:
the bridge after the update will get a new MAC address and the service provider thinks it's a new VM on the machine and not the primary service (i bought additional IPs for other VMs and i had to set the MAC of the VM card to a specific one to assign the correct IP, for the primary IP i do not have this option).
I don't know if the problem will present itself on a clean installation or only after the update.

My one fresh install - from a self-uploaded ISO worked fine.

Upgraded didn't - just like yours.

OVH has protections to keep spoofing, so when your vmbr? interface had a different HW id, that probably caused OVH to not pass traffic.

In my interfaces file, I added "hwaddress ether <mac address>" to the vmbr? config.

I suspect this is a kernel difference in how Proxmox 7 presents itself to the outside world. Not sure why it works at first, maybe OVH isn't actively checking, but runs a scan on traffic and then blocks?
I just know it's been perfect since the change.

chencho · Feb 3, 2022

Any way to upgrade 6 to 7 in ovh?

I'm in the same scenario.

Doing ip -c link I can get the Mac address from eth0, eth1, ...

Code:

6: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000

Then, I copy the Mac from eth0 to vmbr0

Code:

# network interfaces
    auto lo
    iface lo inet loopback

    iface enp193s0f0 inet manual

    auto vmbr0
    iface vmbr0 inet dhcp
      bridge-ports enp193s0f0
      bridge-stp off
      bridge-fd 0
      hwaddress a:b:c:d:e #MAC ADDRESS FORM ETH0

But I dont have any internet connection after reboot, I need to access it form rescue mode

joshin · Feb 3, 2022

@chencho - I don't see an IP Address set for vmbr0

Here's my current config that now works.

Code:

iface eno1 inet manual

auto vmbr0
iface vmbr0 inet static
        address 144.###.###.###
        netmask 255.255.255.0
        gateway 144.###.###.254
        bridge_ports eno1
        bridge_stp off
        bridge_fd 0

chencho · Feb 4, 2022

I change it to use static instead dhcp, and change enp193s0f0 to eno1, but seems have not luck

Code:

iface eno1 inet manual

auto vmbr0
iface vmbr0 inet static
        address 135.###.###.###
        netmask 255.255.255.0
        gateway 135.###.###.254
        address b8:XX:XX:XX:XX:XX
        bridge_ports eno1
        bridge_stp off
        bridge_fd 0

After reboot, I can't access to server

1. Fresh install 6.4 from ooh template
2. Upgrade to 7
3. Change interfaces
4. Reboot

Moayad · Feb 7, 2022

could you please post the output of the ip a (you can hide sensitive information such as the public IP)?

chencho · Feb 7, 2022

Sure

Code:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: bond0: <BROADCAST,MULTICAST,MASTER> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 42:9c:a1:50:xx:xx brd ff:ff:ff:ff:ff:ff
3: dummy0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 7a:21:1f:cb:xx:xx brd ff:ff:ff:ff:ff:ff
4: ifb0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 32
    link/ether 5e:20:51:a3:xx:xx brd ff:ff:ff:ff:ff:ff
5: ifb1: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 32
    link/ether de:f2:8a:17:xx:xx brd ff:ff:ff:ff:ff:ff
6: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether b8:ce:f6:a4:xx:xx brd ff:ff:ff:ff:ff:ff
7: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether b8:ce:f6:a4:xx:xx brd ff:ff:ff:ff:ff:ff
8: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 04:3f:72:d3:xx:xx brd ff:ff:ff:ff:ff:ff
9: eth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 04:3f:72:d3:xx:xx brd ff:ff:ff:ff:ff:ff
10: teql0: <NOARP> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 100
    link/void
11: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ipip 0.0.0.0 brd 0.0.0.0
12: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/gre 0.0.0.0 brd 0.0.0.0
13: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1476 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
14: erspan0@NONE: <BROADCAST,MULTICAST> mtu 1464 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
15: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/sit 0.0.0.0 brd 0.0.0.0
16: ip6tnl0@NONE: <NOARP> mtu 1452 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/tunnel6 :: brd :: permaddr 7e4f:8dbb:f542::

Eth0 is up

My current config

Code:

auto lo
iface lo inet loopback

iface enp193s0f0 inet manual

iface enp133s0f0 inet manual

iface enp133s0f1 inet manual

iface enp193s0f1 inet manual

iface eth0 inet manual

auto vmbr0
iface vmbr0 inet static
        address 135.xxx.xxx.xxx
        netmask 255.255.255.255
        gateway 135.xxx.xxx.254
        address b8:ce:f6:a4:xx:xx
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0

I also done:

Code:

wget http://download.proxmox.com/debian/pve/dists/bullseye/pve-no-subscription/binary-amd64/ifupdown2_3.1.0-1%2Bpmx3_all.deb
dpkg -i ifupdown2_3.1.0-1+pmx3_all.deb

Moayad · Feb 7, 2022

The main NIC in your PVE is: eth0 which means you have to set the bridge_ports to the eth0 instead of ~~eno1~~ since you don't have eno1 in the output of ip a command

so, the network config should be as the following:

Code:

iface eth0 inet manual ##<== Edit this

auto vmbr0
iface vmbr0 inet static
        address 135.###.###.###
        netmask 255.255.255.0
        gateway 135.###.###.254
        address b8:XX:XX:XX:XX:XX
        bridge_ports eth0 <== Edit this too
        bridge_stp off
        bridge_fd 0

hopefully, that helps

Moayad · Feb 7, 2022

After editing the network config do an ifreload -a command to take the effect.

chencho · Feb 7, 2022

No way

My actual config

Code:

auto lo
iface lo inet loopback

iface enp193s0f0 inet manual

iface enp133s0f0 inet manual

iface enp133s0f1 inet manual

iface enp193s0f1 inet manual

iface eth0 inet manual

auto vmbr0
iface vmbr0 inet static
        address 135.XXX.XXX.XXX
        netmask 255.255.255.255
        gateway 135.XXX.XXX.254
        address b8:ce:f6:xx:xx:xx
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0

Note my net mask is 255.255.255.255, instead 255.255.255.0

I can't do ifreload because I need to enter it in rescue mode, over chroot

chencho · Feb 7, 2022

I can access it over kvm

Code:

root@nsXXXXX:~# ifreload -a
[  593.580121] vmbr0: the hash_elasticity option has been deprecated and is always 16
error: vmbr0: convert string ip address into IPNetwork object: 'b8:ce:f6:xx:xx:xx' does not appear to be an IPv4 or IPv6 address

Obiusly, in b8:ce:f6:xx:xx:xx i set my Mac, no xx values.

EDIT!!

address instead hwaddress !!!!!!

EDIT 2!!

Same result

Code:

error: 'b8:ce:f6:xx:xx:xx' does not appear to be an IPv4 or IPv6 address

chencho · Feb 7, 2022

After remove hwaddress line, now works fine

Moayad · Feb 7, 2022

Glad that you solved the issue yourself!

aapkagan · Jul 10, 2022

chencho said:
After remove hwaddress line, now works fine

where did u you remove it from ??

PVE 6.4 to 7 no network only for host

Active Member

Proxmox Staff Member

Active Member

Active Member

Proxmox Staff Member

Renowned Member

Active Member

Renowned Member

Well-Known Member

Renowned Member

Well-Known Member

Proxmox Staff Member

Well-Known Member

Proxmox Staff Member

Proxmox Staff Member

Well-Known Member

Well-Known Member

Well-Known Member

Proxmox Staff Member

New Member