Hetzner, PVE 7.0 problems after reboot. Maybe ifupdown2

F

Fischje

Renowned Member
Sep 25, 2014
64
1
73
Mönchengladbach/GER
Hello,
as my Proxmox story goes on, after updating my running proxmox 7.0 to some updates I reboot the system.

after that i can only connect with the kvm console.

no ping working.

Here my /etc/network/interfaces:
1626713886721.png

if i try ifreload -a i get this:
1626713923202.png

Can somebody find a mistake? i can only reach the machine if i use eno1 dirct without bridge as commentet part.
 
Last edited:
Okay, for now i figured out that if i uncomment the line with the pointopoint i cannot connect to the server.

my /etc/network/interrfaces looks now like this. maybee something wrong in the latest patches?

Bash: 
auto lo
iface lo inet loopback

iface eno1 inet manual

auto vmbr0
iface vmbr0 inet static
        address 78.xx.xx.03/32
        gateway 78.46.82.97
        hwaddress 0c:c4:yy:yy:38:c0
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0
#       pointopoint 78.xx.xx.97
        up sysctl -w net.ipv4.ip_forward=1
        up sysctl -w net.ipv4.conf.eno1.send_redirects=0

auto vmbr1
iface vmbr1 inet manual
        bridge-ports none
        bridge-stp off
        bridge-fd 0

I read this:
https://forum.proxmox.com/threads/p...t-tcp-only-works-sometimes.92664/#post-404760
As my server is standing at hetzner too maybee there is the problem?
 
Last edited:
I have exaclty the same issue and the same error message. It worked yesterday with PVE7 but the latest kernel patch made it impossible to connect to my server.
 
I restored the server from backup, removing the listed patches. This made the network work again.
Could please anyone from the proxmox Team look into this?

Error with the listed patches:
Code: 
Jul 22 09:08:03 pegasus networking[1336]: error: netlink: vmbr0: cannot add address xxx.xx.3.186/32 dev vmbr0: argument of type 'IPNetwork' is not iterable
Jul 22 09:08:03 pegasus networking[1336]: warning: vmbr0: up cmd 'ip route add 192.168.0.0/16 via xxx.xx.3.182 dev vmbr0' failed: returned 2 (Error: Nexthop has invalid gateway.
Jul 22 09:08:03 pegasus networking[1336]: )
Jul 22 09:08:03 pegasus networking[1336]: warning: vmbr0: up cmd 'ip route add 10.0.0.0/8 via xxx.xx.3.182 dev vmbr0' failed: returned 2 (Error: Nexthop has invalid gateway.
Jul 22 09:08:03 pegasus networking[1336]: )
Jul 22 09:08:03 pegasus systemd-udevd[957]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 22 09:08:03 pegasus systemd-udevd[957]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.

network config
Code: 
auto lo
iface lo inet loopback
iface enp193s0 inet manual

auto vmbr0
iface vmbr0 inet static
        address xxx.xx.3.186
        netmask 255.255.255.255
        gateway xxx.xx.3.129
        pointopoint xxx.xx.3.129
        bridge-ports enp193s0
        bridge-stp off
        bridge-fd 0
        up ip route add 192.168.0.0/16 via xxx.xx.3.182 dev vmbr0
        up ip route add 10.0.0.0/8 via xxx.xx.3.182 dev vmbr0
        up sysctl -w net.ipv4.ip_forward=1
        up sysctl -w net.ipv4.conf.enp193s0.send_redirects=0
#WAN

auto vmbr1
iface vmbr1 inet manual
        bridge-ports none
        bridge-stp off
        bridge-fd 0
#DMZ

auto vmbr2
iface vmbr2 inet manual
        bridge-ports none
        bridge-stp off
        bridge-fd 0
#LAN



After removing these patches network worked again:
Code: 
Start-Date: 2021-07-22  06:07:08
Commandline: /usr/bin/unattended-upgrade -d
Upgrade: proxmox-widget-toolkit:amd64 (3.3-4, 3.3-5)
End-Date: 2021-07-22  06:07:08

Start-Date: 2021-07-22  06:07:09
Commandline: /usr/bin/unattended-upgrade -d
Upgrade: libpam-systemd:amd64 (247.3-5, 247.3-6), libsystemd0:amd64 (247.3-5, 247.3-6), systemd:amd64 (247.3-5, 247.3-6)
End-Date: 2021-07-22  06:07:13

Start-Date: 2021-07-22  06:07:14
Commandline: /usr/bin/unattended-upgrade -d
Upgrade: lxc-pve:amd64 (4.0.9-2, 4.0.9-4)
End-Date: 2021-07-22  06:07:16

Start-Date: 2021-07-22  06:07:28
Commandline: /usr/bin/unattended-upgrade -d
Upgrade: pve-kernel-5.11.22-2-pve:amd64 (5.11.22-3, 5.11.22-4)
End-Date: 2021-07-22  06:07:52

Start-Date: 2021-07-22  06:07:53
Commandline: /usr/bin/unattended-upgrade -d
Upgrade: libproxmox-acme-plugins:amd64 (1.1.1, 1.2.0)
End-Date: 2021-07-22  06:07:53

Start-Date: 2021-07-22  06:07:54
Commandline: /usr/bin/unattended-upgrade -d
Upgrade: systemd-sysv:amd64 (247.3-5, 247.3-6)
End-Date: 2021-07-22  06:07:54

Start-Date: 2021-07-22  06:07:55
Commandline: /usr/bin/unattended-upgrade -d
Upgrade: libproxmox-acme-perl:amd64 (1.1.1, 1.2.0)
End-Date: 2021-07-22  06:07:58

Start-Date: 2021-07-22  06:07:59
Commandline: /usr/bin/unattended-upgrade -d
Upgrade: ifupdown2:amd64 (3.0.0-1+pve6, 3.1.0-1+pmx2)
End-Date: 2021-07-22  06:08:00

Start-Date: 2021-07-22  06:08:01
Commandline: /usr/bin/unattended-upgrade -d
Upgrade: udev:amd64 (247.3-5, 247.3-6), libudev1:amd64 (247.3-5, 247.3-6)
End-Date: 2021-07-22  06:08:17

Start-Date: 2021-07-22  06:08:18
Commandline: /usr/bin/unattended-upgrade -d
Upgrade: pve-manager:amd64 (7.0-9, 7.0-10)
End-Date: 2021-07-22  06:08:22

Start-Date: 2021-07-22  08:54:29
Commandline: apt dist-upgrade
Upgrade: linux-libc-dev:amd64 (5.10.40-1, 5.10.46-2)
End-Date: 2021-07-22  08:54:30
 
Ahh. Good example. I only downgraded my ifupdown2 package! And it works now. Can someone on proxmox take a look at this?

I think that in the Updates not only the MAC-adress related thing is changed.
 
Last edited:
I have the same problem......... PVE 7.0.4 BETA is running. With the update to 7.0.10 I lost my connection.

With the update the VMBR0 MAC address changed again but I´m not able to change it back to the Host´s MAC address. I get the same error.

It´s the update from ifupdown2 3.0.0-1 to 3.1.0-1. I downgraded it and now PVE 7.0.10 is working.
 
Last edited:
I wonder why no one from the Proxmox team writes anything about this. It seems to affect a number of users. And not having any access to a server in a data center is really a worst case scenario. At least a "we are looking into it" would be nice.

After all, we are talking about the enterprise repository.
 
> After all, we are talking about the enterprise repository.

Oh wow, that's bad! I thought only the no-subscription repository is affected, because my PVE (subscription with enterprise repo) was fine, but my PBS (no subscription, because it's just too expensive) didn't come up again.

But it seems like I just got lucky then, because my PVE still had the old ifupdown package instead of the modern ifupdown2 one.
 
Another one bite the dust. Just upgraded my PVE7, machine seems to come up but w/o (reachable) network.
But I knew this thread and marked it as "important", so I kept and downgraded ifupdown2 to ifupdown2_3.0.0-1 and everythin is okay again.
 
Another thing I'm stumbling on: I'm supporting a bunch of Hetzner servers (1*EX51, 1*AX41, 2*AX60). The AX41 has only one public IP address, all others a having three public IP addresses. All /etc/network/interfaces are having a line containing pointopoint xxx.xxx.xxx.xxx for eth0 according to Hetzner's Docs.
EX51 is still at PVE6. I've successfuly upgraded the AX41 to PVE7 a week ago. Went flawlessly and is running stable.
Upgrading the AX60's failed as one of them is hanging at "A start job is running for Network initialization (XXm / no limit)", the other is fully booting up but isn't reachable, so I've restored both from backup (thanks to PBS).

Today I've restored this backup w/o VMs to my local PVE6 for testing (PVE inside PVE/nested virtualisation), upgraded PVE6 to the latest update, made a fresh backup and upgraded this VM/PVE6 to PVE7. Went also flawlessly so far but is booting up to login without a reachable network. I've commented out the pointopoint line in /etc/network/interfaces then, rebooted the VM and everything seems to be okay now (can't test the public IP addresses).

So the question here is (or as it looks to me), that maybe the parameter pointopoint is the culprit (in connection with ifupdown2? Respectively when using more IP addresses)?

Working with PVE6 and PVE7 / one IP address
auto lo iface lo inet loopback auto eth0 iface eth0 inet static address xxx.xxx.xxx.xxx/26 gateway xxx.xxx.xxx.129 pointopoint xxx.xxx.xxx.xxx iface eth0 inet6 static address [IPv6 address]/64 gateway fe80::1 up sysctl -p auto vmbr0 iface vmbr0 inet static address 192.168.111.1/24 hwaddress aa:bb:cc:dd:ee:ff bridge-ports none bridge-stp off bridge-fd 0


Not working with PVE7 (but with PVE6) / thee IP addresses
auto lo iface lo inet loopback auto eth0 iface eth0 inet static address xxx.xxx.xxx.xxx/26 gateway xxx.xxx.xxx.129 pointopoint xxx.xxx.xxx.xxx iface eth0 inet static address [2nd IPv4 address]/26 iface eth0 inet static address [3rd IPv4 address]/26 iface eth0 inet6 static address [IPv6 address]/64 gateway fe80::1 up sysctl -p auto vmbr0 iface vmbr0 inet static address 192.168.222.1/24 hwaddress aa:bb:cc:dd:ee:ff bridge-ports none bridge-stp off bridge-fd 0
 
I'm using this on kernel 5.11.22-3-pve (and it worked on 5.11.22-2-pve)
on kernel 5.11.22-1-pve I needed to add hwaddress

Server is an AX51 with Proxmox VE 7

Code: 
auto lo
iface lo inet loopback

iface enp9s0 inet manual

auto vmbr0
iface vmbr0 inet static
        address <hauptip>/27
        gateway <gateway>
        bridge-ports enp9s0
        bridge-stp off
        bridge-fd 1
        bridge_hello 2
        bridge_maxage 12

auto vmbr1
iface vmbr1 inet manual
        bridge-ports none
        bridge-stp off
        bridge-fd 0

I have extra IP + extra Network both routed from Hetzner to the MAC of the VM which works as Firewall
 
TravisWilder said:
I'm using this on kernel 5.11.22-3-pve (and it worked on 5.11.22-2-pve)
on kernel 5.11.22-1-pve I needed to add hwaddress

Server is an AX51 with Proxmox VE 7

Code: 
auto lo
iface lo inet loopback

iface enp9s0 inet manual

auto vmbr0
iface vmbr0 inet static
        address <hauptip>/27
        gateway <gateway>
        bridge-ports enp9s0
        bridge-stp off
        bridge-fd 1
        bridge_hello 2
        bridge_maxage 12

auto vmbr1
iface vmbr1 inet manual
        bridge-ports none
        bridge-stp off
        bridge-fd 0

I have extra IP + extra Network both routed from Hetzner to the MAC of the VM which works as Firewall
Click to expand...

Without "pointopoint" you will sooner or later get a message from Hetzner that your server has been taken offline due to misconfiguration. Apart from the fact that anyone on your switch can read all your traffic.
 
Thalix said:
Without "pointopoint" you will sooner or later get a message from Hetzner that your server has been taken offline due to misconfiguration. Apart from the fact that anyone on your switch can read all your traffic.
Click to expand...
I had this but with the kernel -2 it was not working anymore and as the IP was not assigned to the brigde with errors
error: netlink: vmbr0: cannot add address <mainip>/27 dev vmbr0: argument of type 'IPNetwork' is not iterable
(which does not depend on the network mask - tried 32 as well)
 
TravisWilder said:
I had this but with the kernel -2 it was not working anymore and as the IP was not assigned to the brigde with errors
error: netlink: vmbr0: cannot add address <mainip>/27 dev vmbr0: argument of type 'IPNetwork' is not iterable
(which does not depend on the network mask - tried 32 as well)
Click to expand...
You can downgrade ifupdown2 to ifupdown2_3.0.0-1 (and pin it) - this works as well until the bug in ifupdown2 is fixed.
 
Just some explanation regarding the pointopoint requirement:
Your server IP usually resides in a /26 or /27 subnet. Inside this subnet is also your gateway. You could configure this /26 or /27 subnet and the gateway as normal. However, the Hetzner switch will drop all your packets that are not directed towards the gateway. With a /26 or /27 subnet configuration, your system attempts to directly communicate with IPs from that subnet, not going through the gateway. Due to Hetzner's safety measures you won't be able to reach your neighbors from the same subnet.
So you have to convince your system to direct all packets to the gateway and not do any direct communication with your neighbors. This can be done using the /32 subnet. However, your system then does not know on which network interface it can reach the gateway IP. That is exactly the purpose of the pointopoint configuration.
 
  • Like
Reactions: TravisWilder
Sebastian256 said:
Just some explanation regarding the pointopoint requirement:
Your server IP usually resides in a /26 or /27 subnet. Inside this subnet is also your gateway. You could configure this /26 or /27 subnet and the gateway as normal. However, the Hetzner switch will drop all your packets that are not directed towards the gateway. With a /26 or /27 subnet configuration, your system attempts to directly communicate with IPs from that subnet, not going through the gateway. Due to Hetzner's safety measures you won't be able to reach your neighbors from the same subnet.
So you have to convince your system to direct all packets to the gateway and not do any direct communication with your neighbors. This can be done using the /32 subnet. However, your system then does not know on which network interface it can reach the gateway IP. That is exactly the purpose of the pointopoint configuration.
Click to expand...
Thanks
I will reenter /32 and pointopoint after the update of ifupdown is out. Do not want to touch for now.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom