[SOLVED] IPv6 problems with Hetzner (did work before)

[André Dierker]

Hello,

We have the following problem and I have no more ideas what could be wrong
. I can no longer reach two of our three hosts in a small cluster at Hetzner via IPv6. In my opinion, everything is set up correctly and it has already worked as it is set up. We have split up a larger cluster, kept two hosts and added another server, the new server is also accessible via IPv6. All of this actually worked until it stopped working...
What could I have missed?

The Hosts:
Hyperion (old)
Mimas (old)
Dione (new)

Facts:
3 Nodes, Proxmox 8.2.2
IPv4 (192.168.100.0/24) Cluster Network via vswitch
we removed seperate nics on the old nodes which we used as a ceph network

- ssh works fine from all nodes to all nodes
- I already renewed all ssh keys
- The web interface shows a ‘Connection error - Timeout’ when I want to edit Dione from Hyperion or Mimas.
- It is possible to edit Hyperion and Mimas from Dione web ui.
- IP forwarding s enabled
- I can't ping the old hosts
- same config does work on Dione

IPv6 is enabled

➜  ~ cat /proc/sys/net/ipv6/conf/all/disable_ipv6
0

/etc/network/interfaces

auto lo
iface lo inet loopback


iface enp7s0 inet manual


iface enp7s0.4000 inet manual
        mtu 1400


iface enp7s0.4001 inet manual
        mtu 1400


auto vmbr0
iface vmbr0 inet static
        address 65.xx.xx.xx/32
        gateway 65.21.79.129
        bridge-ports enp7s0
        bridge-stp off
        bridge-fd 0
#Hostnetwork


iface vmbr0 inet6 static
        address 2a01:4f9:xx:xxxx::2/64
        gateway fe80::1

[snip]

 

[André Dierker]

Funfact: After a reboot of one of the hosts the connection does work for a short time.

 

[poly01]

Hi,

"have you removed all servers then restarted the vSwitch and afterwards added them back? This is sometimes needed."

This is actually a statement which I got from Hetzner Support.

 

[mkorourke]

On one environment I've been what sounds like a similar issue. Other environments with same pve version and similar HW have been fine.

For the environment with issues, adding "bridge-mcsnoop 0" to our vmbr0/vlan-aware bridge has resolved. Ref https://forum.proxmox.com/threads/ipv6-neighbor-solicitation-not-forwarded-to-vm.96758/

 

[André Dierker]

Hi,

"have you removed all servers then restarted the vSwitch and afterwards added them back? This is sometimes needed."

This is actually a statement which I got from Hetzner Support.

Yes, I created a new vswitch because I got the same statement from Hetzner

 

[André Dierker]

On one environment I've been what sounds like a similar issue. Other environments with same pve version and similar HW have been fine.

For the environment with issues, adding "bridge-mcsnoop 0" to our vmbr0/vlan-aware bridge has resolved. Ref https://forum.proxmox.com/threads/ipv6-neighbor-solicitation-not-forwarded-to-vm.96758/

This didn't helped. This is for IPv6 communication on the bridge right? I cant reach the both old host from the internet via ipv6 too.

 

[mkorourke]

This didn't helped. This is for IPv6 communication on the bridge right? I cant reach the both old host from the internet via ipv6 too.

Communication both on the bridge and external to the bridge (over bond0) for host/node and VMs.

The combined config looking like:

auto vmbr0
iface vmbr0 inet manual
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
        mtu 9000
        bridge-mcsnoop 0
#Default VLAN Bridge

auto vmbr0.1
iface vmbr0.1 inet static
        address xx.xx.xx.144/24
        gateway xx.xx.xx.1
#Hypervisor Management

iface vmbr0.1 inet6 static
        address xxxx:xxxx:xxxx:xxxx::144/64
        gateway xxxx:xxxx:xxxx:xxxx::1

 

[André Dierker]

It was a network problem at Hetzner, the IPv6 Network was bound to a additional ip addres.

 

[rcd]

iface vmbr0.1 inet6 static
address xxxx:xxxx:xxxx:xxxx::144/64
gateway xxxx:xxxx:xxxx:xxxx::1 <<---- where do you get this from?

Normally, in my home lab, which of course use different IPv6 coonfig, I'd use
gateway fe80::1

But that's reported stale here....