IPv6 Connection lost

[Volker Lieder]

Hi,

we have a promox cluster based on Virtual Environment 5.4-13.
We have some Debian clients which lose ipv6 connectivity sporadic.
If we insert

ip -6 addr del 2a00:d0c0:xxx::xxx/64 dev ens18 and
ip -6 addr add 2a00:d0c0:xxx::xxx/64 dev ens18

the ipv6 communication is back again.

Any ideas what could happen?

Best regards,
Volker

 

[mira]

Please provide the network config (cat /etc/network/interfaces, mask any public IPs) and the VM config (qm config <vmid>) as well as the output of pveversion -v.

 

[Volker Lieder]

Please provide the network config (cat /etc/network/interfaces, mask any public IPs) and the VM config (qm config <vmid>) as well as the output of pveversion -v.

Hi,

here is the output:

pveversion -v
proxmox-ve: 5.4-2 (running kernel: 4.15.18-24-pve)
pve-manager: 5.4-13 (running version: 5.4-13/aee6f0ec)
pve-kernel-4.15: 5.4-12
pve-kernel-4.15.18-24-pve: 4.15.18-52
pve-kernel-4.15.18-11-pve: 4.15.18-34
pve-kernel-4.13.8-1-pve: 4.13.8-27
ceph: 12.2.12-1~bpo90+1
corosync: 2.4.4-pve1
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: not correctly installed
libjs-extjs: 6.0.1-2
libpve-access-control: 5.1-12
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-56
libpve-guest-common-perl: 2.0-20
libpve-http-server-perl: 2.0-14
libpve-storage-perl: 5.0-44
libqb0: 1.0.3-1~bpo9
lvm2: 2.02.168-pve6
lxc-pve: 3.1.0-7
lxcfs: 3.0.3-pve1
novnc-pve: 1.0.0-3
proxmox-widget-toolkit: 1.0-28
pve-cluster: 5.0-38
pve-container: 2.0-41
pve-docs: 5.4-2
pve-edk2-firmware: 1.20190312-1
pve-firewall: 3.0-22
pve-firmware: 2.0-7
pve-ha-manager: 2.0-9
pve-i18n: 1.1-4
pve-libspice-server1: 0.14.1-2
pve-qemu-kvm: 3.0.1-4
pve-xtermjs: 3.12.0-1
qemu-server: 5.0-54
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3

For Network on that host we use netplan:

/etc# cat netplan/50-cloud-init.yaml
# This file is generated from information provided by
# the datasource. Changes to it will not persist across an instance.
# To disable cloud-init's network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
ethernets:
ens18:
addresses: [172.16.48.11/24, "2a00:d0c0:XXXX:XXXX::11/64"]
dhcp4: false
gateway4: 172.16.48.1
gateway6: 2a00:d0c0:XXXX:XXXX::1
nameservers:
addresses:
- 37.2XX.XXX.XXX
- 37.2XX.XXX.XXX
- "2a00:d0c0:XXXX:XXXX::100"
- "2a00:d0c0:XXXX:XXXX::100"
search:
- domain.com
version: 2


qm config 140
bootdisk: scsi0
cores: 2
ide2: nfs-back:140/vm-140-cloudinit.qcow2,media=cdrom,size=5M
ipconfig0: ip=172.16.48.12/24,gw=172.16.48.1
memory: 8192
name: web01.domain.com
net0: virtio=0A:89:92:1B:AB:CC,bridge=vmbr1,tag=408
numa: 0
ostype: l26
scsi0: ssd02:vm-140-disk-1,size=150G
scsihw: virtio-scsi-pci
smbios1: uuid=b4c43976-829a-4673-9047-3328ab58c09d
sockets: 2

We have several hosts with that problem, no problem in Xen-based hosts or dedicated hosts without proxmox.

For further information, we use pfsense as firewall on extra virtual proxmox machine. But if the error happens, the host cant ping6 a direct neighbour.

Best regards,
Volker

 

[mira]

Please use /etc/network/interfaces on the host. It would allow us to rule out netplan as the source of the issue.
Neither netplan, nor systemd-networkd nor anything else other than /etc/network/interfaces is supported by us.

 

[mifr]

I have this patch to make neighbour discovery work correctly on my hosts:
/etc/network/interfaces:

iface vmbr0 inet static
    post-up echo -n 0 > /sys/class/net/vmbr0/bridge/multicast_snooping
    post-up echo -n 1 > /sys/class/net/vmbr0/bridge/multicast_querier

 

[Volker Lieder]

Hi, we can test to use interfaces, but a host which uses /etc/network/interfaces has the same error:

cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto ens18
iface ens18 inet static
address 172.16.57.43
netmask 255.255.255.0
network 172.16.57.0
broadcast 172.16.57.255
gateway 172.16.57.1
# dns-* options are implemented by the resolvconf package, if installed
dns-nameservers 37.2XX.XXX.XXX 37.2XX.XXX.X
dns-search domain.com

iface ens18 inet6 static
address 2a00:d0c0:XXXX:XXXX::201
netmask 64
up /sbin/ip -6 r a default via 2a00:d0c0:XXXX:XXXX::1 dev ens18

db02:~# ping6 2a00:d0c0:XXXX:XXXX::1
PING 2a00:d0c0:XXXX:XXXX::1(2a00:d0c0:XXXX:XXXX::1) 56 data bytes
^C
--- 2a00:d0c0:XXXX:XXXX::1 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2028ms

db01:~# ping6 2a00:d0c0:XXXX:XXXX::1
PING 2a00:d0c0:XXXX:XXXX::1(2a00:d0c0:XXXX:XXXX::1) 56 data bytes
64 bytes from 2a00:d0c0:XXXX:XXXX::1: icmp_seq=1 ttl=64 time=0.408 ms
64 bytes from 2a00:d0c0:XXXX:XXXX::1 icmp_seq=2 ttl=64 time=0.434 ms
^C
--- 2a00:d0c0:XXXX:XXXX::1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1003ms
rtt min/avg/max/mdev = 0.408/0.421/0.434/0.013 ms

db02:~# ip neigh show
172.16.57.44 dev ens18 lladdr 1e:b4:c2:79:de:60 STALE
172.16.57.1 dev ens18 lladdr a2:3c:54:0f:3f:df DELAY
2a00:d0c0:XXXX:XXXX::1 dev ens18 FAILED

db01:~# ip neigh show
172.16.57.41 dev ens18 lladdr ae:14:5c:c3:98:38 REACHABLE
172.16.57.44 dev ens18 lladdr 1e:b4:c2:79:de:60 STALE
172.16.57.1 dev ens18 lladdr a2:3c:54:0f:3f:df DELAY
2a00:d0c0:XXXX:XXXX::1 dev ens18 lladdr a2:3c:54:0f:3f:df router STALE
fe80::a03c:54ff:fe0f:3fdf dev ens18 lladdr a2:3c:54:0f:3f:df router STALE

The echo command above doesnt work on the client.

Regards,
Volker

 

[mifr]

The echo command should be on the host.

The echo command above doesnt work on the client.

Regards,
Volker

The echo command should be on the host

 

[Volker Lieder]

Hi, thank you, i tested the echo on 2 affected hosts and the gateway is available now.
I will configure it on all our hosts and will check in a few days and come back to you.

Best regards,
Volker

 

[jcmraats]

The two echo commands doesn't work on my proxmox host. The FreeBSD client loses it;s ipv6 connection after a while.
Anyone ideas?