VXLan Error 512 disable_ipv6

I

IamLunchbox

Member
Oct 17, 2022
16
3
8
Dear Proxmox-Team and other users,
I am currently working on a proxmox-lab combining a root server and a lab server at home. I've establshed an VXNat-Zone at the cluster level and a Vnet based on this.

After some testing I changed the network-setup of my Cloud-Init Guest and since this point, qemu is unable to boot this system. The following error is reported:

Code: 
generating cloud-init ISO
failed to open /proc/sys/net/ipv6/conf/tap101i0/disable_ipv6 for writing: No such file or directory
kvm: -netdev type=tap,id=net0,ifname=tap101i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on: network script /var/lib/qemu-server/pve-bridge failed with status 512
start failed: QEMU exited with code 1

The error seems to be caused by using a Vnet in conjunction with Cloud-Init - using any other network device besides the Vnet works flawlessly. Since this error starts to appear, I am completely unable to use a VNet with cloud-init.

What i've tried so far to resolve this:
- Triage using different clones and cloud-init configurations
- Restart network services
- Remove and add the vnet
- Remove and add the Zone
- Several reboots of the home-hypervisor

After all this triage, using ifreload -c results into: warning: error writing to file /proc/sys/net/ipv6/conf/test/forwarding([Errno 2] No such file or directory: '/proc/sys/net/ipv6/conf/test/forwarding'). I don't use ipv6 in my home-network, due to some VPN-issues as of now. Could this be the root-case? The 2nd node has ipv6 enabled.

My Network config:
/etc/network/interfaces
Code: 
source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

auto enp0s31f6
iface enp0s31f6 inet static
        address [redacted]
        gateway [redacted]

auto vmbr0
iface vmbr0 inet static
        address 10.0.0.1/30
        bridge-ports none
        bridge-stp off
        bridge-fd 0
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up iptables -t nat -A POSTROUTING -s '10.0.0.2/30' -o enp0s31f6 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '10.0.0.2/30' -o enp0s31f6 -j MASQUERADE

/etc/network/interfaces.d/sdn
Code: 
#version:14

auto test
iface test
        bridge_ports vxlan_test
        bridge_stp off
        bridge_fd 0
        mtu 1000

auto vxlan_test
iface vxlan_test
        vxlan-id 10
        vxlan_remoteip 192.168.254.3
        mtu 1000
 
Last edited:
do you have disabled ipv6 globally on your host ? (with sysctl for example ?)

for the tap interface ,they are already some check in the code to check if "/proc/sys/net/ipv6" exist
before disabling ipv6,so this is strange.

Code: 
sub disable_ipv6 {
    my ($iface) = @_;
    return if !-d '/proc/sys/net/ipv6'; # ipv6 might be completely disabled
    my $file = "/proc/sys/net/ipv6/conf/$iface/disable_ipv6";
    open(my $fh, '>', $file) or die "failed to open $file for writing: $!\n";
    print {$fh} "1\n" or die "failed to disable link-local ipv6 for $iface\n";
    close($fh);
    return;
}



you don't have same problem when using vmbr0 on the same host ?
 
Hi, both nodes have ipv6 enabled. The homelab server has ipv6, but no assigned address and the public facing one is fully using ivp6:
sysctl -p
Code: 
net.ipv6.conf.all.forwarding = 1
net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.default.disable_ipv6 = 0

The mentioned tap-interface, which is checked by the script, is not existing on both systems - so I have no idea, why the disable function fires in the first place.

When I use vmbr0 everything works fine.
 
IamLunchbox said:
The mentioned tap-interface, which is checked by the script, is not existing on both systems - so I have no idea, why the disable function fires in the first place.
Click to expand...
the tap interface is dynamicaly created when vm is starting, that's why you don't see it if vm start has failed. I'll try to reproduce on my side.
 
I am unable to boot a "normal" debian system using the VNet as well - I guess it is not cloud init after all but rather my VXLan...

Since I am working on a single node right now, the error and corresponding tap-interface changed to tap109:
Code: 
failed to open /proc/sys/net/ipv6/conf/tap109i0/disable_ipv6 for writing: No such file or directory
kvm: -netdev type=tap,id=net0,ifname=tap109i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on: network script /var/lib/qemu-server/pve-bridge failed with status 512
TASK ERROR: start failed: QEMU exited with code 1

ls -lah /proc/sys/net/ipv6/conf/
Code: 
dr-xr-xr-x 1 root root 0 Oct 14 01:47 .
dr-xr-xr-x 1 root root 0 Oct 14 01:47 ..
dr-xr-xr-x 1 root root 0 Oct 14 01:47 all
dr-xr-xr-x 1 root root 0 Oct 14 01:47 default
dr-xr-xr-x 1 root root 0 Oct 14 01:47 enp0s31f6
dr-xr-xr-x 1 root root 0 Oct 16 02:06 fwbr102i0
dr-xr-xr-x 1 root root 0 Oct 16 02:14 fwbr104i0
dr-xr-xr-x 1 root root 0 Oct 16 22:11 fwbr105i0
dr-xr-xr-x 1 root root 0 Oct 16 02:28 fwbr106i0
dr-xr-xr-x 1 root root 0 Oct 16 02:37 fwbr107i0
dr-xr-xr-x 1 root root 0 Oct 16 02:44 fwbr303i0
dr-xr-xr-x 1 root root 0 Oct 16 02:59 fwbr305i0
dr-xr-xr-x 1 root root 0 Oct 16 03:11 fwbr306i0
dr-xr-xr-x 1 root root 0 Oct 16 03:25 fwbr307i0
dr-xr-xr-x 1 root root 0 Oct 16 03:34 fwbr308i0
dr-xr-xr-x 1 root root 0 Oct 16 04:19 fwbr309i0
dr-xr-xr-x 1 root root 0 Oct 16 04:45 fwbr404i0
dr-xr-xr-x 1 root root 0 Oct 16 02:06 fwln102i0
dr-xr-xr-x 1 root root 0 Oct 16 02:14 fwln104i0
dr-xr-xr-x 1 root root 0 Oct 16 22:11 fwln105i0
dr-xr-xr-x 1 root root 0 Oct 16 02:28 fwln106i0
dr-xr-xr-x 1 root root 0 Oct 16 02:37 fwln107i0
dr-xr-xr-x 1 root root 0 Oct 16 02:44 fwln303i0
dr-xr-xr-x 1 root root 0 Oct 16 02:59 fwln305i0
dr-xr-xr-x 1 root root 0 Oct 16 03:11 fwln306i0
dr-xr-xr-x 1 root root 0 Oct 16 03:25 fwln307i0
dr-xr-xr-x 1 root root 0 Oct 16 03:34 fwln308i0
dr-xr-xr-x 1 root root 0 Oct 16 04:19 fwln309i0
dr-xr-xr-x 1 root root 0 Oct 16 04:45 fwln404i0
dr-xr-xr-x 1 root root 0 Oct 16 02:06 fwpr102p0
dr-xr-xr-x 1 root root 0 Oct 16 02:14 fwpr104p0
dr-xr-xr-x 1 root root 0 Oct 16 22:11 fwpr105p0
dr-xr-xr-x 1 root root 0 Oct 16 02:28 fwpr106p0
dr-xr-xr-x 1 root root 0 Oct 16 02:37 fwpr107p0
dr-xr-xr-x 1 root root 0 Oct 16 02:44 fwpr303p0
dr-xr-xr-x 1 root root 0 Oct 16 02:59 fwpr305p0
dr-xr-xr-x 1 root root 0 Oct 16 03:11 fwpr306p0
dr-xr-xr-x 1 root root 0 Oct 16 03:25 fwpr307p0
dr-xr-xr-x 1 root root 0 Oct 16 03:34 fwpr308p0
dr-xr-xr-x 1 root root 0 Oct 16 04:19 fwpr309p0
dr-xr-xr-x 1 root root 0 Oct 16 04:45 fwpr404p0
dr-xr-xr-x 1 root root 0 Oct 17 13:31 lo
dr-xr-xr-x 1 root root 0 Oct 16 02:00 tap100i0
dr-xr-xr-x 1 root root 0 Oct 16 02:00 tap100i1
dr-xr-xr-x 1 root root 0 Oct 16 02:00 tap100i2
dr-xr-xr-x 1 root root 0 Oct 16 02:00 tap100i3
dr-xr-xr-x 1 root root 0 Oct 16 02:00 tap100i4
dr-xr-xr-x 1 root root 0 Oct 16 02:06 tap102i0
dr-xr-xr-x 1 root root 0 Oct 16 02:14 tap104i0
dr-xr-xr-x 1 root root 0 Oct 16 22:11 tap105i0
dr-xr-xr-x 1 root root 0 Oct 16 02:28 tap106i0
dr-xr-xr-x 1 root root 0 Oct 16 02:37 tap107i0
dr-xr-xr-x 1 root root 0 Oct 16 02:44 tap303i0
dr-xr-xr-x 1 root root 0 Oct 16 02:59 tap305i0
dr-xr-xr-x 1 root root 0 Oct 16 03:11 tap306i0
dr-xr-xr-x 1 root root 0 Oct 16 03:25 tap307i0
dr-xr-xr-x 1 root root 0 Oct 16 03:34 tap308i0
dr-xr-xr-x 1 root root 0 Oct 16 04:19 tap309i0
dr-xr-xr-x 1 root root 0 Oct 16 04:45 tap404i0
dr-xr-xr-x 1 root root 0 Oct 14 01:47 vmbr0
dr-xr-xr-x 1 root root 0 Oct 14 01:47 vmbr1
dr-xr-xr-x 1 root root 0 Oct 17 13:31 vmbr2
dr-xr-xr-x 1 root root 0 Oct 14 01:47 vmbr3
dr-xr-xr-x 1 root root 0 Oct 14 01:47 vmbr4
dr-xr-xr-x 1 root root 0 Oct 17 13:31 wg0
 
Last edited:
that's really strange than /proc/sys/net/ipv6/conf/* don't have any /proc/sys/net/ipv6/conf/*/disable_ipv6

What is your running kernel version ?

As workaround, Could you try to:

edit /usr/share/perl5/PVE/Network.pm

and change
Code: 
sub disable_ipv6 {
    my ($iface) = @_;
    return if !-d '/proc/sys/net/ipv6'; # ipv6 might be completely disabled
    my $file = "/proc/sys/net/ipv6/conf/$iface/disable_ipv6";
    open(my $fh, '>', $file) or die "failed to open $file for writing: $!\n";
    print {$fh} "1\n" or die "failed to disable link-local ipv6 for $iface\n";
    close($fh);
    return;
}

by

Code: 
sub disable_ipv6 {
    my ($iface) = @_;
    return if !-d "/proc/sys/net/ipv6/conf/$iface/disable_ipv6"; # ipv6 might be completely disabled
    my $file = "/proc/sys/net/ipv6/conf/$iface/disable_ipv6";
    open(my $fh, '>', $file) or die "failed to open $file for writing: $!\n";
    print {$fh} "1\n" or die "failed to disable link-local ipv6 for $iface\n";
    close($fh);
    return;
}

then

#systemctl restart pvedaemon

and try to start vm again
 
The workaround seemed to fix the problem - starting a vm works now.

Regarding my kernel:
Linux virt 5.15.60-2-pve #1 SMP PVE 5.15.60-2 (Tue, 04 Oct 2022 16:52:28 +0200) x86_64 GNU/Linux
 
This workaround fixed the issue, I can use VXlan between both nodes now.

But my node cannot use apt update - could this be a MTU issue? I chose 1200...
 
IamLunchbox said:
This workaround fixed the issue, I can use VXlan between both nodes now.
Click to expand...
I'll send an official patch
IamLunchbox said:
But my node cannot use apt update - could this be a MTU issue? I chose 1200...
Click to expand...
possible. does it work with http repos instead https repos ? (https don't allow fragmented packets)

Do you have reduce mtu too inside your vm os ? (It's not done automatically)
 
I did create a new VM and everything worked now ... I found your MTU-Fix in another thread and changed my template, to preseed the correct MTU. Maybe that did the trick.

Just if anyone runs into this forum post: A MTU of 1200 worked in my setup using VXLan on top of a wireguard configuration, 1220 led to failed TLS-Handshakes.
 
Last edited:
Since I dont have a proxmox subscription - will the proposed fix of spirit find its way to PVE on itself? Or do I have to intervene somewhere else?
 
I just updated my instance yesterday and the perl script resets, crashing my machines. I will contact the mailing list.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom