DHCPv6 Issue After Proxmox Update: Seeking Guidance on iPXE, Virtio, QEMU, or Proxmox?

[Tharyrok]

Hello,
I'm not sure where to turn for help. I updated Proxmox to version 8.1 and QEMU to 8.1.2. I have an ISO with iPXE that does DHCPv6 and then fetches its configuration from a web server. Since the update, I get this message during DHCPv6:

No such file or directory (https://ipxe.org/2d0c203b)

I added more debugging and got this:

iPXE>  ifconf --configurator ipv6
NETDEV net0 opening
NETDEV net0 starting configuration via ipv6
Configuring [ipv6] (net0 66:b5:3e:97:7d:4e)...
NETDEV net0 failed to receive 0x0: Error 0x440e6003 (https://ipxe.org/440e6003)
NETDEV net0 failed to receive 0x0: Error 0x440e6003 (https://ipxe.org/440e6003)
NETDEV net0 failed to receive 0x0: Error 0x440e6003 (https://ipxe.org/440e6003)
DHCPv6 net0 could not create DUID-UUID: No such file or directory (https://ipxe.org/2d0c203b)
NETDEV net0 configuration via ipv6 failed: No such file or directory (https://ipxe.org/2d0c203b)
NETDEV net0 failed to receive 0x0: No such file or directory (https://ipxe.org/2d0c203b)
No such file or directory (https://ipxe.org/2d0c203b)
NETDEV net0 closing

I can't figure out where to look, whether it's an issue with iPXE, Virtio, QEMU, or Proxmox.

Do you have any leads?

 

[Tharyrok]

From my investigations, I found that in QEMU 8.1, the default BIOS version switches to SMBIOS 3.0, which is in 64 bits (https://github.com/qemu/qemu/commit/bf376f3020dfd7bcb2c4158b4ffa85c04d44f56d). After reconfiguring the VM to use SMBIOS 2.8, DHCPv6 works. Here's the trace:

iPXE> ifconf --configurator ipv6
Configuring [ipv6] (net0 bc:24:11:ed:9c:33)...
NEIGHBOUR net0 IPv6 fe80::48b9:78ff:febb:5bf6 created
NEIGHBOUR net0 IPv6 fe80::48b9:78ff:febb:5bf6 is Ethernet 4a:b9:78:bb:5b:f6
DHCPv6 net0 has XID dad6b8
DHCPv6 net0 entering SOLICIT state
DHCPv6 net0 received ADVERTISE from fe80::48b9:78ff:febb:5bf6
DHCPv6 net0 received ADVERTISE is for fd5f:7d49:c3b0:40::ff:929
DHCPv6 net0 entering REQUEST state
DHCPv6 net0 received REPLY from fe80::48b9:78ff:febb:5bf6
DHCPv6 net0 received REPLY is for fd5f:7d49:c3b0:40::ff:929
DHCPv6 net0 complete
ok

To answer some questions, PXE boot worked well with QEMU 8.0.2. In terms of the network, I'm using Linux bridges with VLANs, and on the VM side, it's untagged. I have pfSense 2.7.2 for DHCPv6 and SLAAC.

So, I suspect a problem with SMBIOS 3.0 and the creation of DUID-UUID for DHCPv6.Do you have any ideas on how to investigate further to fix this bug?

 

[Tharyrok]

I've just finished creating the tickets on the QEMU side (https://gitlab.com/qemu-project/qemu/-/issues/2024) and on the iPXE side (https://github.com/ipxe/ipxe/issues/1094).

 

[blackline]

EDIT: I can confirm that by using the machine: pc-i440fx-8.0 option in Proxmox, the problem is solved. Thanks for finding it!

Not sure if what I got is related, but when using seaBIOS, it looks like iPXE is not able to read the SMBIOS UUID anymore. With EFI it works:

(pardon the debug output, it took me quite some time to find that one out)

# working with UEFI, see HTTP GET /ipxe?uuid=VALUE
2024/01/22 07:06:37 HTTP GET /ipxe?uuid=b708b5ae-e35d-459c-909d-8198c228cdd9&mac=bc-24-11-39-8f-3d&domain=local.berchtold.live&hostname=&serial=&arch=x86_64 192.168.50.84:22325
2024/01/22 07:06:37 FLO: got key: hostname with value:
2024/01/22 07:06:37 FLO: got key: serial with value:
2024/01/22 07:06:37 FLO: got key: arch with value: x86_64
2024/01/22 07:06:37 FLO: got key: uuid with value: b708b5ae-e35d-459c-909d-8198c228cdd9
2024/01/22 07:06:37 FLO: got mac: bc-24-11-39-8f-3d
2024/01/22 07:06:37 FLO: got key: domain with value: local.berchtold.live
2024/01/22 07:06:37 FLO: got labels: map[arch:x86_64 domain:local.berchtold.live hostname: mac:bc:24:11:39:8f:3d serial: uuid:b708b5ae-e35d-459c-909d-8198c228cdd9]
2024/01/22 07:06:37 FLO: looking up server:
2024/01/22 07:06:37 FLO: server: &{{Server metal.sidero.dev/v1alpha2} {b708b5ae-e35d-459c-909d-8198c228cdd9    480f4688-f7dd-464d-bbba-d9fbfc012371 16100688 1 2024-01-22 07:03:27 +0000 UTC <nil> <nil> map] map] ] [storage.finalizers.server.k8s.io] [{manager Update metal.sidero.dev/v1alpha2 2024-01-22 07:03:28 +0000 UTC FieldsV1 {"f:metadata":{"f:finalizers":{".":{},"v:\"storage.finalizers.server.k8s.io\"":{}}},"f:spec":{".":{},"f:accepted":{},"f:hardware":{".":{},"f:compute":{".":{},"f:processorCount":{},"f:processors":{},"f:totalCoreCount":{},"f:totalThreadCount":{}},"f:memory":{".":{},"f:moduleCount":{},"f:modules":{},"f:totalSize":{}},"f:network":{".":{},"f:interfaceCount":{},"f:interfaces":{}},"f:storage":{".":{},"f:deviceCount":{},"f:devices":{},"f:totalSize":{}},"f:system":{".":{},"f:manufacturer":{},"f:productName":{},"f:uuid":{}}},"f:hostname":{}}} } {manager Update metal.sidero.dev/v1alpha2 2024-01-22 07:03:28 +0000 UTC FieldsV1 {"f:status":{".":{},"f:addresses":{},"f:conditions":{},"f:isClean":{},"f:power":{},"f:ready":{}}} status}]} {nil 0xc000a2ae40 (none) <nil> <nil> ] true false false  } {true false true [{PowerCycle True  2024-01-22 07:03:28 +0000 UTC  }] [{InternalIP 192.168.50.85}] on}}
2024/01/22 07:06:37 FLO: server: <nil>
2024/01/22 07:06:37 FLO: server: context.Background.WithValue(type *http.contextKey, val <not Stringer>).WithValue(type *http.contextKey, val 192.168.50.50:8081).WithCancel.WithCancel
2024/01/22 07:06:37 FLO: server: b708b5ae-e35d-459c-909d-8198c228cdd9
2024/01/22 07:06:37 FLO: using agent environment, because serverBinding is nil
2024/01/22 07:06:37 Using "agent-amd64" environment for "b708b5ae-e35d-459c-909d-8198c228cdd9"
2024/01/22 07:06:37 HTTP GET /env/agent-amd64/vmlinuz 192.168.50.84:22325
2024/01/22 07:06:37 HTTP GET /env/agent-amd64/initramfs.xz 192.168.50.84:22325
2024/01/22 07:06:45 HTTP GET /boot.ipxe 127.0.0.1:43788
2024/01/22 07:06:45 HTTP GET /boot.ipxe 127.0.0.1:43778
2024/01/22 07:06:55 HTTP GET /boot.ipxe 127.0.0.1:43778
2024/01/22 07:06:55 HTTP GET /boot.ipxe 127.0.0.1:43788
2024/01/22 07:07:05 HTTP GET /boot.ipxe 127.0.0.1:43778
2024/01/22 07:07:05 HTTP GET /boot.ipxe 127.0.0.1:43788
2024/01/22 07:07:08 86360 bytes sent

now switching BIOS to default(seaBIOS):

# not working with BIOS, see HTTP GET /ipxe?uuid=MISSING_VALUE
2024/01/22 07:07:11 HTTP GET /ipxe?uuid=&mac=bc-24-11-39-8f-3d&domain=local.berchtold.live&hostname=&serial=&arch=i386 192.168.50.84:25397
2024/01/22 07:07:11 FLO: got key: uuid with value:
2024/01/22 07:07:11 FLO: got mac: bc-24-11-39-8f-3d
2024/01/22 07:07:11 FLO: got key: domain with value: local.berchtold.live
2024/01/22 07:07:11 FLO: got key: hostname with value:
2024/01/22 07:07:11 FLO: got key: serial with value:
2024/01/22 07:07:11 FLO: got key: arch with value: i386
2024/01/22 07:07:11 FLO: got labels: map[arch:i386 domain:local.berchtold.live hostname: mac:bc:24:11:39:8f:3d serial: uuid:]
2024/01/22 07:07:11 FLO: looking up server:
2024/01/22 07:07:11 FLO: server: <nil>
2024/01/22 07:07:11 FLO: server: <nil>
2024/01/22 07:07:11 FLO: server: context.Background.WithValue(type *http.contextKey, val <not Stringer>).WithValue(type *http.contextKey, val 192.168.50.50:8081).WithCancel.WithCancel
2024/01/22 07:07:11 FLO: server:
2024/01/22 07:07:11 FLO: using agent environment, because server is nil
2024/01/22 07:07:11 Using "agent-amd64" environment
2024/01/22 07:07:11 HTTP GET /env/agent-amd64/vmlinuz 192.168.50.84:25397
2024/01/22 07:07:11 HTTP GET /env/agent-amd64/initramfs.xz 192.168.50.84:25397
2024/01/22 07:07:15 HTTP GET /boot.ipxe 127.0.0.1:43788
2024/01/22 07:07:15 HTTP GET /boot.ipxe 127.0.0.1:43778
2024/01/22 07:07:24 FLO: CreateServer uuid b708b5ae-e35d-459c-909d-8198c228cdd9
2024/01/22 07:07:24 Server "b708b5ae-e35d-459c-909d-8198c228cdd9" needs BMC setup
2024/01/22 07:07:25 HTTP GET /boot.ipxe 127.0.0.1:43788
2024/01/22 07:07:25 HTTP GET /boot.ipxe 127.0.0.1:43778
2024/01/22 07:07:35 HTTP GET /boot.ipxe 127.0.0.1:43778
2024/01/22 07:07:35 HTTP GET /boot.ipxe 127.0.0.1:43788
2024/01/22 07:07:41 86360 bytes sent