PXE booting VMs not working.

[LnxBil]

For a few weeks I run in this problem too. Unfortunately, I cannot reproduce it on every day. There are days where it works out-of-the-box and there are days like today. Using the iso-image of ipxe, it works.

I debugged the tap-network device with tcpdump while booting and find out that
- stock ipxe send one dhcp request and got one answer, yet no ip and no booting
- cd-based ipxe send multiple (identical) dhcp requests and got multiple (identitical) answers back. Then it boots fine.

I was able to replace ipxe in Proxmox itself and will report it upstream:
- Clone http://git.ipxe.org/ipxe.git
- build virt-net.rom
make bin/virtio-net.rom
- copy to proxmox:
scp bin/virtio-net.rom proxmox:/usr/share/kvm/pxe-virtio.rom

Of course this does not survive a package update.

 

[jht3]

i too am having no success PXE booting a new KVM vm on 4.0. i can boot and install a bare metal machine perfectly from my LXC centos vm w/ dnsmasq so i know the configuration is correct and my networking is proper. i can always ping the pxe client from the dnsmasq server.

i've tried booting off the latest ipxe .iso but the end result is the same as when using the native/embedded version; the vm gets an IP address via dhcp, but "times out" when talking to the pxe server. i can see three or four attempts in the dnsmasq logs after the IP is leased out. i have also tried both E1000 and virtio nic settings.

EDIT: added some logs

example of a non working KVM vm:

Nov 20 08:56:42 ks dnsmasq-dhcp[926]: DHCPDISCOVER(eth0) 36:36:39:36:30:66 
Nov 20 08:56:42 ks dnsmasq-dhcp[926]: DHCPOFFER(eth0) 192.168.11.117 36:36:39:36:30:66 
Nov 20 08:56:42 ks dnsmasq-dhcp[926]: DHCPDISCOVER(eth0) 36:36:39:36:30:66 
Nov 20 08:56:42 ks dnsmasq-dhcp[926]: DHCPOFFER(eth0) 192.168.11.117 36:36:39:36:30:66 
Nov 20 08:56:42 ks dnsmasq-dhcp[926]: DHCPREQUEST(eth0) 192.168.11.117 36:36:39:36:30:66 
Nov 20 08:56:42 ks dnsmasq-dhcp[926]: DHCPACK(eth0) 192.168.11.117 36:36:39:36:30:66 
Nov 20 08:57:05 ks dnsmasq-dhcp[926]: PXE(eth0) 192.168.11.117 36:36:39:36:30:66 pxelinux.0
Nov 20 08:57:06 ks dnsmasq-dhcp[926]: PXE(eth0) 192.168.11.117 36:36:39:36:30:66 pxelinux.0
Nov 20 08:57:08 ks dnsmasq-dhcp[926]: PXE(eth0) 192.168.11.117 36:36:39:36:30:66 pxelinux.0

example of a working bare-metal laptop

Nov 20 09:09:01 ks dnsmasq-dhcp[926]: DHCPDISCOVER(eth0) 14:fe:b5:bf:d6:f2 
Nov 20 09:09:01 ks dnsmasq-dhcp[926]: DHCPOFFER(eth0) 192.168.11.106 14:fe:b5:bf:d6:f2 
Nov 20 09:09:01 ks dnsmasq-dhcp[926]: DHCPREQUEST(eth0) 192.168.11.106 14:fe:b5:bf:d6:f2 
Nov 20 09:09:01 ks dnsmasq-dhcp[926]: DHCPACK(eth0) 192.168.11.106 14:fe:b5:bf:d6:f2 
Nov 20 09:09:15 ks dnsmasq-dhcp[926]: PXE(eth0) 192.168.11.106 14:fe:b5:bf:d6:f2 pxelinux.0
Nov 20 09:09:16 ks dnsmasq-tftp[926]: TFTP sent /var/lib/tftpboot/pxelinux.0 to 192.168.11.106
Nov 20 09:09:16 ks dnsmasq-tftp[926]: TFTP error 0 TFTP Aborted received from 192.168.11.106
Nov 20 09:09:16 ks dnsmasq-tftp[926]: TFTP failed sending /var/lib/tftpboot/pxelinux.0 to 192.168.11.106
Nov 20 09:09:16 ks dnsmasq-tftp[926]: TFTP sent /var/lib/tftpboot/pxelinux.0 to 192.168.11.106
Nov 20 09:09:16 ks dnsmasq-tftp[926]: TFTP sent /var/lib/tftpboot/pxelinux.cfg/default to 192.168.11.106
Nov 20 09:09:16 ks dnsmasq-tftp[926]: TFTP sent /var/lib/tftpboot/vesamenu.c32 to 192.168.11.106
Nov 20 09:09:17 ks dnsmasq-tftp[926]: TFTP sent /var/lib/tftpboot/pxelinux.cfg/default to 192.168.11.106

# pveversion -v
proxmox-ve: 4.0-22 (running kernel: 4.2.3-2-pve)
pve-manager: 4.0-57 (running version: 4.0-57/cc7c2b53)
pve-kernel-4.2.3-2-pve: 4.2.3-22
lvm2: 2.02.116-pve1
corosync-pve: 2.3.5-1
libqb0: 0.17.2-1
pve-cluster: 4.0-24
qemu-server: 4.0-35
pve-firmware: 1.1-7
libpve-common-perl: 4.0-36
libpve-access-control: 4.0-9
libpve-storage-perl: 4.0-29
pve-libspice-server1: 0.12.5-2
vncterm: 1.2-1
pve-qemu-kvm: 2.4-12
pve-container: 1.0-21
pve-firewall: 2.0-13
pve-ha-manager: 1.0-13
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u1
lxc-pve: 1.1.4-3
lxcfs: 0.10-pve2
cgmanager: 0.39-pve1
criu: 1.6.0-1
zfsutils: 0.6.5-pve6~jessie

EDIT2:
ok, got it working in a vm by downloading the gPXE bootable .iso and launching it first. this is annoying workaround but at least it works. not sure why the newer iPXE .iso isn't working for me, or the native iPXE boot rom.

Nov 20 09:20:22 ks dnsmasq-dhcp[926]: DHCPDISCOVER(eth0) 36:36:39:36:30:66 
Nov 20 09:20:22 ks dnsmasq-dhcp[926]: DHCPOFFER(eth0) 192.168.11.117 36:36:39:36:30:66 
Nov 20 09:20:22 ks dnsmasq-dhcp[926]: DHCPREQUEST(eth0) 192.168.11.117 36:36:39:36:30:66 
Nov 20 09:20:22 ks dnsmasq-dhcp[926]: DHCPACK(eth0) 192.168.11.117 36:36:39:36:30:66 
Nov 20 09:20:38 ks dnsmasq-dhcp[926]: PXE(eth0) 192.168.11.117 36:36:39:36:30:66 pxelinux.0
Nov 20 09:20:38 ks dnsmasq-tftp[926]: TFTP sent /var/lib/tftpboot/pxelinux.0 to 192.168.11.117
Nov 20 09:20:38 ks dnsmasq-tftp[926]: TFTP sent /var/lib/tftpboot/pxelinux.cfg/default to 192.168.11.117
Nov 20 09:20:38 ks dnsmasq-tftp[926]: TFTP sent /var/lib/tftpboot/vesamenu.c32 to 192.168.11.117
Nov 20 09:20:38 ks dnsmasq-tftp[926]: TFTP sent /var/lib/tftpboot/pxelinux.cfg/default to 192.168.11.117

EDIT3:
under further testing, i replaced dnsmasq with dhcpd and xinetd tftp. the stock iPXE rom works with this setup. wasn't expecting that to be an issue, but it is.

EDIT4:
ok, you may be getting tired of me now, but i did get dnsmasq to function as both my DHCP and TFTP server. i'm doing all this on a VLAN with only DNS provided by the default gateway. the tutorials on the internet are all over the place but this config is working. the pxe menu system i couldn't get working so i pulled all references to "pxe-service" from my /etc/dnsmasq.conf. now it boots straight to pxelinux.0 like the standalone xinetd tftp service was doing.

interface=eth0
# Disable DNS
port=0
# DHCP range-leases
dhcp-range= 192.168.11.100,192.168.11.119,255.255.255.0,1h
# Gateway
dhcp-option=3,192.168.11.1
# DNS
dhcp-option=6,192.168.11.1
# Broadcast Address
dhcp-option=28,192.168.11.255
# PXE
dhcp-boot=pxelinux.0
enable-tftp
tftp-root=/var/lib/tftpboot