Guest PXE on bridge device

[Manpreet]

I am running a following setup

Internet <-----[Gateway/VPN/Firewall/DHCP/PXE Host](192.168.1.1- <----------Proxmox Host(ip 192.168.1.30)

The network is bridged with the device being vmbr0.
I am trying to install KVM vms on the host but wheather i I try to use DHCP for PXE or boot using ISO, the host is not able to obtain DHCP ip.

Running TCPdump on the tap device on proxmox host corresponding to the vm, I see dhcp server sending DHCP offer but it never reaches the client

 

[Alexander Pytlev]

Another computer in your network boot on PXE?
Firewall on Proxmox Host on/off? configured?

Show from Proxmox Host:
/etc/network/interfaces
/etc/pve/qemu-server/xxx.conf - config your VM

 

[Manpreet]

Firewall on proxmox is off. yes other systems are booting on PXE.

/etc/network/interfaces


iface eth0 inet manual

auto vmbr0
iface vmbr0 inet static
address 10.168.12.239
netmask 255.255.255.0
gateway 10.168.12.1
broadcast 10.168.12.255
bridge_ports eth0
bridge_stp off
bridge_fd 0
network 10.168.12.0
bridge_maxwait 0


/etc/pve/qemu-server/100.conf

boot: dcn
bootdisk: sata0
cores: 2
ide2: local:iso/debian-9.0.0-amd64-netinst.iso,media=cdrom,size=290M
memory: 512
name: vmtest
net0: e1000=12:0b:b3:8a:32:6a,bridge=vmbr0
numa: 0
ostype: l26
sata0: VMs:vm-100-disk-1,size=10G
scsihw: virtio-scsi-pci
smbios1: uuid=f378588b-f2ce-4b5e-a760-c35d73a965ff
sockets: 1



Currently its booting from iso. But gets stuck at network config since its not able to obtain ip from dhcp

 

[Alexander Pytlev]

Try to use another type network card in VM (not e1000)

 

[Manpreet]

I have tried all 3 options e1000, virtio device, realtek

 

[Manpreet]

Jun 23 09:13:14 dhcp-server dhcpd[30749]: DHCPDISCOVER from 12:0b:b3:8a:32:6a via ens8
Jun 23 09:13:14 dhcp-server dhcpd[30749]: DHCPOFFER on 10.168.12.11 to 12:0b:b3:8a:32:6a via ens8
Jun 23 09:13:18 dhcp-server dhcpd[30749]: DHCPDISCOVER from 12:0b:b3:8a:32:6a via ens8
Jun 23 09:13:18 dhcp-server dhcpd[30749]: DHCPOFFER on 10.168.12.11 to 12:0b:b3:8a:32:6a via ens8

 

[Alexander Pytlev]

Install dhcpdump on Proxmox Host and analyze, what happened

 

[Manpreet]

On the DHCP Server side

---------------------------------------------------------------------------

TIME: 2017-06-26 02:19:48.559
IP: 0.0.0.0 (12:b:b3:8a:32:6a) > 255.255.255.255 (ff:ff:ff:ff:ff:ff)
OP: 1 (BOOTPREQUEST)
HTYPE: 1 (Ethernet)
HLEN: 6
HOPS: 0
XID: 16225672
SECS: 16
FLAGS: 0
CIADDR: 0.0.0.0
YIADDR: 0.0.0.0
SIADDR: 0.0.0.0
GIADDR: 0.0.0.0
CHADDR: 12:0b:b3:8a:32:6a:00:00:00:00:00:00:00:00:00:00
SNAME: .
FNAME: .
OPTION: 53 ( 1) DHCP message type 1 (DHCPDISCOVER)
OPTION: 57 ( 2) Maximum DHCP message size 1472
OPTION: 93 ( 2) Client System 0000 ..
OPTION: 94 ( 3) Client NDI 010201 ...
OPTION: 60 ( 32) Vendor class identifier PXEClient:Arch:00000:UNDI:002001
OPTION: 77 ( 4) User-class Identification 69505845 iPXE
OPTION: 55 ( 22) Parameter Request List 1 (Subnet mask)
3 (Routers)
6 (DNS server)
7 (Log server)
12 (Host name)
15 (Domainname)
17 (Root path)
43 (Vendor specific info)
60 (Vendor class identifier)
66 (TFTP server name)
67 (Bootfile name)
119 (Domain Search)
128 (???)
129 (???)
130 (???)
131 (???)
132 (???)
133 (???)
134 (???)
135 (???)
175 (???)
203 (???)

OPTION: 175 ( 45) ??? b1050110ec8139eb ......9.
0301000017010122 ......."
0101130101110101 ........
2701011901011001 '.......
0221010115010118 .!......
0101120101 .....
OPTION: 61 ( 7) Client-identifier 01:12:0b:b3:8a:32:6a
OPTION: 97 ( 17) UUID/GUID 008b5878f3cef25e ..Xx...^
4ba760c35d73a965 K.`.]s.e
ff .
---------------------------------------------------------------------------

TIME: 2017-06-26 02:19:48.559
IP: 10.168.12.1 (52:54:0:70:1c:bb) > 10.168.12.11 (12:b:b3:8a:32:6a)
OP: 2 (BOOTPREPLY)
HTYPE: 1 (Ethernet)
HLEN: 6
HOPS: 0
XID: 16225672
SECS: 16
FLAGS: 0
CIADDR: 0.0.0.0
YIADDR: 10.168.12.11
SIADDR: 10.168.12.1
GIADDR: 0.0.0.0
CHADDR: 12:0b:b3:8a:32:6a:00:00:00:00:00:00:00:00:00:00
SNAME: .
FNAME: fai/pxelinux.0.
OPTION: 53 ( 1) DHCP message type 2 (DHCPOFFER)
OPTION: 54 ( 4) Server identifier 10.168.12.1
OPTION: 51 ( 4) IP address leasetime 600 (10m)
OPTION: 1 ( 4) Subnet mask 255.255.255.0
OPTION: 3 ( 4) Routers 10.168.12.1
OPTION: 6 ( 12) DNS server 10.168.12.1,10.71.175.101,8.8.8.8
OPTION: 15 ( 16) Domainname webintensive.com
---------------------------------------------------------------------------




On the proxmox base host checking the the "tap" device only the request shows up. even on the bridge device only dhcp client packets are visible

 

[Alexander Pytlev]

What switch/router you use between pxe host and proxmox host?

 

[Manpreet]

both pxe host and proxmox host are themselves VMs running on KVM using host passthrough and macvtap driver. the host itself is connected to a bonded device which in turn is connected to switch

 

[Manpreet]

guest vm
------------------
dhcp-server proxmox host
--------------------------------------
Base Host


I think the proxmox host is not able to see the mac of the guest over bridge

 

[Alexander Pytlev]

I think, that is not proxmox problem, dig your system, where you running KVM.

 

[Manpreet]

Here is something intresting

on the dhcp server
#arp
10.168.12.11 (incomplete) ens8

on proxmox host

04:23:43 [root@nebula ~]# arp
Address HWtype HWaddress Flags Mask Iface
10.168.12.1 ether 52:54:00:70:1c:bb C vmbr0
10.168.12.238 ether 52:54:00:c4:4f:21 C vmbr0

so proxmox host can't see the the MAC of the guest which is sending the dhcpdiscover and hence it doesnt accept packets for that mac