Debian 9.4 not starting on nested ESXi

[Max P]

Hi,

We have a 4 node proxmox cluster (5.3) where we have a nested ESXi (mostly for migrating old vmware VMs to ceph).
On this nested ESXi a Windows 10 VM (uefi) starts and runs fine.
A Debian 9.3 VM runs and starts fine too.
But a Debian 9.4 VM does not start (when booting a 9.4 netinstall you get to the boot menu, but once you start the installer it stops).
( PVE (5.3) -> ESXi (6.5.0 build 10884925) -> Debian )

There are no errors on PVE (that I can find) and the only logs from the ESXi are:
.....
| vcpu-0| I125: pciBridge7:7: ISA/VGA decoding enabled (ctrl 0004)
| vcpu-0| I125: DISKUTIL: scsi0:0 : geometry=2610/255/63
| vcpu-0| I125: DISKUTIL: scsi0:0 : capacity=41943040 logical sector size=512
| vcpu-1| I125: CPU reset: soft (mode 2)
| vcpu-0| I125: BIOS-UUID is 42 3f ba 49 8a 34 f8 fc-d7 ef 43 96 94 ed 39 6e
| mks| I125: VNCENCODE 3 VNCEncodeChooseRegionEncoder: region encoder adaptive. Resolution: 720 x 400
| vcpu-0| I125: Unknown int 10h func 0x0000
| vcpu-0| I125: Unknown int 10h func 0x0000
| svga| I125: SVGA enabling SVGA
| svga| I125: SVGA-ScreenMgr: Screen type changed to RegisterMode
| mks| I125: VNCENCODE 3 VNCEncodeChooseRegionEncoder: region encoder adaptive. Resolution: 640 x 480
| vcpu-0| I125: HBACommon: First write on scsi0:0.fileName='/vmfs/volumes/5941226a-a3f46ec1/TRANSPORTER/TRANSPORTER.vmdk'
| vcpu-0| I125: DDB: "longContentID" = "734b1ed0dfce89ea050ad93c52418a0b" (was "1c838dd8a817389f24109d5d8c345423")
| vcpu-0| I125: DISKLIB-CHAIN : DiskChainUpdateContentID: old=0x8c345423, new=0x52418a0b (734b1ed0dfce89ea050ad93c52418a0b)
| svga| I125: SVGA disabling SVGA
| mks| I125: VNCENCODE 3 VNCEncodeChooseRegionEncoder: region encoder adaptive. Resolution: 720 x 400
| vcpu-0| I125: Guest MSR write (0xc0011029: 0x2)
| vcpu-0| W115: MONITOR PANIC: vcpu-0:VMM fault 13: src=MONITOR rip=0xfffffffffc46b974 regs=0xfffffffffc607d90
| vcpu-0| I125: Core dump with build build-10390116
| vcpu-1| I125: Exiting vcpu-1
| vcpu-0| I125: Writing monitor file `vmmcores.gz`
| vcpu-0| W115: Dumping core for vcpu-0
| vcpu-0| I125: CoreDump: dumping core with superuser privileges
| vcpu-0| I125: VMK Stack for vcpu 0 is at 0x439124293000
| vcpu-0| I125: Beginning monitor coredump
| vcpu-0| I125: End monitor coredump
| vcpu-0| W115: Dumping core for vcpu-1
| vcpu-0| I125: CoreDump: dumping core with superuser privileges
| vcpu-0| I125: VMK Stack for vcpu 1 is at 0x439124413000
| vcpu-0| I125: Beginning monitor coredump
| mks| W115: Panic in progress... ungrabbing
| mks| I125: MKS: Release starting (Panic)
| mks| I125: MKS: Release finished (Panic)
| vcpu-0| I125: End monitor coredump
| vcpu-0| W115: A core file is available in "/vmfs/volumes/5941226a-a3f46ec1/TRANSPORTER/vmx-zdump.001"
| vcpu-0| I125: Msg_Post: Error
| vcpu-0| I125: [msg.log.error.unrecoverable] VMware ESX unrecoverable error: (vcpu-0)
| vcpu-0| I125+ vcpu-0:VMM fault 13: src=MONITOR rip=0xfffffffffc46b974 regs=0xfffffffffc607d90
| vcpu-0| I125: [msg.panic.haveLog] A log file is available in "/vmfs/volumes/5941226a-a3f46ec1/TRANSPORTER/vmware.log".
| vcpu-0| I125: [msg.panic.requestSupport.withoutLog] You can request support.
| mks| W115: Panic in progress... ungrabbing
| mks| I125: MKS: Release starting (Panic)
| mks| I125: MKS: Release finished (Panic)
| vcpu-0| I125: [msg.panic.requestSupport.vmSupport.vmx86]
| vcpu-0| I125+ To collect data to submit to VMware technical support, run "vm-support".
| vcpu-0| I125: [msg.panic.response] We will respond on the basis of your support entitlement.
| vcpu-0| I125: ----------------------------------------

The PVE host has an AMD EPYC 7351P and the qemu machine is the default "pc-i440fx-2.11" .
On a physical ESXi 6.5.0 host Debian 9.4 boots and runs without a problem.

Any ideas what the problem could be?

regards
max

 

[Stoiko Ivanov]

Please post the vm-config of the ESXi (`qm config <VMID>`)

pc-i440fx-2.11" .

the default should be pc-i44fx without the version number.

There were a few threads recently with troubles with nested virtualization and EPYC cpus:
https://bugzilla.proxmox.com/show_bug.cgi?id=2008
https://forum.proxmox.com/threads/nested-virtualization-not-working-anymore.49665/

maybe the hints there could help.

 

[Max P]

https://forum.proxmox.com/threads/nested-virtualization-not-working-anymore.49665/

This is were I got the idea from to set the machine type specifically. So I ran this command:

qm set 109 -machine pc-i440fx-2.11

I have now removed the config entry specifying the machine type again and stopped and started the ESXi VM again but the behaviour hasn't changed.
Here is the complete vm config:

bootdisk: ide0
cores: 8
cpu: host
ide0: rbd_hdd_vm:vm-109-disk-1,size=4G
ide2: none,media=cdrom
keyboard: de
memory: 16384
name: ESXi
net0: vmxnet3=6A:B0:57:57:C4:92,bridge=vmbr1,tag=4
numa: 0
ostype: other
sata1: rbd_hdd_vm:vm-109-disk-0,size=100G
scsihw: virtio-scsi-pci
smbios1: uuid=708e9fce-656c-4244-8cd2-fa8fa7d91e54
sockets: 1
vga: qxl

On the console of the ESXi it says:
"QEMU Standard PC (i440FX + PIIX, 1996)"