Windows VMs stuck on boot after Proxmox Upgrade to 7.0

Just adding to the voices that I have observed this behavior as well.
Code: 
proxmox-ve: 7.1-1 (running kernel: 5.13.19-5-pve)
pve-manager: 7.1-10 (running version: 7.1-10/6ddebafe)
pve-kernel-helper: 7.1-12
pve-kernel-5.13: 7.1-8
pve-kernel-5.11: 7.0-10
pve-kernel-5.3: 6.1-6
pve-kernel-5.13.19-5-pve: 5.13.19-13
pve-kernel-5.13.19-3-pve: 5.13.19-7
pve-kernel-5.13.19-2-pve: 5.13.19-4
pve-kernel-5.11.22-7-pve: 5.11.22-12
pve-kernel-5.3.18-3-pve: 5.3.18-3
pve-kernel-5.3.10-1-pve: 5.3.10-1
ceph: 16.2.7
ceph-fuse: 16.2.7
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: not correctly installed
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.1
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-6
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.1-2
libpve-guest-common-perl: 4.0-3
libpve-http-server-perl: 4.1-1
libpve-storage-perl: 7.0-15
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.11-1
lxcfs: 4.0.11-pve1
novnc-pve: 1.3.0-1
proxmox-backup-client: 2.1.5-1
proxmox-backup-file-restore: 2.1.5-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.4-5
pve-cluster: 7.1-3
pve-container: 4.1-3
pve-docs: 7.1-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.3-5
pve-ha-manager: 3.3-3
pve-i18n: 2.6-2
pve-qemu-kvm: 6.1.1-1
pve-xtermjs: 4.16.0-1
qemu-server: 7.1-4
smartmontools: 7.2-pve2
spiceterm: 3.2-2
swtpm: 0.7.0~rc1+2
vncterm: 1.7-1
zfsutils-linux: 2.1.2-pve1
 
  • Like
Reactions: weehooey-bh
Also happens rare on Ubuntu 20.04 LTS Guest when planed reboot is scheduled.
 
I can confirm. The problem does NOT show up on 2008r2. 8 VMs (with extended update service) did not have any problems rebooting after a month of uptime. And never in the past have they given it. The problem occurs from 2012 onwards. Who knows, some low-level interaction that doesn't show up with 2008r2?
 
  • Like
Reactions: weehooey-bh
Any tests with version 7.1-11? The list of updated packages does not include QEMU and/or the kernel, so I assume the problem remains. I'm terrified of finding dozens and dozens of VMs crashed on boot at the next reboot for Windows updates. Sigh
 
Would it help, if I have a VM that is currently stuck rebooting? The VM is a Windows Server 2019 STD and is currently not in use so I can just leave it hanging for now.
 
Hi,

vulture said:
Would it help, if I have a VM that is currently stuck rebooting? The VM is a Windows Server 2019 STD and is currently not in use so I can just leave it hanging for now.
Click to expand...
Can you reproduce the issue on the same VM? if so please do the following:
1- Since at the moment you cannot predict whether the issue will happen after the next reboot, please take a snapshot if you feel this reboot the VM will hang at boot.
2- If the issue occurs please rollback the snapshot and make a backup for the VM and provide the backup file

This will help us to more digging about the issue.
 
  • Like
Reactions: weehooey-bh
I have a VM snapshot including RAM that seems to reliably show the problem.

Does the Backup contain RAM? Because when just using the exact snapshot without RAM (so the VM does a "normal" startup), the problem cannot be replicated anymore...
 
Emilien said:
Can you post a vm config?
Click to expand...
Here are 2 VM confs. Both were imports from Hyper-V. Both have sporadic issues with reboot fail:
VM 1 -
Code: 
agent: 1
balloon: 0
bios: ovmf
boot: order=scsi0;sata2
cores: 8
cpu: host
efidisk0: NVMe:vm-106-disk-1,size=128K
machine: pc-i440fx-6.0
memory: 16384
name: VM-WEB05
net0: virtio=9A:0D:EA:18:83:09,bridge=vmbr0,tag=12
numa: 0
ostype: win10
parent: offline_test
sata2: none,media=cdrom
scsi0: NVMe:vm-106-disk-0,size=170G
scsihw: virtio-scsi-pci
smbios1: uuid=3e6b9f0c-1eeb-4135-9e81-665abcc63436
sockets: 1
vmgenid: 5d66ac2c-1ac1-4e4e-a9b8-180c5fefba57
[offline_test]
agent: 1
balloon: 0
bios: ovmf
boot: order=scsi0;sata2
cores: 8
efidisk0: NVMe:vm-106-disk-1,size=128K
machine: pc-i440fx-6.0
memory: 16384
name: VM-WEB05
net0: virtio=9A:0D:EA:18:83:09,bridge=vmbr0,tag=12
numa: 0
ostype: win10
sata2: none,media=cdrom
scsi0: NVMe:vm-106-disk-0,size=170G
scsihw: virtio-scsi-pci
smbios1: uuid=3e6b9f0c-1eeb-4135-9e81-665abcc63436
snaptime: 1646059538
sockets: 1
vmgenid: 5d66ac2c-1ac1-4e4e-a9b8-180c5fefba57

VM2-
Code: 
agent: 1
balloon: 0
bios: ovmf
boot: order=scsi0;net0;sata1
cores: 4
efidisk0: NVMe:vm-113-disk-2,size=128K
machine: pc-i440fx-6.0
memory: 4096
name: VM-WEB01
net0: virtio=7E:D3:3E:D4:46:80,bridge=vmbr0,tag=60
numa: 0
ostype: win10
parent: fail_boot
sata1: none,media=cdrom
scsi0: NVMe:vm-113-disk-1,size=50G
scsihw: virtio-scsi-pci
smbios1: uuid=b1946c07-ed77-4c5b-b6c6-fdaae402a69e
sockets: 1
vmgenid: 41b6dbbe-f41a-4832-ad02-6490166ce0d7
[PENDING]
cpu: host
[fail_boot]
agent: 1
balloon: 0
bios: ovmf
boot: order=scsi0;net0;sata1
cores: 4
efidisk0: NVMe:vm-113-disk-2,size=128K
machine: pc-i440fx-6.0
memory: 4096
name: VM-WEB01
net0: virtio=7E:D3:3E:D4:46:80,bridge=vmbr0,tag=60
numa: 0
ostype: win10
sata1: none,media=cdrom
scsi0: NVMe:vm-113-disk-1,size=50G
scsihw: virtio-scsi-pci
smbios1: uuid=b1946c07-ed77-4c5b-b6c6-fdaae402a69e
snaptime: 1646426831
sockets: 1
vmgenid: 41b6dbbe-f41a-4832-ad02-6490166ce0d7
 
  • Like
Reactions: weehooey-bh
I have tested 2 additional VMs tonight while installing updates.

2K16 win std VMs.

Updates install, reboot VMs (from inside the VM), watch the console and stuck at the boot screen with the spinning dots.

Stop/Start

VM comes back up, check updates and they have installed.

Reboot VM (from inside the VM), watch the console and stuck at the boot screen. Sigh.

I did take advice from someone else on this forum and changed the : Hardware>Processors>Type: from Default (kvm64) to Host and then they start rebooting normally.

Edit: one was a Hyper-V import, the other was a raw/KVM based image import.
 
Last edited:
vulture said:
Does the Backup contain RAM? Because when just using the exact snapshot without RAM (so the VM does a "normal" startup), the problem cannot be replicated anymore...
Click to expand...
Yes, doing a snapshot with include the ram. If you suspect that restarting the VM will get stuck do a snapshot including the RAM. If the issue happens then do a rollback and then backup the VM and provide us with the backup file.
 
Moayad said:
Yes, doing a snapshot with include the ram. If you suspect that restarting the VM will get stuck do a snapshot including the RAM. If the issue happens then do a rollback and then backup the VM and provide us with the backup file.
Click to expand...

Doing so there is a problem. If the backup of the VM is given to you, it is very likely that post restore is working correctly. The snapshot with RAM should also be delivered to you. this is because if the VM does not have suitable boundary conditions the problem does not occur. IMHO
 
dea said:
Any tests with version 7.1-11? The list of updated packages does not include QEMU and/or the kernel, so I assume the problem remains. I'm terrified of finding dozens and dozens of VMs crashed on boot at the next reboot for Windows updates. Sigh
Click to expand...
Still exists on 7.1-11....
 
  • Like
Reactions: weehooey-bh
Hi,

We are currently evaluating Proxmox as a replacement for our VMware environment and we are also seeing this problem.

We have a migration process from VMware where we transform our VMDK disks to qcow2 format and run virt-v2v on them before we attach them to a new VM in proxmox. Everything works fine except that we have encountered this problem on Proxmox after the VM has been migrated.

After reading through this thread, it seems to be the exact same problem that we are experiencing. However, I have noted that if we skip the process of installing VirtIO drivers and QEMU guest agents, our VMs reboot just fine. So it seems to me like this problem has something to do with the communication between the VM and Proxmox during reboot when the QEMU guest agent is installed.
 
  • Like
Reactions: weehooey-bh
dea said:
Doing so there is a problem. If the backup of the VM is given to you, it is very likely that post restore is working correctly. The snapshot with RAM should also be delivered to you. this is because if the VM does not have suitable boundary conditions the problem does not occur. IMHO
Click to expand...

Yes, exactly that. A plain backup is useless. You may be able to replicate the problem by running the VM like at least 4 weeks and then try rebooting it. The VM itself is just a plain windows server installation without any modifications (not even virtio drivers, VM is using sata disks)

I can provide the raw data, if you tell me how you'd like the snapshot to be exported. The VM is currently running with a "Westmere" CPU.
 
While debugging, we activated the bootlog and noticed that the boot seems to be hanging/looping when loading a driver (see below)

on a normal boot, the wdNisDrv.sys is only loaded once. Unfortunately I'm not really deep into windows internals, but maybe it gives someone else an idea.

Code: 
Microsoft (R) Windows (R) Version 10.0 (Build 17763)
 1 13 2022 09:51:36.500
BOOTLOG_LOADED \SystemRoot\system32\ntoskrnl.exe
BOOTLOG_LOADED \SystemRoot\system32\hal.dll
BOOTLOG_LOADED \SystemRoot\system32\kd.dll
BOOTLOG_LOADED \SystemRoot\system32\mcupdate_GenuineIntel.dll
BOOTLOG_LOADED \SystemRoot\System32\drivers\msrpc.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\ksecdd.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\werkernel.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\CLFS.SYS
BOOTLOG_LOADED \SystemRoot\System32\drivers\tm.sys
BOOTLOG_LOADED \SystemRoot\system32\PSHED.dll
BOOTLOG_LOADED \SystemRoot\system32\BOOTVID.dll
BOOTLOG_LOADED \SystemRoot\System32\drivers\FLTMGR.SYS
BOOTLOG_LOADED \SystemRoot\System32\drivers\clipsp.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\cmimcext.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\ntosext.sys
BOOTLOG_LOADED \SystemRoot\system32\CI.dll
BOOTLOG_LOADED \SystemRoot\System32\drivers\cng.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\Wdf01000.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\WDFLDR.SYS
BOOTLOG_LOADED \SystemRoot\system32\drivers\WppRecorder.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\SleepStudyHelper.sys
BOOTLOG_LOADED \SystemRoot\System32\Drivers\acpiex.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\mssecflt.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\SgrmAgent.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\ACPI.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\WMILIB.SYS
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdBoot.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\WindowsTrustedRT.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\intelpep.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\WindowsTrustedRTProxy.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\pcw.sys
BOOTLOG_LOADED \SystemRoot\system32\DRIVERS\sacdrv.sys
BOOTLOG_LOADED \SystemRoot\system32\DRIVERS\NDIS.SYS
BOOTLOG_LOADED \SystemRoot\system32\DRIVERS\NETIO.SYS
BOOTLOG_LOADED \SystemRoot\System32\drivers\msisadrv.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\vdrvroot.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\pci.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\pdc.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\CEA.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\partmgr.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\intelide.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\PCIIDEX.SYS
BOOTLOG_LOADED \SystemRoot\System32\drivers\spaceport.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\volmgr.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\volmgrx.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\mountmgr.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\atapi.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\ataport.SYS
BOOTLOG_LOADED \SystemRoot\System32\drivers\vioscsi.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\storport.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\EhStorClass.sys
BOOTLOG_LOADED \SystemRoot\System32\Drivers\Wof.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdFilter.sys
BOOTLOG_LOADED \SystemRoot\System32\Drivers\Ntfs.sys
BOOTLOG_LOADED \SystemRoot\System32\Drivers\Fs_Rec.sys
BOOTLOG_LOADED \SystemRoot\System32\Drivers\ksecpkg.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\tcpip.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\fwpkclnt.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\wfplwfs.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\volume.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\volsnap.sys
BOOTLOG_LOADED \SystemRoot\System32\Drivers\mup.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\hwpolicy.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\disk.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\CLASSPNP.SYS
BOOTLOG_LOADED \SystemRoot\System32\drivers\cdrom.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\filecrypt.sys
BOOTLOG_LOADED \SystemRoot\System32\Drivers\Null.SYS
BOOTLOG_LOADED \SystemRoot\System32\drivers\dxgkrnl.sys
BOOTLOG_NOT_LOADED \SystemRoot\System32\drivers\dxgkrnl.sys
BOOTLOG_LOADED \SystemRoot\System32\DriverStore\FileRepository\basicdisplay.inf_amd64_5103ac179273be89\BasicDisplay.sys
BOOTLOG_NOT_LOADED \SystemRoot\System32\drivers\dxgkrnl.sys
BOOTLOG_LOADED \SystemRoot\System32\DriverStore\FileRepository\basicrender.inf_amd64_0b8d03c3bc0e7fd9\BasicRender.sys
BOOTLOG_LOADED \SystemRoot\System32\Drivers\Npfs.SYS
BOOTLOG_LOADED \SystemRoot\System32\Drivers\Msfs.SYS
BOOTLOG_LOADED \SystemRoot\system32\DRIVERS\tdx.sys
BOOTLOG_LOADED \SystemRoot\System32\DRIVERS\netbt.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\afunix.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\afd.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\pacer.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\netbios.sys
BOOTLOG_LOADED \SystemRoot\system32\DRIVERS\rdbss.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\nsiproxy.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\npsvctrig.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\mssmbios.sys
BOOTLOG_LOADED \SystemRoot\System32\Drivers\dfsc.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\bam.sys
BOOTLOG_LOADED \SystemRoot\system32\DRIVERS\ahcache.sys
BOOTLOG_LOADED \SystemRoot\System32\DriverStore\FileRepository\compositebus.inf_amd64_e4d35af746093dc3\CompositeBus.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\kdnic.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\umbus.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\i8042prt.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\kbdclass.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\mouclass.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\fdc.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\usbuhci.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\netkvm.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\intelppm.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\vmgencounter.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\NdisVirtualBus.sys
BOOTLOG_LOADED \SystemRoot\System32\DriverStore\FileRepository\swenum.inf_amd64_31f554b660026323\swenum.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\rdpbus.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\usbhub.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\hidusb.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\mouhid.sys
BOOTLOG_NOT_LOADED \SystemRoot\System32\drivers\dxgkrnl.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\monitor.sys
BOOTLOG_NOT_LOADED \SystemRoot\System32\drivers\dxgkrnl.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wcifs.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\luafv.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\cldflt.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\storqosflt.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\lltdio.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\mslldp.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\rspndr.sys
BOOTLOG_LOADED \SystemRoot\system32\DRIVERS\bowser.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\mpsdrv.sys
BOOTLOG_LOADED \SystemRoot\system32\DRIVERS\mrxsmb.sys
BOOTLOG_LOADED \SystemRoot\system32\DRIVERS\mrxsmb20.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\winquic.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\HTTP.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\peauth.sys
BOOTLOG_LOADED \SystemRoot\System32\DRIVERS\srvnet.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\tcpipreg.sys
BOOTLOG_LOADED \SystemRoot\System32\DRIVERS\srv2.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\condrv.sys
BOOTLOG_NOT_LOADED \SystemRoot\system32\drivers\wd\WdFilter.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdFilter.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdFilter.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
 
  • Like
Reactions: jlebherz and weehooey-bh
vulture said:
While debugging, we activated the bootlog and noticed that the boot seems to be hanging/looping when loading a driver (see below)

on a normal boot, the wdNisDrv.sys is only loaded once. Unfortunately I'm not really deep into windows internals, but maybe it gives someone else an idea.

Code: 
Microsoft (R) Windows (R) Version 10.0 (Build 17763)
 1 13 2022 09:51:36.500
BOOTLOG_LOADED \SystemRoot\system32\ntoskrnl.exe
BOOTLOG_LOADED \SystemRoot\system32\hal.dll
BOOTLOG_LOADED \SystemRoot\system32\kd.dll
BOOTLOG_LOADED \SystemRoot\system32\mcupdate_GenuineIntel.dll
BOOTLOG_LOADED \SystemRoot\System32\drivers\msrpc.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\ksecdd.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\werkernel.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\CLFS.SYS
BOOTLOG_LOADED \SystemRoot\System32\drivers\tm.sys
BOOTLOG_LOADED \SystemRoot\system32\PSHED.dll
BOOTLOG_LOADED \SystemRoot\system32\BOOTVID.dll
BOOTLOG_LOADED \SystemRoot\System32\drivers\FLTMGR.SYS
BOOTLOG_LOADED \SystemRoot\System32\drivers\clipsp.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\cmimcext.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\ntosext.sys
BOOTLOG_LOADED \SystemRoot\system32\CI.dll
BOOTLOG_LOADED \SystemRoot\System32\drivers\cng.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\Wdf01000.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\WDFLDR.SYS
BOOTLOG_LOADED \SystemRoot\system32\drivers\WppRecorder.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\SleepStudyHelper.sys
BOOTLOG_LOADED \SystemRoot\System32\Drivers\acpiex.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\mssecflt.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\SgrmAgent.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\ACPI.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\WMILIB.SYS
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdBoot.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\WindowsTrustedRT.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\intelpep.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\WindowsTrustedRTProxy.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\pcw.sys
BOOTLOG_LOADED \SystemRoot\system32\DRIVERS\sacdrv.sys
BOOTLOG_LOADED \SystemRoot\system32\DRIVERS\NDIS.SYS
BOOTLOG_LOADED \SystemRoot\system32\DRIVERS\NETIO.SYS
BOOTLOG_LOADED \SystemRoot\System32\drivers\msisadrv.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\vdrvroot.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\pci.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\pdc.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\CEA.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\partmgr.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\intelide.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\PCIIDEX.SYS
BOOTLOG_LOADED \SystemRoot\System32\drivers\spaceport.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\volmgr.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\volmgrx.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\mountmgr.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\atapi.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\ataport.SYS
BOOTLOG_LOADED \SystemRoot\System32\drivers\vioscsi.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\storport.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\EhStorClass.sys
BOOTLOG_LOADED \SystemRoot\System32\Drivers\Wof.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdFilter.sys
BOOTLOG_LOADED \SystemRoot\System32\Drivers\Ntfs.sys
BOOTLOG_LOADED \SystemRoot\System32\Drivers\Fs_Rec.sys
BOOTLOG_LOADED \SystemRoot\System32\Drivers\ksecpkg.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\tcpip.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\fwpkclnt.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\wfplwfs.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\volume.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\volsnap.sys
BOOTLOG_LOADED \SystemRoot\System32\Drivers\mup.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\hwpolicy.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\disk.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\CLASSPNP.SYS
BOOTLOG_LOADED \SystemRoot\System32\drivers\cdrom.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\filecrypt.sys
BOOTLOG_LOADED \SystemRoot\System32\Drivers\Null.SYS
BOOTLOG_LOADED \SystemRoot\System32\drivers\dxgkrnl.sys
BOOTLOG_NOT_LOADED \SystemRoot\System32\drivers\dxgkrnl.sys
BOOTLOG_LOADED \SystemRoot\System32\DriverStore\FileRepository\basicdisplay.inf_amd64_5103ac179273be89\BasicDisplay.sys
BOOTLOG_NOT_LOADED \SystemRoot\System32\drivers\dxgkrnl.sys
BOOTLOG_LOADED \SystemRoot\System32\DriverStore\FileRepository\basicrender.inf_amd64_0b8d03c3bc0e7fd9\BasicRender.sys
BOOTLOG_LOADED \SystemRoot\System32\Drivers\Npfs.SYS
BOOTLOG_LOADED \SystemRoot\System32\Drivers\Msfs.SYS
BOOTLOG_LOADED \SystemRoot\system32\DRIVERS\tdx.sys
BOOTLOG_LOADED \SystemRoot\System32\DRIVERS\netbt.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\afunix.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\afd.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\pacer.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\netbios.sys
BOOTLOG_LOADED \SystemRoot\system32\DRIVERS\rdbss.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\nsiproxy.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\npsvctrig.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\mssmbios.sys
BOOTLOG_LOADED \SystemRoot\System32\Drivers\dfsc.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\bam.sys
BOOTLOG_LOADED \SystemRoot\system32\DRIVERS\ahcache.sys
BOOTLOG_LOADED \SystemRoot\System32\DriverStore\FileRepository\compositebus.inf_amd64_e4d35af746093dc3\CompositeBus.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\kdnic.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\umbus.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\i8042prt.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\kbdclass.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\mouclass.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\fdc.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\usbuhci.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\netkvm.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\intelppm.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\vmgencounter.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\NdisVirtualBus.sys
BOOTLOG_LOADED \SystemRoot\System32\DriverStore\FileRepository\swenum.inf_amd64_31f554b660026323\swenum.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\rdpbus.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\usbhub.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\hidusb.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\mouhid.sys
BOOTLOG_NOT_LOADED \SystemRoot\System32\drivers\dxgkrnl.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\monitor.sys
BOOTLOG_NOT_LOADED \SystemRoot\System32\drivers\dxgkrnl.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wcifs.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\luafv.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\cldflt.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\storqosflt.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\lltdio.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\mslldp.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\rspndr.sys
BOOTLOG_LOADED \SystemRoot\system32\DRIVERS\bowser.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\mpsdrv.sys
BOOTLOG_LOADED \SystemRoot\system32\DRIVERS\mrxsmb.sys
BOOTLOG_LOADED \SystemRoot\system32\DRIVERS\mrxsmb20.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\winquic.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\HTTP.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\peauth.sys
BOOTLOG_LOADED \SystemRoot\System32\DRIVERS\srvnet.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\tcpipreg.sys
BOOTLOG_LOADED \SystemRoot\System32\DRIVERS\srv2.sys
BOOTLOG_LOADED \SystemRoot\System32\drivers\condrv.sys
BOOTLOG_NOT_LOADED \SystemRoot\system32\drivers\wd\WdFilter.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdFilter.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdFilter.sys
BOOTLOG_LOADED \SystemRoot\system32\drivers\wd\WdNisDrv.sys
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
BOOTLOG_LOADED
Click to expand...

​

This is the Windows Defender Antivirus Network Inspection System Driver.
As we have also VMs without Defender at all or with Defender uninstalled, I believe there is not really a relation to this and the hanging reboot....
 
I have been having the same thing happen to me as well. two different clusters. Someone in this thread https://forum.proxmox.com/threads/windows-vms-freezing-during-reboots-after-patching.99547/ said if update to Linux pve-kernel-5.15 it fixed it for them but it's not integrated in the updates and not fully tested. When we updated our Windows VMs to the latest virtio-win-0.1.215 it seemed to have helped but it still does it, just not quite as much it seems, could be it just SEEMS like it though.
This has started to become more than an anoyance.
link to pve-kernal-5.15 thread https://forum.proxmox.com/threads/opt-in-linux-kernel-5-15-for-proxmox-ve-7-x-available.100936/
 
Last edited:
  • Like
Reactions: weehooey-bh
I confirm that with the latest version of the virt-io driver 0.1.215 the problem occurs anyway, indeed, it also occurs on vm not configured with paravirtualized devices (for example sata), see above.

Perhaps only the change of the kernel can be the solution (maybe if the cause is identified we can hope for a backport towards 5.13).
 
  • Like
Reactions: weehooey-bh
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back