Blue screen with 5.1

[cybermcm]

sure, the vm with the GUI option

Spoiler: windows_hidden_devices

 

[due]

Hello Wolfgang,

here the output of my vm config and running win version:

Spoiler: vm config

root@pve01:~# cat /etc/pve/qemu-server/100.conf
agent: 1
bootdisk: ide0
cores: 4
ide0: local-zfs:vm-100-disk-1,size=50G
ide2: none,media=cdrom
memory: 8096
name: WIN10-GNS-02
net0: virtio=72:86:00:AA:999,bridge=vmbr0
numa: 0
ostype: win10
scsihw: virtio-scsi-pci
smbios1: uuid=5aef7c74-12e6-4c2f-94ec-256c2df7c1fe
sockets: 1
root@pve01:~#

Spoiler: Win Version

OS Name: Microsoft Windows 10 Pro
OS Version: 10.0.15063 N/A Build 15063
OS Manufacturer: Microsoft Corporation
OS Configuration: Standalone Workstation
OS Build Type: Multiprocessor Free

System Manufacturer: QEMU
System Model: Standard PC (i440FX + PIIX, 1996)
System Type: x64-based PC
Processor(s): 1 Processor(s) Installed.
[01]: Intel64 Family 15 Model 6 Stepping 1 GenuineIntel ~1900 Mhz
BIOS Version: SeaBIOS rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org, 01.04.2014
Windows Directory: C:\WINDOWS
System Directory: C:\WINDOWS\system32
Boot Device: \Device\HarddiskVolume1
Time Zone: (UTC+01:00) Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna
Total Physical Memory: 8'095 MB
Available Physical Memory: 6'186 MB
Virtual Memory: Max Size: 9'375 MB
Virtual Memory: Available: 7'374 MB
Virtual Memory: In Use: 2'001 MB
Page File Location(s): C:\pagefile.sys
Domain: WORKGROUP
Hotfix(s): 4 Hotfix(s) Installed.
[01]: KB4022405
[02]: KB4038806
[03]: KB4049179
[04]: KB4041676
Network Card(s): 1 NIC(s) Installed.
[01]: Red Hat VirtIO Ethernet Adapter
Connection Name: Ethernet 2
DHCP Enabled: Yes
DHCP Server: 192.168.200.33
IP address(es)
[01]: 192.168.200.115
[02]: fe80::b5ee:ae0:1087:f977
Hyper-V Requirements: A hypervisor has been detected. Features required for Hyper-V will not be displayed.

Thanks for looking into this issue.
cheers

 

[spirit]

Hier. Dors win2016 is working on core2 ? ( Without virtu). I Wonder if some software emulated cpu flags could be broken ?

 

[cybermcm]

@spirit: What do you mean with core2 (without virtu)? How can I test this?

 

[iamsparticus]

Just want to confirm downgrading to pve-kernel-4.10.17-4 fixed the BSODs on my environment. It was occurring with Windows 2012R2 and 2016.

I tried changing the CPU type to Host and Core2Duo, and both continued to cause crashes.

I would love to know how to change grub to default to the earlier kernel. Anyone have a cheat sheet?

 

[cpierr03]

Can confirm I also have this bug on two machines running PVE 5.1 One Xeon E5-2683 V3, one i5-3470. Only Windows VMs crash, BSOD is CRITICAL_STRUCTURE_CORRUPTION. All VMs are using VirtIO devices/drivers, and are using the host CPU config.

 

[spirit]

@spirit: What do you mean with core2 (without virtu)? How can I test this?

I mean : Can you install win2016 directly on your server without proxmox ? Physical install. I m not sûre that win2016 is working on this old cpu

 

[cybermcm]

I didn't try to install it directly. You are right, my system is old but other users have the same issue with newer hardware so I think the problem is more or less important to solve.

 

[talos]

I have lots of trouble with Windows 10 KVMs since 5.1 upgrade. Random lookup, bluescreens and reboots with hanging uefi bootscreen. Currently KVM is unsuable. Tried it with kvm64 and host cpu (sandy bridge xeon). Containers running fine.

Downgrade is no option because zfs pool is already upgraded.

 

[wolfgang]

I can't reproduce it with no extra Software on Win.

On your Windows VM which Software is installed.

I test on the same HW as @due

Are all your Windows VM updated?

The complete build version and hotfixes would be instresting.
Run this commands in the powershell

Get-HotFix
[System.Environment]::OSVersion.Version

Please check your Windows is updated.

 

[cybermcm]

2 servers with bluescreens: server 1 domain controller, core version; only sw installed: KVM drivers and QEMU agent
2nd server: Server 2016 standard, Virtio drivers, 7zip, Firefox, Java, Project Honolulu

Spoiler: version

PS C:\Windows\system32> Get-HotFix

Source Description HotFixID InstalledBy InstalledOn
------ ----------- -------- ----------- -----------
SRV10 Update KB3186568 NT AUTHORITY\SYSTEM 26.06.2017 00:00:00
SRV10 Update KB3192137 NT AUTHORITY\SYSTEM 12.09.2016 00:00:00
SRV10 Update KB3199986 NT AUTHORITY\SYSTEM 19.12.2016 00:00:00
SRV10 Update KB3211320 NT AUTHORITY\SYSTEM 25.01.2017 00:00:00
SRV10 Update KB4013418 NT AUTHORITY\SYSTEM 15.03.2017 00:00:00
SRV10 Update KB4023834 NT AUTHORITY\SYSTEM 24.06.2017 00:00:00
SRV10 Update KB4035631 NT AUTHORITY\SYSTEM 08.08.2017 00:00:00
SRV10 Security Update KB4041691 NT AUTHORITY\SYSTEM 10.10.2017 00:00:00


PS C:\Windows\system32> [System.Environment]::OSVersion.Version

Major Minor Build Revision
----- ----- ----- --------
10 0 14393 0

 

[talos]

Today the same issue with Windows 10 16299.19

Start VM > wait a little bit > boom > reboot > OVMF Bios stuck with Proxmox logo and freeze with KVM sitting there at 100% CPU load. The only way to get rid of the VM is by killing KVM.

VM is running virtio drivers 0.1.126

VM Config:

agent: 1
balloon: 4096
bios: ovmf
boot: c
bootdisk: virtio0
cores: 2
efidisk0: pve-vm-ssd1:vm-109-disk-3,size=4M
ide2: none,media=cdrom
keyboard: de
memory: 6144
name: win10test1
net0: virtio=xx:xx:xx:xx:xx:xx,bridge=vmbr0
numa: 0
ostype: win10
protection: 1
scsihw: virtio-scsi-pci
smbios1: uuid=xxx
sockets: 1
vga: qxl
virtio0: pve-vm-ssd1:vm-109-disk-1,size=50G
virtio1: pve-vm-ssd1:vm-109-disk-2,size=50G

PVE Version:

pveversion -v
proxmox-ve: 5.1-25 (running kernel: 4.13.4-1-pve)
pve-manager: 5.1-36 (running version: 5.1-36/131401db)
pve-kernel-4.13.4-1-pve: 4.13.4-25
libpve-http-server-perl: 2.0-6
lvm2: 2.02.168-pve6
corosync: 2.4.2-pve3
libqb0: 1.0.1-1
pve-cluster: 5.0-15
qemu-server: 5.0-17
pve-firmware: 2.0-3
libpve-common-perl: 5.0-20
libpve-guest-common-perl: 2.0-13
libpve-access-control: 5.0-7
libpve-storage-perl: 5.0-16
pve-libspice-server1: 0.12.8-3
vncterm: 1.5-2
pve-docs: 5.1-12
pve-qemu-kvm: 2.9.1-2
pve-container: 2.0-17
pve-firewall: 3.0-3
pve-ha-manager: 2.0-3
ksm-control-daemon: 1.2-2
glusterfs-client: 3.8.8-1
lxc-pve: 2.1.0-2
lxcfs: 2.0.7-pve4
criu: 2.11.1-1~bpo90
novnc-pve: 0.6-4
smartmontools: 6.5+svn4324-1
zfsutils-linux: 0.7.2-pve1~bpo90
openvswitch-switch: 2.7.0-2

Windows 10 Version and fixes:

Source        Description      HotFixID      InstalledBy          InstalledOn
------        -----------      --------      -----------          -----------
WIN10TEST1    Security Update  KB4049179     NT AUTHORITY\SYSTEM  29.10.2017 00:00:00
WIN10TEST1    Update           KB4051613     NT AUTHORITY\SYSTEM  02.11.2017 00:00:00
WIN10TEST1    Security Update  KB4043961     NT AUTHORITY\SYSTEM  18.10.2017 00:00:00

Major  Minor  Build  Revision
-----  -----  -----  --------
10     0      16299  0

Event Log in Windows says "The system has rebooted without cleanly shutting down first. This error could be caused if the system stopped responding, crashed, or lost power unexpectedly."

There is no dump or anything, just hard reset.

 

[wolfgang]

@talos
please use the virtio 0.1.141 (current stable)
The 0.1.126 is know to be buggy with Win2016 and Win 10

 

[wolfgang]

@talos
What cpu do you use on this Host?

 

[cpierr03]

I can't reproduce it with no extra Software on Win.

On your Windows VM which Software is installed.

I test on the same HW as @due

Are all your Windows VM updated?

The complete build version and hotfixes would be instresting.
Run this commands in the powershell

Get-HotFix
[System.Environment]::OSVersion.Version

Please check your Windows is updated.

This is on a clean Win 10 1709 build with nothing but stable VirtIO drivers installed and the only KB available for this platform: KB4043961. The image came from Microsoft and is SHA1 verified.

Really don't think this is a Windows problem here, as others have booted the 4.10 kernel and had no issue. Unfortunately this isn't an option for me, as reverting the kernel prevents me from mounting my ZFS disks that were created after the upgrade.

Furthermore, I was able to replicate the problem even just booting the Windows ISO with no VirtIO devices.

The issue seems more pronounced on my 3470 machine than the Xeon machine.

 

[wolfgang]

Really don't think this is a Windows problem here, as others have booted the 4.10 kernel and had no issue.

It has nothing to do where the problem is but i need a basic to debug.
But thanks for your report.

 

[wolfgang]

@cpierr03
What Xeon do you use.
Please the Product Name.

 

[cpierr03]

@cpierr03
What Xeon do you use.
Please the Product Name.

Xeon E5-2683-V3 (Haswell)

i5 machine is a Core i5-3470 (Ivy Bridge)

 

[cybermcm]

Just to be sure: One user claimed that it only happens with VM on a SSD drive. I've different servers with different CPUs running 5.1 and Server 2016 without issues. My problem "server" has a SSD drive. I've no clue how this can be connected but maybe other users have the same setup?

 

[wolfgang]

@cybermcm
I test on a Samsung SSD and Crucial SSD MX200.