High VM-EXIT and Host CPU usage on idle with Windows Server 2025
- Thread starter orange
- Start date
@lucius: The timers for Windows 11 are per core, so more cores is more (apparent) CPU usage. On 2 vCPUs you won't see the issue, I only see the issue appearing beyond 4 cores (hardware or virtual). Also dependent on how old your hardware is and what other settings you have.
useplatformtick yes - forces the kernel to maintain ticking clocks (like HPET or ACPI)
useplatformclock no - disables HPET
TSC is a tickless clock AND you're disabling HPET, so is Windows falling back to ACPI clocks? Not sure what this ultimately 'does', someone has to go in and poke around with some low level Windows utilities (because Microsoft kernel is weird). This *should* increase your power usage, because now your kernel continues waking up to take care of timers, but this may reduce your *apparent* CPU usage as reported in various statistics as your CPU is continuously 'at full power' and thus doesn't have to wake (which can take hundreds of clock cycles) to service a timer.
IF that 'fixes' the problem, you'll likely see issues especially in gaming/timing/audio applications and significant latency increases in things like VDI. I guess if you're worried about pure energy efficiency you should really take care and measure, but I'm assuming for most people this will result in significant performant impacts for workloads where timing/clocks are necessary (anything with networking, audio and interactivity).
useplatformtick yes - forces the kernel to maintain ticking clocks (like HPET or ACPI)
useplatformclock no - disables HPET
TSC is a tickless clock AND you're disabling HPET, so is Windows falling back to ACPI clocks? Not sure what this ultimately 'does', someone has to go in and poke around with some low level Windows utilities (because Microsoft kernel is weird). This *should* increase your power usage, because now your kernel continues waking up to take care of timers, but this may reduce your *apparent* CPU usage as reported in various statistics as your CPU is continuously 'at full power' and thus doesn't have to wake (which can take hundreds of clock cycles) to service a timer.
IF that 'fixes' the problem, you'll likely see issues especially in gaming/timing/audio applications and significant latency increases in things like VDI. I guess if you're worried about pure energy efficiency you should really take care and measure, but I'm assuming for most people this will result in significant performant impacts for workloads where timing/clocks are necessary (anything with networking, audio and interactivity).
Last edited:
https://lwn.net/Articles/1051782/
I'm starting to wonder if AMD will fix this, too.
This is for Intel, right?
Code:
This series introduces support for Intel Mode-Based Execute Control
(MBEC) to KVM and nested VMX virtualization. By exposing MBEC to L2
guests, it enables a dramatic reduction in VMexits (up to 24x) for
Windows guests running with Hypervisor-Protected Code Integrity (HVCI),
significantly improving virtualization performance.I'm starting to wonder if AMD will fix this, too.
This is for Intel, right?
I can confirm the same. For windows 11 24H2 with all updates installed. I have VBS enabled and also WSL2 installed(but not working!). I am using x86-64-v3 + nested-virt for cpu flags. The kernel version is 6.17.4-2. It is still about 100% increase of idle cpu but not 300/400 %. WSL2 does not like x86-64-X. I am still playing with this to understand what is the best option.
Last edited:
I guess this needs to get into the kernel. Qemu also needs to adopt this. Proxmox will follow ...
You can also disable HVCI in Windows in the mean time, provided that is what is causing this behavior.
MBEC is only for newer CPU, given most people that would complain are running pre-scalable that are technically compatible but not fully compatible with more modern implementations of Windows 10/11. For AMD, you likewise need a ~5-6 year old machine or newer (Zen 2) to get an equivalent Guest Mode Execution Trap.
MBEC is only for newer CPU, given most people that would complain are running pre-scalable that are technically compatible but not fully compatible with more modern implementations of Windows 10/11. For AMD, you likewise need a ~5-6 year old machine or newer (Zen 2) to get an equivalent Guest Mode Execution Trap.
Is it about time?
backport MBEC/GMET support
https://github.com/proxmox/pve-kernel/commit/1e810d779b50dd5e7395fa65ad7b3cbf89b910c1
backport MBEC/GMET support
https://github.com/proxmox/pve-kernel/commit/1e810d779b50dd5e7395fa65ad7b3cbf89b910c1
Fantastic, and I was wrong in my previous post, Proxmox is leading the game
Well done to get it back-ported into 7.0.x I will test it as soon as it gets into the test repo.I haven’t tested it in a test environment yet, but I have confirmed that the flag is recognized on Windows by adding the kernel.
proxmox-kernel-7.0.0-2-pve-signed: 7.0.0-2
proxmox-kernel-7.0.0-3-pve-signed: 7.0.0-3
Enable virtualization-based protection of code integrity
https://learn.microsoft.com/en-us/w...ed-protection-of-code-integrity?tabs=security
edit :
You might be able to configure this below
While the CPU-Z benchmark showed a performance penalty when memory protection was enabled, we confirmed that there was no penalty on the virtual machine after applying the above settings and enabling them.
edit
I said there are no penalties, but it’s unclear
It fluctuates up and down due to various factors
*The benchmarks immediately after the reboot didn't seem to show any performance penalty, but I'm not sure if things improved since subsequent benchmarks showed slower performance.
proxmox-kernel-7.0.0-2-pve-signed: 7.0.0-2
Code:
PS C:\Users\admin> (Get-CimInstance -ClassName Win32_DeviceGuard -Namespace root\Microsoft\Windows\DeviceGuard).AvailableSecurityProperties
1
2
3
4
5
PS C:\Users\admin>proxmox-kernel-7.0.0-3-pve-signed: 7.0.0-3
Code:
PS C:\Users\admin> (Get-CimInstance -ClassName Win32_DeviceGuard -Namespace root\Microsoft\Windows\DeviceGuard).AvailableSecurityProperties
1
2
3
4
5
7 <- MBEC/GMET is available.
PS C:\Users\admin>Enable virtualization-based protection of code integrity
https://learn.microsoft.com/en-us/w...ed-protection-of-code-integrity?tabs=security
Code:
Validate enabled VBS and memory integrity features
Use Win32_DeviceGuard WMI class
7 If present, MBEC/GMET is available.edit :
You might be able to configure this below
Code:
cpu: max,hidden=1,flags=+pdpe1gb
args: -cpu max,migratable=off
Code:
PS C:\Users\admin> (Get-CimInstance -ClassName Win32_DeviceGuard -Namespace root\Microsoft\Windows\DeviceGuard).AvailableSecurityProperties
1
2
5
7
8
PS C:\Users\admin>While the CPU-Z benchmark showed a performance penalty when memory protection was enabled, we confirmed that there was no penalty on the virtual machine after applying the above settings and enabling them.
edit
I said there are no penalties, but it’s unclear
It fluctuates up and down due to various factors
*The benchmarks immediately after the reboot didn't seem to show any performance penalty, but I'm not sure if things improved since subsequent benchmarks showed slower performance.
Last edited:
I can confirm the same:
Code:
PS C:\Users\rl> (Get-CimInstance -ClassName Win32_DeviceGuard -Namespace root\Microsoft\Windows\DeviceGuard).AvailableSecurityProperties
>>
1
2
5
7
8But only if I use this:
Code:
cpu: max,hidden=1,flags=+pdpe1gb
args: -cpu max,migratable=offOtherwise MBEC is not available.
I have not seen this option "max" before. I thought that host is the "max".
Also in your example you showed that "3" is available, how did you get that enabled ? That is DMA protection, right ?
When I booted it on the virtual machine I use as a console, it included 3, but I’m not sure what setting caused that.
*Note: With this setting, if I enable memory protection and then reboot, the setting reverts to its default, so there seems to be some kind of issue.
That's why I just tried using max.
Code:
affinity: 0-15
agent: 1
allow-ksm: 0
args: -cpu host,migratable=off,hv_passthrough,-hypervisor,level=30,+vmx,invtsc=on,guest-phys-bits=39 -global intel-iommu.aw-bits=39
balloon: 0
bios: ovmf
boot: order=ide0;scsi0
cores: 16
cpu: host,hidden=1,flags=+pdpe1gb
efidisk0:
hookscript: local:snippets/rx9070_reset.sh
hostpci0: 0000:04:00,pcie=1,rombar=0
hostpci1: 0000:83:00,pcie=1
hostpci2: 0000:01:00.0,pcie=1
machine: pc-q35-10.1,viommu=intel
memory: 49152
meta: creation-qemu=10.1.2,ctime=1768884417
name: etc
net0: virtio=,bridge=vmbr0,firewall=1,mtu=1,queues=2
net1: virtio=,bridge=vmbr1,firewall=1,mtu=1,queues=2
numa: 0
onboot: 0
ostype: win11
scsi0:
scsihw: virtio-scsi-single
smbios1:
sockets: 1
tags: default
tpmstate0: zfs_husmm3280ass201:vm-926-disk-1,size=4M,version=v2.0
usb0: host=001f:0b21
usb1: host=152d:9561
vga: none
vmgenid:It was added the other day.
I'm not sure if this will work, but there might be a flag that's only available on the host side. I think that difference is what caused the discrepancy in the command's results. In that sense, “max” probably isn't actually the “maximum” value.
Last edited:
Have you tested MBEC a bit more ? I am testing WIn 11 VM now and I already got a few BSOD, some Watchdog - I forgot to take a screenshoot.
Also I noticed that using RDP makes VM freeze, while through console it works fine. And it is almost instant.
Maybe it is just my VM, I have "abused" this VM quite a lot due to VBS/Hyper-V testing.
This kernel itself only just appeared in apt, so I haven’t tested it extensively.
I simply installed a fresh copy of Windows 11 to verify whether the kernel update actually works (i.e., whether it can be enabled).
It will be a while before I apply it to the virtual machine I use primarily.
For now, at least, I haven’t encountered any issues.
*Though I haven’t touched anything other than CPU-Z and the OS executable.
I tried using RDP for now, and it didn't cause a BSOD.
I simply installed a fresh copy of Windows 11 to verify whether the kernel update actually works (i.e., whether it can be enabled).
It will be a while before I apply it to the virtual machine I use primarily.
For now, at least, I haven’t encountered any issues.
*Though I haven’t touched anything other than CPU-Z and the OS executable.
I tried using RDP for now, and it didn't cause a BSOD.
While reviewing the logs, I noticed ISCSI timeouts and entries like the one below, so it’s still not at a usable level.
I won’t investigate further; I’ll just quietly revert the changes and wrap up this check.
*I believe files such as kvm/vmx/nested.c are the ones affected by this fix, and I think the error is caused by this backport.
Besides, these logs weren't appearing at all before I updated the kernel.
I won’t investigate further; I’ll just quietly revert the changes and wrap up this check.
*I believe files such as kvm/vmx/nested.c are the ones affected by this fix, and I think the error is caused by this backport.
Besides, these logs weren't appearing at all before I updated the kernel.
Code:
Apr 23 10:24:37 pve1 kernel: ------------[ cut here ]------------
Apr 23 10:24:37 pve1 kernel: WARNING: arch/x86/kvm/vmx/nested.c:4462 at vmx_check_nested_events+0x910/0x920 [kvm_intel], CPU#13: CPU 2/KVM/5429
Apr 23 10:24:37 pve1 kernel: Modules linked in: tcp_diag inet_diag cmac nls_utf8 cifs nls_ucs2_utils rdma_cm iw_cm ib_cm ib_core cifs_md4 netfs ebtable_filter ebtables ip_set ip6table_raw iptable_raw ip6table_filter ip6_tables iptable_filter nf_tables softdog sunrpc binfmt_misc nfnetlink_cttimeout bonding tls openvswitch nsh nf_conncount nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 psample nfnetlink_log snd_hda_codec_intelhdmi snd_sof_pci_intel_mtl snd_sof_intel_hda_generic soundwire_intel snd_sof_intel_hda_sdw_bpt snd_sof_intel_hda_common snd_soc_hdac_hda snd_sof_intel_hda_mlink snd_sof_intel_hda soundwire_cadence snd_sof_pci snd_sof_xtensa_dsp snd_sof snd_sof_utils snd_hda_codec_alc662 iwlmvm snd_hda_ext_core snd_hda_codec_realtek_lib snd_soc_acpi_intel_match intel_uncore_frequency intel_uncore_frequency_common snd_soc_acpi_intel_sdca_quirks snd_hda_codec_generic soundwire_generic_allocation xe snd_hda_codec_atihdmi intel_pmc_core snd_soc_sdw_utils snd_hda_codec_hdmi mac80211 snd_soc_acpi soundwire_bus btusb drm_gpusvm_helper
Apr 23 10:24:37 pve1 kernel: snd_hda_intel altera_cvp x86_pkg_temp_thermal snd_soc_sdca btmtk fpga_mgr drm_gpuvm intel_powerclamp amdgpu libarc4 snd_hda_codec snd_usb_audio snd_soc_core btrtl kvm_intel snd_hda_core snd_usbmidi_lib pmt_telemetry amdxcp pmt_discovery snd_ump btbcm snd_intel_dspcfg snd_compress drm_panel_backlight_quirks i915 mxl301rf mei_gsc_proxy pmt_class intel_rapl_msr gpu_sched btintel snd_intel_sdw_acpi ac97_bus snd_rawmidi kvm drm_ttm_helper snd_hwdep drm_buddy snd_pcm_dmaengine processor_thermal_device_pci qm1d1c0042 drm_exec snd_seq_device drm_suballoc_helper cmdlinepart ttm processor_thermal_device ghash_clmulni_intel iwlwifi processor_thermal_wt_hint tc90522 aesni_intel drm_display_helper platform_temperature_control spi_nor snd_pcm rapl processor_thermal_soc_slider earth_pt3 processor_thermal_rfim cec snd_timer ses processor_thermal_rapl mtd crc8 intel_cstate dvb_core pcspkr wmi_bmof cfg80211 intel_rapl_common snd rc_core enclosure bluetooth mc intel_vpu mei_me soundcore i2c_algo_bit processor_thermal_wt_req
Apr 23 10:24:37 pve1 kernel: intel_pmc_ssram_telemetry processor_thermal_power_floor mei intel_vsec processor_thermal_mbox platform_profile int3403_thermal int340x_thermal_zone acpi_tad int3400_thermal acpi_thermal_rel acpi_pad input_leds joydev mac_hid sch_fq_codel msr vhost_net vhost vhost_iotlb tap coretemp nct6683 vfio_pci vfio_pci_core irqbypass vfio_iommu_type1 vfio iommufd efi_pstore nfnetlink dmi_sysfs ip_tables x_tables autofs4 zfs(PO) spl(O) uas btrfs libblake2b xor raid6_pq usbmouse usbkbd hid_generic usbhid hid usb_storage xhci_pci_renesas i40e nvme mpt3sas i2c_i801 xhci_pci r8169 i2c_mux spi_intel_pci nvme_core intel_lpss_pci thunderbolt raid_class ahci libie i2c_smbus intel_lpss spi_intel nvme_keyring realtek xhci_hcd libie_adminq scsi_transport_sas video libahci nvme_auth idma64 vmd hkdf wmi
Apr 23 10:24:37 pve1 kernel: CPU: 13 UID: 0 PID: 5429 Comm: CPU 2/KVM Tainted: P O 7.0.0-3-pve #1 PREEMPT(lazy)
Apr 23 10:24:37 pve1 kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE
Apr 23 10:24:37 pve1 kernel: Hardware name: ASRock Z890 Pro RS WiFi White/Z890 Pro RS WiFi White, BIOS 3.24 01/28/2026
Apr 23 10:24:37 pve1 kernel: RIP: 0010:vmx_check_nested_events+0x910/0x920 [kvm_intel]
Apr 23 10:24:37 pve1 kernel: Code: 48 89 df e8 82 09 8c ff e9 c6 f7 ff ff 45 85 ff 0f 85 55 fc ff ff c6 83 90 21 00 00 01 48 89 df e8 55 03 8c ff e9 4d f9 ff ff <0f> 0b e9 46 f9 ff ff 66 0f 1f 84 00 00 00 00 00 90 90 90 90 90 90
Apr 23 10:24:37 pve1 kernel: RSP: 0018:ffffd05c8f833c30 EFLAGS: 00010286
Apr 23 10:24:37 pve1 kernel: RAX: 00000000ffffffff RBX: ffff8eea48dcc900 RCX: 0000000000000000
Apr 23 10:24:37 pve1 kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
Apr 23 10:24:37 pve1 kernel: RBP: ffffd05c8f833c70 R08: 0000000000000000 R09: 0000000000000000
Apr 23 10:24:37 pve1 kernel: R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000001
Apr 23 10:24:37 pve1 kernel: R13: 00000000ffffffff R14: 0000000000000001 R15: 0000000000000000
Apr 23 10:24:37 pve1 kernel: FS: 0000707e9b3ff6c0(0000) GS:ffff8f1861b8f000(0000) knlGS:ffff9f01e23e8000
Apr 23 10:24:37 pve1 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Apr 23 10:24:37 pve1 kernel: CR2: 00007ffb951cc5e0 CR3: 000000014540b006 CR4: 0000000000f72ef0
Apr 23 10:24:37 pve1 kernel: PKRU: 55555554
Apr 23 10:24:37 pve1 kernel: Call Trace:
Apr 23 10:24:37 pve1 kernel: <TASK>
Apr 23 10:24:37 pve1 kernel: kvm_check_nested_events+0x22/0x50 [kvm]
Apr 23 10:24:37 pve1 kernel: kvm_check_and_inject_events+0x26c/0x550 [kvm]
Apr 23 10:24:37 pve1 kernel: kvm_arch_vcpu_ioctl_run+0x489/0x18c0 [kvm]
Apr 23 10:24:37 pve1 kernel: ? kvm_on_user_return+0x4a/0x90 [kvm]
Apr 23 10:24:37 pve1 kernel: ? __x64_sys_ioctl+0xbf/0x100
Apr 23 10:24:37 pve1 kernel: ? kvm_on_user_return+0x4a/0x90 [kvm]
Apr 23 10:24:37 pve1 kernel: ? fire_user_return_notifiers+0x37/0x70
Apr 23 10:24:37 pve1 kernel: kvm_vcpu_ioctl+0x312/0xba0 [kvm]
Apr 23 10:24:37 pve1 kernel: ? __x64_sys_ioctl+0xbf/0x100
Apr 23 10:24:37 pve1 kernel: ? kvm_on_user_return+0x4a/0x90 [kvm]
Apr 23 10:24:37 pve1 kernel: ? fire_user_return_notifiers+0x37/0x70
Apr 23 10:24:37 pve1 kernel: __x64_sys_ioctl+0xa5/0x100
Apr 23 10:24:37 pve1 kernel: x64_sys_call+0x103b/0x2390
Apr 23 10:24:37 pve1 kernel: do_syscall_64+0x11c/0x14e0
Apr 23 10:24:37 pve1 kernel: ? do_syscall_64+0x3c2/0x14e0
Apr 23 10:24:37 pve1 kernel: ? fire_user_return_notifiers+0x37/0x70
Apr 23 10:24:37 pve1 kernel: ? do_syscall_64+0x3c2/0x14e0
Apr 23 10:24:37 pve1 kernel: ? do_syscall_64+0x3c2/0x14e0
Apr 23 10:24:37 pve1 kernel: entry_SYSCALL_64_after_hwframe+0x76/0x7e
Apr 23 10:24:37 pve1 kernel: RIP: 0033:0x707fad52191b
Apr 23 10:24:37 pve1 kernel: Code: 00 48 89 44 24 18 31 c0 48 8d 44 24 60 c7 04 24 10 00 00 00 48 89 44 24 08 48 8d 44 24 20 48 89 44 24 10 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 ff ff 77 1c 48 8b 44 24 18 64 48 2b 04 25 28 00 00
Apr 23 10:24:37 pve1 kernel: RSP: 002b:0000707e9b3fab30 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
Apr 23 10:24:37 pve1 kernel: RAX: ffffffffffffffda RBX: 000000000000ae80 RCX: 0000707fad52191b
Apr 23 10:24:37 pve1 kernel: RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 000000000000001b
Apr 23 10:24:37 pve1 kernel: RBP: 00005e3a6b64a5b0 R08: 0000000000000000 R09: 0000000000000000
Apr 23 10:24:37 pve1 kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
Apr 23 10:24:37 pve1 kernel: R13: 0000000000000004 R14: 0000000000000608 R15: 0000000000000000
Apr 23 10:24:37 pve1 kernel: </TASK>
Apr 23 10:24:37 pve1 kernel: ---[ end trace 0000000000000000 ]---
Apr 23 13:06:05 pve1 kernel: ------------[ cut here ]------------
Apr 23 13:06:05 pve1 kernel: WARNING: arch/x86/kvm/vmx/nested.c:4462 at vmx_check_nested_events+0x910/0x920 [kvm_intel], CPU#16: CPU 2/KVM/24634
Apr 23 13:06:05 pve1 kernel: Modules linked in: tcp_diag inet_diag cmac nls_utf8 cifs nls_ucs2_utils rdma_cm iw_cm ib_cm ib_core cifs_md4 netfs ebtable_filter ebtables ip_set ip6table_raw iptable_raw ip6table_filter ip6_tables iptable_filter nf_tables softdog sunrpc binfmt_misc nfnetlink_cttimeout bonding tls openvswitch nsh nf_conncount nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 psample nfnetlink_log snd_sof_pci_intel_mtl snd_sof_intel_hda_generic soundwire_intel snd_sof_intel_hda_sdw_bpt snd_sof_intel_hda_common snd_soc_hdac_hda snd_sof_intel_hda_mlink snd_sof_intel_hda soundwire_cadence snd_sof_pci snd_sof_xtensa_dsp snd_sof snd_sof_utils snd_hda_ext_core intel_uncore_frequency iwlmvm snd_soc_acpi_intel_match intel_uncore_frequency_common snd_hda_codec_intelhdmi xe intel_pmc_core snd_soc_acpi_intel_sdca_quirks soundwire_generic_allocation mac80211 snd_hda_codec_alc662 snd_soc_sdw_utils snd_hda_codec_realtek_lib drm_gpusvm_helper altera_cvp x86_pkg_temp_thermal snd_hda_codec_atihdmi fpga_mgr drm_gpuvm intel_powerclamp
Apr 23 13:06:05 pve1 kernel: snd_soc_acpi snd_hda_codec_generic snd_hda_codec_hdmi amdgpu libarc4 snd_hda_intel soundwire_bus kvm_intel snd_soc_sdca amdxcp mxl301rf snd_hda_codec i915 snd_usb_audio drm_panel_backlight_quirks pmt_telemetry snd_hda_core snd_soc_core gpu_sched qm1d1c0042 pmt_discovery snd_usbmidi_lib mei_gsc_proxy snd_intel_dspcfg drm_ttm_helper kvm intel_rapl_msr pmt_class btusb tc90522 drm_buddy snd_ump snd_compress drm_exec earth_pt3 btmtk snd_intel_sdw_acpi iwlwifi snd_rawmidi ttm ac97_bus processor_thermal_device_pci cmdlinepart snd_hwdep drm_suballoc_helper snd_seq_device btrtl ghash_clmulni_intel snd_pcm_dmaengine processor_thermal_device aesni_intel drm_display_helper spi_nor processor_thermal_wt_hint btbcm dvb_core snd_pcm platform_temperature_control rapl btintel ses processor_thermal_soc_slider mtd intel_cstate cec pcspkr wmi_bmof crc8 enclosure mc mei_me snd_timer processor_thermal_rfim bluetooth processor_thermal_rapl cfg80211 snd mei intel_rapl_common rc_core soundcore i2c_algo_bit
Apr 23 13:06:05 pve1 kernel: intel_pmc_ssram_telemetry processor_thermal_wt_req processor_thermal_power_floor intel_vsec intel_vpu processor_thermal_mbox platform_profile int3403_thermal int340x_thermal_zone acpi_tad int3400_thermal acpi_thermal_rel acpi_pad input_leds joydev mac_hid sch_fq_codel msr vhost_net vhost vhost_iotlb tap coretemp nct6683 vfio_pci vfio_pci_core irqbypass vfio_iommu_type1 vfio iommufd efi_pstore nfnetlink dmi_sysfs ip_tables x_tables autofs4 zfs(PO) spl(O) uas btrfs libblake2b xor raid6_pq usbmouse usbkbd hid_generic usbhid hid usb_storage xhci_pci_renesas nvme xhci_pci i2c_i801 mpt3sas i40e nvme_core r8169 spi_intel_pci ahci intel_lpss_pci i2c_mux raid_class thunderbolt libie xhci_hcd spi_intel intel_lpss i2c_smbus realtek nvme_keyring scsi_transport_sas libie_adminq libahci video idma64 vmd nvme_auth hkdf wmi
Apr 23 13:06:05 pve1 kernel: CPU: 16 UID: 0 PID: 24634 Comm: CPU 2/KVM Tainted: P O 7.0.0-3-pve #1 PREEMPT(lazy)
Apr 23 13:06:05 pve1 kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE
Apr 23 13:06:05 pve1 kernel: Hardware name: ASRock Z890 Pro RS WiFi White/Z890 Pro RS WiFi White, BIOS 3.24 01/28/2026
Apr 23 13:06:05 pve1 kernel: RIP: 0010:vmx_check_nested_events+0x910/0x920 [kvm_intel]
Apr 23 13:06:05 pve1 kernel: Code: 48 89 df e8 82 b9 8d ff e9 c6 f7 ff ff 45 85 ff 0f 85 55 fc ff ff c6 83 90 21 00 00 01 48 89 df e8 55 b3 8d ff e9 4d f9 ff ff <0f> 0b e9 46 f9 ff ff 66 0f 1f 84 00 00 00 00 00 90 90 90 90 90 90
Apr 23 13:06:05 pve1 kernel: RSP: 0018:ffffcc8f0ff5f930 EFLAGS: 00010286
Apr 23 13:06:05 pve1 kernel: RAX: 00000000ffffffff RBX: ffff897e19430000 RCX: 0000000000000000
Apr 23 13:06:05 pve1 kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
Apr 23 13:06:05 pve1 kernel: RBP: ffffcc8f0ff5f970 R08: 0000000000000000 R09: 0000000000000000
Apr 23 13:06:05 pve1 kernel: R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000001
Apr 23 13:06:05 pve1 kernel: R13: 00000000ffffffff R14: 0000000000000001 R15: 0000000000000000
Apr 23 13:06:05 pve1 kernel: FS: 00007c3f4e7fc6c0(0000) GS:ffff89a12ff0f000(0000) knlGS:ffffe5019d6e0000
Apr 23 13:06:05 pve1 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Apr 23 13:06:05 pve1 kernel: CR2: 00007ffe308d6f50 CR3: 0000002048d59004 CR4: 0000000000f72ef0
Apr 23 13:06:05 pve1 kernel: PKRU: 55555554
Apr 23 13:06:05 pve1 kernel: Call Trace:
Apr 23 13:06:05 pve1 kernel: <TASK>
Apr 23 13:06:05 pve1 kernel: kvm_check_nested_events+0x22/0x50 [kvm]
Apr 23 13:06:05 pve1 kernel: kvm_check_and_inject_events+0x26c/0x550 [kvm]
Apr 23 13:06:05 pve1 kernel: kvm_arch_vcpu_ioctl_run+0x489/0x18c0 [kvm]
Apr 23 13:06:05 pve1 kernel: ? kvm_arch_vcpu_put+0x1ab/0x210 [kvm]
Apr 23 13:06:05 pve1 kernel: ? vcpu_put+0x22/0x60 [kvm]
Apr 23 13:06:05 pve1 kernel: kvm_vcpu_ioctl+0x312/0xba0 [kvm]
Apr 23 13:06:05 pve1 kernel: ? kvm_vcpu_ioctl+0x2a7/0xba0 [kvm]
Apr 23 13:06:05 pve1 kernel: __x64_sys_ioctl+0xa5/0x100
Apr 23 13:06:05 pve1 kernel: ? bsearch+0x5b/0xa0
Apr 23 13:06:05 pve1 kernel: x64_sys_call+0x103b/0x2390
Apr 23 13:06:05 pve1 kernel: do_syscall_64+0x11c/0x14e0
Apr 23 13:06:05 pve1 kernel: ? __x64_sys_ioctl+0xbf/0x100
Apr 23 13:06:05 pve1 kernel: ? kvm_on_user_return+0x4a/0x90 [kvm]
Apr 23 13:06:05 pve1 kernel: ? fire_user_return_notifiers+0x37/0x70
Apr 23 13:06:05 pve1 kernel: ? do_syscall_64+0x3c2/0x14e0
Apr 23 13:06:05 pve1 kernel: ? kvm_get_linear_rip+0x10a/0x120 [kvm]
Apr 23 13:06:05 pve1 kernel: ? x2apic_send_IPI+0x43/0x60
Apr 23 13:06:05 pve1 kernel: ? native_send_call_func_single_ipi+0x13/0x20
Apr 23 13:06:05 pve1 kernel: ? __smp_call_single_queue+0xf6/0x150
Apr 23 13:06:05 pve1 kernel: ? ttwu_queue_wakelist+0xff/0x110
Apr 23 13:06:05 pve1 kernel: ? try_to_wake_up+0x3bb/0x870
Apr 23 13:06:05 pve1 kernel: ? wake_up_q+0x4b/0xa0
Apr 23 13:06:05 pve1 kernel: ? futex_wake+0xa8/0x1d0
Apr 23 13:06:05 pve1 kernel: ? do_futex+0x18e/0x260
Apr 23 13:06:05 pve1 kernel: ? __x64_sys_futex+0x127/0x200
Apr 23 13:06:05 pve1 kernel: ? x64_sys_call+0x198e/0x2390
Apr 23 13:06:05 pve1 kernel: ? do_syscall_64+0x15a/0x14e0
Apr 23 13:06:05 pve1 kernel: ? fire_user_return_notifiers+0x37/0x70
Apr 23 13:06:05 pve1 kernel: ? do_syscall_64+0x3c2/0x14e0
Apr 23 13:06:05 pve1 kernel: entry_SYSCALL_64_after_hwframe+0x76/0x7e
Apr 23 13:06:05 pve1 kernel: RIP: 0033:0x7c3f5832491b
Apr 23 13:06:05 pve1 kernel: Code: 00 48 89 44 24 18 31 c0 48 8d 44 24 60 c7 04 24 10 00 00 00 48 89 44 24 08 48 8d 44 24 20 48 89 44 24 10 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 ff ff 77 1c 48 8b 44 24 18 64 48 2b 04 25 28 00 00
Apr 23 13:06:05 pve1 kernel: RSP: 002b:00007c3f4e7f7b30 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
Apr 23 13:06:05 pve1 kernel: RAX: ffffffffffffffda RBX: 000000000000ae80 RCX: 00007c3f5832491b
Apr 23 13:06:05 pve1 kernel: RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000021
Apr 23 13:06:05 pve1 kernel: RBP: 00005a03af649300 R08: 0000000000000000 R09: 0000000000000000
Apr 23 13:06:05 pve1 kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
Apr 23 13:06:05 pve1 kernel: R13: 0000000000000004 R14: 0000000000000608 R15: 0000000000000001
Apr 23 13:06:05 pve1 kernel: </TASK>
Apr 23 13:06:05 pve1 kernel: ---[ end trace 0000000000000000 ]---I did some extensive testing of Windows Server, with VBS (Virtualization Based Security) enabled using the new Kernel (7.0) with the backported MBEC/GMET support and found the following:
Setup Guest:
On Intel (Intel(R) Xeon(R) Gold 6426Y):
I used the following setup:
At the end there is the
On AMD (AMD EPYC 7302P):
For AMD the
For more specifics about what these flags do, I can recommend looking at the QEMU Hyper-V Enlightenments Documentation.
Both these setups were stable for me running Cinebench R32.200 whilst migrating between two Host, AMD <-> AMD & Intel <-> Intel.
Other findings:
I also experienced BSOD (DPC Watchdog Violation & Clock Watchdog Timeout), especially when these two flags were set:
I also would not recommend using virtio-win 0.1.285 drivers these seem to lead to instability.
I narrowed it down to this new property (8):
AvailableSecurityProperties: 1,2,4,5,7,8
"8, If present, APIC virtualization is available."
see: Enable virtualization-based protection of code integrity
I was able to stabilize the system by explicitly disabling
Setup Guest:
- OS: Windows Server 2024H2 (Build 26100.1742)
- Drivers: virtio-win 0.1.271
- Platform Security Level: Secure Boot
- Virtualization Based Protection of Code Integrity: Enabled without lock
- Require UEFI Memory Attributes Table: Checked
On Intel (Intel(R) Xeon(R) Gold 6426Y):
I used the following setup:
Code:
-cpu 'host,+hv-evmcs,+hv-ipi,+hv-relaxed,+hv-runtime,hv-spinlocks=0x1fff,+hv-stimer,+hv-synic,+hv-time,+hv-tlbflush,+hv-tlbflush-ext,+hv-vapic,+hv-vpindex,+hv-xmm-input,+kvm_pv_eoi,+kvm_pv_unhalt,level=30,+vmx-mbec'At the end there is the
+vmx-mbec flag which is currently not in pve-qemu, but it can be cherry picked from this upstream commit: MBEC QEMUOn AMD (AMD EPYC 7302P):
Code:
-cpu 'host,+gmet,+hv-emsr-bitmap,+hv-ipi,+hv-relaxed,+hv-runtime,hv-spinlocks=0x1fff,+hv-stimer,+hv-synic,+hv-time,+hv-tlbflush,+hv-tlbflush-ext,+hv-vapic,+hv-vpindex,+hv-xmm-input,+kvm_pv_eoi,+kvm_pv_unhalt,level=30'For AMD the
+gmet flag has been added by cherry picking this commit: GMET QEMUFor more specifics about what these flags do, I can recommend looking at the QEMU Hyper-V Enlightenments Documentation.
Both these setups were stable for me running Cinebench R32.200 whilst migrating between two Host, AMD <-> AMD & Intel <-> Intel.
Other findings:
I also experienced BSOD (DPC Watchdog Violation & Clock Watchdog Timeout), especially when these two flags were set:
hv-stimer-direct,hv-tlbflush-directI also would not recommend using virtio-win 0.1.285 drivers these seem to lead to instability.
I narrowed it down to this new property (8):
AvailableSecurityProperties: 1,2,4,5,7,8
"8, If present, APIC virtualization is available."
see: Enable virtualization-based protection of code integrity
I was able to stabilize the system by explicitly disabling
-x2apic
Code:
-cpu 'host,+kvm_pv_eoi,+kvm_pv_unhalt,level=30,-x2apic'