High VM-EXIT and Host CPU usage on idle with Windows Server 2025

[Upstairs_Cycle384]

According to the readme [0], stable should kinda match the most recent Red Hat Enterprise Linux release. Seems like November was the last one!?

On Fedora People, the last release was from November 2025.

On Rocky Linux, the last release was from March 24th 2026.

The `release-drivers-versions.txt` in the Rocky Linux RPM outlines the various versions of the drivers shipped in their version of the ISO.

I skimmed through it. The latest is driver build is 0.1.297 on Rocky Linux. This is newer than 0.1.285 on Fedora People. Additionally, the tools are newer -- The qemu guest agent for Windows is from Feb 19th 2026.

 

[Upstairs_Cycle384]

I narrowed it down to this new property (8):
AvailableSecurityProperties: 1,2,4,5,7,8
"8, If present, APIC virtualization is available."
see: Enable virtualization-based protection of code integrity

I'm not sure how you guys got APIC virtualization enabled. on my EPYC Milan, I only have 1, 2, 5, and 7. Is anyone seeing 8 on Zen3 or above or is that just showing up for Intel users?

I've been trying to trigger APIC virtualization so I can generate a BSOD dump on my end. I've been trying a bunch of stuff between settings in the bios, kvm_amd parameters, and vm conf file changes.

My motherboard is a bit unique in that I do have AVIC forced enabled "force_avic=Y" if that's somehow a factor in preventing APIC virtualization from showing up on the guest

 

[prinskarnatie]

I'm in a similar situation with Zen5 9955HX. I also had to go the force_avic=Y route. I did add the +hv-avic cpu flag, but propery 8 not showing in the Windows VM .

 

[Upstairs_Cycle384]

I'm in a similar situation with Zen5 9955HX. I also had to go the force_avic=Y route. I did add the +hv-avic cpu flag, but propery 8 not showing in the Windows VM .

Can you post the output of:

dmesg | grep AMD-Vi

 

[prinskarnatie]

Can you post the output of:

dmesg | grep AMD-Vi

I'm on kernel 7.0.0-3-pve and stock pve-qemu (unpatched for GMET)

dmesg | grep AMD-Vi
AMD-Vi: Using global IVHD EFR:0x246577efa2254afa, EFR2:0x0
pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
AMD-Vi: Extended features (0x246577efa2254afa, 0x0): PPR NX GT [5] IA GA PC GA_vAPIC
AMD-Vi: Interrupt remapping enabled
AMD-Vi: Virtual APIC enabled

dmesg | grep -i kvm_amd
kvm_amd: TSC scaling supported
kvm_amd: Nested Virtualization enabled
kvm_amd: Nested Paging enabled
kvm_amd: LBR virtualization supported
kvm_amd: AVIC unsupported in CPUID but force enabled, your system might crash and burn
kvm_amd: AVIC enabled
kvm_amd: x2AVIC enabled (max 512 vCPUs)
kvm_amd: Virtual VMLOAD VMSAVE supported
kvm_amd: Virtual GIF supported
kvm_amd: Virtual NMI enabled

cpuid -l 0x8000000a -1
CPU:
SVM Secure Virtual Machine (0x8000000a/eax):
SvmRev: SVM revision = 0x1 (1)
SVM Secure Virtual Machine (0x8000000a/edx):
nested paging = true
LBR virtualization = true
SVM lock = true
NRIP save = true
MSR based TSC rate control = true
VMCB clean bits support = true
flush by ASID = true
decode assists = true
PMC virtualization = true
SSSE3/SSE5 opcode set disable = false
pause intercept filter = true
pause filter threshold = true
AVIC: AMD virtual interrupt controller = false
virtualized VMLOAD/VMSAVE = true
virtualized global interrupt flag (GIF) = true
GMET: guest mode execute trap = true
X2AVIC: virtualized X2APIC = true
supervisor shadow stack = true
guest Spec_ctl support = true
ROGPT: read-only guest page table = true
host MCE override = true
INVLPGB/TLBSYNC hyperv interc enable = false
VNMI: NMI virtualization = true
IBS virtualization = true
extended LVT AVIC access changes = true
guest VMCB addr check = true
bus lock threshold = true
idlt HLT intercept = true
EXITINFO1 non-interceptible shutdown = true
NASID: number of address space identifiers = 0x8000 (32768)

 

[ltctech]

Adding my findings from testing Windows Server 2025 / Windows 11 VBS guests on a mixed Intel Proxmox cluster.

Cluster CPUs involved in my testing:

• Cascade Lake: Intel Xeon Gold 6244 @ 3.60 GHz
• Ice Lake: Intel Xeon Gold 6348 @ 2.60 GHz
• Emerald Rapids: Intel Xeon Gold 6548Y+ @ 2.50 GHz

Idea is to keep live migration working while reducing the high VM-exit / host CPU usage and visible guest lag seen with Windows Server 2025 (and 11 Pro), especially when VBS is enabled. With the wrong CPU flags, the VM is not just inefficient at idle; it is visibly laggy.

Testing was done on Proxmox 9.2.2 with Linux 7.x. vmx-mbec was recently added to the kernel and makes VBS in a VM feasible. Without MBEC, VBS/HVCI performance is significantly worse.

Custom CPU models are defined here:

nano /etc/pve/virtual-guest/cpu-models.conf

This file only needs to be edited on one cluster member, since it is under /etc/pve and is replicated automatically to the other cluster nodes.

The custom model names below are used in VM configs with the custom- prefix, for example:

cpu: custom-icelake-win-vbs

Current test models:

cpu-model: cascadelake-lnx
reported-model Cascadelake-Server-v5
phys-bits 46
flags +md-clear;+rfds-no;+flush-l1d;+stibp;+tsc-adjust;+ss

cpu-model: cascadelake-win
reported-model Cascadelake-Server-v5
phys-bits 46
flags +md-clear;+rfds-no;+flush-l1d;+stibp;+tsc-adjust;+ss;+hv-apicv;+hv-frequencies;+hv-tlbflush;+hv-tlbflush-ext;+hv-xmm-input

cpu-model: cascadelake-win-vbs
reported-model Cascadelake-Server-v5
phys-bits 46
flags +md-clear;+rfds-no;+flush-l1d;+stibp;+tsc-adjust;+ss;+hv-apicv;+hv-frequencies;+hv-tlbflush;+hv-tlbflush-ext;+hv-xmm-input;+hv-evmcs;+hv-emsr-bitmap;+hv-tlbflush-direct;+vmx;+vmx-mbec;+hv-reenlightenment;+hv-stimer-direct

cpu-model: icelake-lnx
reported-model Icelake-Server-v6
phys-bits 46
flags -taa-no;+sbdr-ssdp-no;+psdp-no;+md-clear;+rdctl-no;+rfds-no;+flush-l1d;+stibp;+tsc-adjust;+ss;+dtes64;+pdcm;+pmu

cpu-model: icelake-win
reported-model Icelake-Server-v6
phys-bits 46
flags -taa-no;+sbdr-ssdp-no;+psdp-no;+md-clear;+rdctl-no;+rfds-no;+flush-l1d;+stibp;+tsc-adjust;+ss;+dtes64;+pdcm;+pmu;+hv-apicv;+hv-frequencies;+hv-tlbflush;+hv-tlbflush-ext;+hv-xmm-input

cpu-model: icelake-win-vbs
reported-model Icelake-Server-v6
phys-bits 46
flags -taa-no;+sbdr-ssdp-no;+psdp-no;+md-clear;+rdctl-no;+rfds-no;+flush-l1d;+stibp;+tsc-adjust;+ss;+dtes64;+pdcm;+pmu;+hv-apicv;+hv-frequencies;+hv-tlbflush;+hv-tlbflush-ext;+hv-xmm-input;+hv-evmcs;+hv-emsr-bitmap;+hv-tlbflush-direct;+vmx;+vmx-mbec;+hv-reenlightenment;+hv-stimer-direct

A few notes from testing:

1. For Windows VBS guests, +vmx-mbec is the most important accelerator. With VBS/HVCI enabled, MBEC makes a major difference to guest responsiveness.
   
2. For Windows VBS guests, +hv-evmcs also helped in my environment. Removing it made the VM slower.
   
3. +hv-reenlightenment requires pinning a virtual TSC frequency for live migration. The hosts in this migration pool have nominal TSC/base frequencies of 2.5 GHz, 2.6 GHz, and 3.6 GHz. I pinned the VM TSC to 2.5 GHz, the lowest value in the pool:
   
   

   qm set <vmid> --args "-global cpu.tsc-frequency=2500000000"

   
   This is not a CPU performance cap. It is the virtual invariant TSC rate exposed to the guest for migration compatibility.
   
4. +invtsc and +hv-stimer-direct appear to be mutually exclusive for VBS guests. When both are set, the VM hangs as the Windows bootloader loads if VBS is enabled. In my testing, +invtsc and +hv-stimer-direct were similar in performance, but +hv-stimer-direct felt subjectively faster, so I stopped using +invtsc for these VBS guests. Both require TSC pinning.
   
5. Migration from Emerald Rapids 6548Y+ to Cascade Lake 6244 failed when +pmu/+pdcm were exposed in the Cascade Lake ABI. The failure looked like this:
   
   

   Failed to set MSR 0x38f to 0x7000000ff

   
   My conclusion: do not expose +pmu/+pdcm in a Cascade Lake cpu model. PMU MSR state can break live migration.
   
6. Emerald Rapids 6548Y+ does not support +fb-clear in my testing, so I do not add +fb-clear to any custom CPU model that needs to migrate there.
   
7. For Ice Lake Server models, I had to keep -taa-no. Removing it caused some VMs to fail to boot. My understanding is that Intel partially disabledTSX behavior on Ice Lake, and exposing taa-no incorrectly can break things.
   
8. Proxmox still adds the normal Windows Hyper-V enlightenments automatically for Windows guests. For example, on one of my Windows VBS VMs, qm showcmd showed the expected base flags such as:
   
   

   hv_relaxed
   hv_reset
   hv_runtime
   hv_spinlocks=0x1fff
   hv_stimer
   hv_synic
   hv_time
   hv_vapic
   hv_vpindex
   hv_ipi

   
   The custom model above adds the extra VBS/nested-Hyper-V related pieces on top, such as:
   
   

   +hv-evmcs
   +hv-emsr-bitmap
   +hv-tlbflush-direct
   +hv-reenlightenment
   +hv-stimer-direct
   +vmx
   +vmx-mbec

So far, the best-performing VBS setup for my Intel hosts has been:

• keep +vmx-mbec
• keep +hv-evmcs
• Keep +hv-stimer-direct
• Keep +hv-reenlightenment
• Pin cpu.tsc-frequency
• Avoid using +invtsc together with +hv-stimer-direct
• Avoid +fb-clear for Emerald Rapids compatibility
• Avoid +pmu/+pdcm in Cascade Lake migration ABIs
• Keep -taa-no on the Ice Lake Server model

This does not eliminate all VBS overhead, but at least the VM is usable and still supports live migration. CPU vulnerabilities reported by Windows and Linux closely match the CPU generation being exposed.

Could the Proxmox team add CPU frequency pinning to the custom CPU config itself. As setting the pinned frequency per VM through the console is not great.