QEMU 11.0 available on pve-test and pve-no-subscription as of now

fiona

fiona

Proxmox Staff Member
Staff member
Aug 1, 2019
6,895
2,214
278
There is a new QEMU 11.0 package available in the pve-test and pve-no-subscription repositories for Proxmox VE 9.

After internally testing QEMU 11.0 for over two weeks and having this version available on the pve-test repository for over a week, we now (2026-05-13) made our QEMU 11.0 package available in the pve-no-subscription repository.

Version 11.0.0-2 of our QEMU package pve-qemu-kvm includes some important stable fixes that have been developed since the original QEMU 11.0 release.

Note: While some of our production workloads already use this version and run stable, we cannot test every possible hardware and configuration combination, so we recommend testing the upgrade before applying it to mission-critical setups.

To upgrade, make sure you have configured either the Proxmox VE No-Subscription repository or the Proxmox VE Test repository.

Either use the web-interface to refresh and then upgrade using the Node -> Updates panel, or use a console with the following standard apt commands:
Bash: 
apt update
apt full-upgrade

The output of the pveversion -v CLI command (or the web-interface's Node Summary -> Packages versions) should then include something like pve-qemu-kvm: 11.0.0-2.

Note, as with all QEMU updates: To run with the new QEMU version, a VM must either be completely restarted (shut it down and then start it again, or use the restart command via the CLI or web-interface) or, to avoid downtime, consider live-migrating to a host that has already been upgraded to the new QEMU package version.

While we have been successfully running our production and many test loads on this version for some time now, no software is bug-free, and often such issues are related to the specific setup. So if you encounter regressions that are definitely caused by installing the new QEMU version (and not some other change), please always include the affected VM configuration and some basic HW (e.g. CPU model) and memory details.

We welcome your feedback!

Known issues:
  • On a host with a recent Intel CPU, when using kernel 7.x, Windows server 2025 VMs with HVCI or RDS or Hyper-V with VM CPU type host or a recent Intel CPU model might fail to boot. This is caused by QEMU 11 automatically exposing the cet-ibt and cet-ssCPU flags, for which an issue in the KVM code in the Linux kernel exists.

    The issue should be mitigated with qemu-server>=9.1.12. If you cannot upgrade to that version for whatever reason, potential workarounds are:
    • using a different, less recent virtual CPU model - one that's close to your physical model, with optionally the nested-virt flag. Models that do not yet have the flag are SapphireRapids-v4, GraniteRapids-v3, SierraForest-v3, and ClearwaterForest (without bumped version)
    • adding custom arguments with qm set ID '-cpu host,level=30,-cet-ibt,-cet-ss'
    • using the 6.17 kernel
 
Last edited:
  • Like
Reactions: Sunilkumar, VictorSTS, rahman and 7 others
Code: 
[  285.467448] ------------[ cut here ]------------
[  285.467451] WARNING: ./arch/x86/kvm/mmu.h:227 at paging64_walk_addr_generic+0x7fc/0xa00 [kvm], CPU#20: CPU 11/KVM/9733
....

I am hitting this WARN_ON_ONCE that was committed here:

https://lkml.org/lkml/2025/9/19/1479

I believe it's because of the shadow stack (PFERR_SS_MASK) that my newer EPYC processor supports. We did not see it on another server that did not have shadow stack support.

Only started seeing this on the QEMU 11 upgrade. Coincidentally, CET virtualization was added with this release:

https://www.phoronix.com/news/QEMU-11.0-Released

Which corresponds to that warning and flag.

CET is an umbrella term coined by Intel for Shadow Stack and Indirect Branch Tracking. My processor only supports Shadow Stack and not IBT since it's AMD. To be clear, that particular code path is agnostic to whether its an Intel or AMD processor.
 
Last edited:
  • Like
Reactions: Sunilkumar
F

[SOLVED] Thread 'Windows Server 2025 Standard startet nach heutigem Update nicht mehr'

Hallo zusammen,

ich bin neu hier. Nach dem heutigen Update von Proxmox VE fährt mein Windows Server 2025 leider nicht mehr hoch und hängt in einer Endlosschleife fest. Windows Server 2022 startet dagegen ohne Probleme.

Ich habe das Verhalten auch mit einer Testmaschine nachvollziehen können: Nach dem Upgrade von Windows Server 2022 auf 2025 bootete auch diese Maschine nicht mehr. Während des Startvorgangs erscheint zunächst das Proxmox-Logo mit dem Ladekreis, kurze Zeit später lande ich jedoch im Bluescreen mit den erweiterten Startoptionen.

Ich habe bereits sämtliche Startoptionen...
 
  • Like
Reactions: Sunilkumar and fw1976
@fw1976 I should probably continue this discussion in the thread you started, but there are still some issues with Qemu11.

Boot loops and startup failures are still occurring on virtual machines with HVCI and RDS roles added.

*This also happens with Hyper-V roles, but since nested virtual environments aren’t meant for production use, I understand that reconfiguration is necessary in this case…

fiona

[SOLVED] Post in thread 'On a PVE system with the pve-qemu-kvm 11.0.0-1 package installed, virtual machines with the Hyper-V role added get stuck in a boot loop.'

Hi,
thank you for the report! My colleague @driley also ran into the issue and is looking into it.
  • Like

However, we should not cause issues with HVCI or RDS, which are not experimental features. It would be counterproductive for the addition of MBEC/GMET used by HVCI to cause problems in environments where HVCI is enabled.

In addition, there are other issues, such as those listed below. These issues remain unresolved.

U

Post in thread 'High VM-EXIT and Host CPU usage on idle with Windows Server 2025'

While reviewing the logs, I noticed ISCSI timeouts and entries like the one below, so it’s still not at a usable level.

I won’t investigate further; I’ll just quietly revert the changes and wrap up this check.

*I believe files such as kvm/vmx/nested.c are the ones affected by this fix, and I think the error is caused by this backport.
Besides, these logs weren't appearing at all before I updated the kernel.

Code: 
Apr 23 10:24:37 pve1 kernel: ------------[ cut here ]------------
Apr 23 10:24:37 pve1 kernel: WARNING: arch/x86/kvm/vmx/nested.c:4462 at vmx_check_nested_events+0x910/0x920...

fiona said:
Known issues:
None at the time of publishing.
Click to expand...

???

fiona

[SOLVED] Post in thread 'On a PVE system with the pve-qemu-kvm 11.0.0-1 package installed, virtual machines with the Hyper-V role added get stuck in a boot loop.'

Hi,
thank you for the report! My colleague @driley also ran into the issue and is looking into it.
  • Like
 
Last edited:
  • Like
Reactions: Sunilkumar
chrcoluk said:
How to get the 6.17 kernel for this?
Click to expand...
To list the installed kernels:
Code: 
proxmox-boot-tool kernel list
In my case, I had `6.17.13-8-pve`.
To pin it for next 1 boot to test out a kernel:
Code: 
proxmox-boot-tool kernel pin 6.17.13-8-pve --next-boot
Then, reboot.
To pin it so it stays the same over multiple reboots:
Code: 
proxmox-boot-tool kernel pin 6.17.13-8-pve
Then, reboot.

Change kernel version as needed or installed.
 
  • Like
Reactions: chrcoluk
fiona said:
Known issues:
  • On a host with a recent Intel CPU, when using kernel 7.x, Windows server 2025 VMs with HVCI or RDS or Hyper-V with VM CPU type host or a recent Intel CPU model might fail to boot. This is caused by QEMU 11 automatically exposing the cet-ibt and cet-ssCPU flags, for which an issue in the KVM code in the Linux kernel exists. [...]
Click to expand...

A patch: [1], thanks to @fiona, to "disable cet-ibt and cet-ss flags for modern Windows by default" is in qemu-server 9.1.12: [2], which already is available in pve-no-subscription.

[1] https://git.proxmox.com/?p=qemu-server.git;a=commit;h=140b30ac23dbdf0ee1c2b17d93aecc0beaeb6bd1
[2] https://git.proxmox.com/?p=qemu-server.git;a=commitdiff;h=9aa9d714bfc03ae0c1c76d703ecc232b36759c22
 
  • Like
Reactions: Sunilkumar, jsterr and fw1976
anutrix said:
To list the installed kernels:
Code: 
proxmox-boot-tool kernel list
In my case, I had `6.17.13-8-pve`.
To pin it for next 1 boot to test out a kernel:
Code: 
proxmox-boot-tool kernel pin 6.17.13-8-pve --next-boot
Then, reboot.
To pin it so it stays the same over multiple reboots:
Code: 
proxmox-boot-tool kernel pin 6.17.13-8-pve
Then, reboot.

Change kernel version as needed or installed.
Click to expand...
Thanks, grabbed it now and is pinned also.
 
  • Like
Reactions: Sunilkumar and anutrix
Neobin said:
A patch: [1], thanks to @fiona, to "disable cet-ibt and cet-ss flags for modern Windows by default" is in qemu-server 9.1.12: [2], which already is available in pve-no-subscription.

[1] https://git.proxmox.com/?p=qemu-server.git;a=commit;h=140b30ac23dbdf0ee1c2b17d93aecc0beaeb6bd1
[2] https://git.proxmox.com/?p=qemu-server.git;a=commitdiff;h=9aa9d714bfc03ae0c1c76d703ecc232b36759c22
Click to expand...

Is there any way to re-enable cet-ss? We have AMD processors in our shop and had no boot issues on Windows with VBS.

Besides reverting those Perl scripts that Fiona touched, I was wondering if there was a better way, perhaps through the VMs .conf file.

Never mind, apparently, even with Fiona's patch, shadow stacks are still exposed to the Windows guest when VBS is enabled. This was confirmed in the guests by verifying that "Kernel-mode Hardware-enforced Stack Protection" was reported in the guest OS, even though "-cet-ss" was present on the QEMU command line for that guest.
 
Last edited:
  • Like
Reactions: Sunilkumar
@Upstairs_Cycle384 Not sure why Windows still detects it, maybe querying the CPUID leaf directly still works or something. Regarding the other question, there is a custom CPU model editor coming with the next Proxmox VE 9.2 point release. There, you can define CPU models with arbitrary flags.
 
  • Like
Reactions: Sunilkumar, VictorSTS, jaminmc and 1 other person
We tested QEMU 11.0 (11.0.0-2) on a small staging cluster before considering rollout.

Observations so far:
- Linux guests (Ubuntu/Debian) are stable with no noticeable regressions
- Windows Server 2022 works fine under default settings
- Windows Server 2025 showed boot issues when HVCI was enabled, and CPU type was set to "host" (Intel-based nodes, kernel 7.x)

Switching to a slightly older CPU model (e.g., avoiding the latest Intel flags) resolved the boot loop temporarily in our case. Also confirmed that disabling CET-related flags (-cet-ibt, -cet-ss) avoids the issue, which aligns with the known KVM limitation mentioned.

We’re planning to validate this further with qemu-server >= 9.1.12 to see if the default mitigation fully resolves it.

One thing we’re still testing:
Has anyone observed any performance impact (especially CPU overhead or VM exits) after disabling CET flags on newer CPUs?
 
  • Like
Reactions: Johannes S
Sunilkumar said:
Has anyone observed any performance impact (especially CPU overhead or VM exits) after disabling CET flags on newer CPUs?
Click to expand...
When I enabled HVCI on a virtual machine running Windows Server 2025, the CPU benchmark results weren’t particularly bad, but I encountered issues such as being unable to use the Start menu, and the system would immediately hit 100% CPU usage and hang after just a few operations.

* We have also confirmed that if the 100% usage persists, a BSOD with error code 0x00000133 (DPC_WATCHDOG_VIOLATION) will occur. This issue is likely to reoccur immediately if you continue to interact with the system while it is frozen. I don't want to look at this problem anymore, so I'm not going to analyze the memory dump.

Although hardware acceleration was added to HVCI, the user experience was so terrible that we ultimately decided to stop using HVCI entirely.

* Winsat formal -restart isn't bad for checking RAM and CPU speeds, even if it doesn't test the GPU.
With the upgrade to Kernel 7, hardware acceleration has been enabled, resulting in improved performance figures. However, the actual user experience is extremely unpleasant.

It wasn't that bad before QEMU 11 came out, so I think the situation has worsened due to the combination with the kernel.

* I think it actually ran much more smoothly when I used the combination of Qemu 10 and Kernel 7.0.0.3 with `migratable=off` enabled.

U

Post in thread 'High VM-EXIT and Host CPU usage on idle with Windows Server 2025'

I haven’t tested it in a test environment yet, but I have confirmed that the flag is recognized on Windows by adding the kernel.

proxmox-kernel-7.0.0-2-pve-signed: 7.0.0-2

Code: 
PS C:\Users\admin> (Get-CimInstance -ClassName Win32_DeviceGuard -Namespace root\Microsoft\Windows\DeviceGuard).AvailableSecurityProperties
1
2
3
4
5
PS C:\Users\admin>

proxmox-kernel-7.0.0-3-pve-signed: 7.0.0-3

Code: 
PS C:\Users\admin> (Get-CimInstance -ClassName Win32_DeviceGuard -Namespace root\Microsoft\Windows\DeviceGuard).AvailableSecurityProperties
1
2
3
4
5
7 <- MBEC/GMET is available.
PS...
  • Like

Code: 
cpu: max,hidden=1,flags=+pdpe1gb
args: -cpu max,migratable=off
 
Last edited:
Upstairs_Cycle384 said:
Do you pass any hugepage kernel parameters on your Proxmox host?
Click to expand...

I've only set `default_hugepagesz=1G`, but whether I specify `hugepages: 1024` or not, the CPU usage remains at 100%.
The performance drop is so noticeable that even mouse movements feel sluggish—don’t you experience the same thing?

However, since we haven’t verified this with AMD, we don’t know if this applies only to Intel.

*By the way, we’re talking about delays here, but you’re not just asking about parameters out of the blue, are you?
If you're referring to “flags=+pdpe1gb,” that's just a remnant of a previous configuration.
 
Last edited:
You must log in or register to reply here.
Top Bottom
Back