Memory leak on Win10PE

[ferium]

Hello,
I'm building a project using iPXE and I've encountered a strange issue (although I'm not sure whether it's actually an iPXE bug).
I have an iPXE server and use it to boot VMs into both Windows 10 PE and Windows 11 PE.
When I boot Windows 10 PE, the VM's memory usage on the host increases significantly and remains high. In some cases, it grows enough for the VM to be killed by the host OOM killer.

VM configuration:

• 8 vCPUs
• 16 GB RAM
• Ballooning disabled

Proxmox VE configuration:

• Current version: 9.2.20 (fully updated)
• The same issue was also present on PVE 8.2.21

I upgraded the hypervisor from PVE 8.2.21 to 9.2.20 hoping that the issue would disappear, but the behavior remains the same.

A similar increase in host-side memory usage can also be observed with Windows 11 PE, but its behavior is different. After boot, memory usage is initially high, but gradually decreases over time.

With Windows 10 PE, however, the host-side memory usage stays consistently high and does not drop in the same way.

I also tried changing the guest OS type in the VM settings, but it had no effect.

Inside Windows 10 PE, Task Manager shows only about 871 MB of RAM in use. The boot.wim file is approximately 417 MB.

For comparison, with Windows 11 PE:

• boot.wim: approximately 564 MB
• RAM usage inside the guest: approximately 1.13 GB
• host-side memory usage is initially high, but gradually decreases after boot

So the main issue is that Windows 10 PE keeps consuming a disproportionately large amount of RAM on the host, even though the guest itself reports less than 1 GB of memory usage, while Windows 11 PE gradually releases or reduces its host-side memory usage over time.

I don't understand why booting Windows 10 PE through iPXE causes this persistent high host-side memory consumption, while Windows 11 PE behaves differently and its memory usage decreases over time.

Win10PE RAM usage


WIn11PE RAM Usage