Nested Hyper-V (Windows 10) with KVM?

enoch85

enoch85

Member
Proxmox Subscriber
Nov 16, 2022
105
12
18
Sweden
Hi,

Been debugging why my Windows 10 machine (VM) is so darn slow. Found this, and it solved the issue. So far so good - until I tried to run a VM in Hyper-V on that same Windows 10 guest.

Turns out you need to use host and not any of the other CPU types. It was absolutely flying on x86-64-v2-[AES] and it's a shame that host is needed since it makes my Windows guest like mud.

So, why isn't it possible? Saw this thread about Qemu, but maybe it's unrelated?
 
What flags are you using? I haven't had the same problems since my early reply when we discussed using the other OS Type. See the first page of that thread.

Have you tried playing around with the CPU type to match your physical CPU type? For example Ive had success with host and Penryn configurations in VMs with working nested virtualization. I have not tried HyperV nesting, though.

This VM has been retired for some time... I'm not sure if setting the CPU type to Penryn while also passing cpu arg host had an impact other than overriding the cpu type to host. But at the time of the discussion you linked these settings seemed to work and nested virtualization was working.

Code: 
args: -cpu host,kvm=off,+kvm_pv_unhalt,+kvm_pv_eoi,+hypervisor,+invtsc,+aes
cpu: Penryn

enoch85 said:
Hi,

Been debugging why my Windows 10 machine (VM) is so darn slow. Found this, and it solved the issue. So far so good - until I tried to run a VM in Hyper-V on that same Windows 10 guest.

Turns out you need to use host and not any of the other CPU types. It was absolutely flying on x86-64-v2-[AES] and it's a shame that host is needed since it makes my Windows guest like mud.

So, why isn't it possible? Saw this thread about Qemu, but maybe it's unrelated?
Click to expand...
 
Republicus said:
What flags are you using? I haven't had the same problems since my early reply when we discussed using the other OS Type. See the first page of that thread.

Have you tried playing around with the CPU type to match your physical CPU type? For example Ive had success with host and Penryn configurations in VMs with working nested virtualization. I have not tried HyperV nesting, though.

This VM has been retired for some time... I'm not sure if setting the CPU type to Penryn while also passing cpu arg host had an impact other than overriding the cpu type to host. But at the time of the discussion you linked these settings seemed to work and nested virtualization was working.

Code: 
args: -cpu host,kvm=off,+kvm_pv_unhalt,+kvm_pv_eoi,+hypervisor,+invtsc,+aes
cpu: Penryn
Click to expand...
Thanks for your reply!

This is what it looks like when it's slow but Nesting is working:

1691608610727.png

The exact same setup with x86-64-v2-[AES] makes the VM fly, but Nesting doesn't work. I want Nesting with x86-64-v2-[AES]. Or at least something that's not "system interupting" me 99%.

 
What reply do you get from (?)

(intel cpu):
# cat /sys/module/kvm_intel/parameters/nested

(amd cpu):
# cat /sys/module/kvm_amd/parameters/nested
 
Was thinking about the flags you mentioned, would this be the correct config?

Note to self:

Code: 
cpu: host,flags=+kvm=off,+kvm_pv_unhalt,+kvm_pv_eoi,+hypervisor,+invtsc,+aes


1691611534392.png
 
Last edited:
I just enabled cpu type host and have no flags set. My VM boots fine and HyperV is now installed. I will install a VM in HyperV to verify.
I think something else is amiss that is causing your VM to be so slow. My CPUs are v2 of the same E5 series.
What is your problem in HyperV?
 
Republicus said:
I just enabled cpu type host and have no flags set. My VM boots fine and HyperV is now installed. I will install a VM in HyperV to verify.
I think something else is amiss that is causing your VM to be so slow. My CPUs are v2 of the same E5 series.
What is your problem in HyperV?
Click to expand...
It's not a problem with Hyper-V. Problem is that the whole Windows 10 host becomes suuuper sluggish when running on host instead of x86-64-v2-[AES] - and then the main problem - that you can't run Nested virtualisation on x86-64-v2-[AES]. My question in the fist post was: why? Is it a limitation of Proxmox, or KVM in general?
enoch85 said:
It was absolutely flying on x86-64-v2-[AES] and it's a shame that host is needed since it makes my Windows guest like mud.
Click to expand...
 
So I'm not seeing the sluggishness you are reporting in the Windows HyperV "host" VM or the nested guest VM inside it. My test is on a PVE 7.4-13 node.
 
Strange. Running 8.0.4. Virtio guest is 229.

But still, questions remains - why you can't run nested when running with the x86-64-v2-[AES] CPU? Would it be possible with x86-64-v4, and what's the difference? Yes, I can read, but just if you know some quick facts? :)
 
Can you try these changes to your VM conf file, add args line and adjust CPU type to Broadwell:

Code: 
args: -cpu Broadwell,vmx=on
cpu: Broadwell

PS I'm not qualified to answer your question concerning the kvm64 or other CPU types
 
Last edited:
vmx is the flags needed for nested virtualization.

args: -cpu kvm64,vmx=on

This does report vmx flags once added (and absent without that argument in conf) but you may need to install Win10 from scratch with that CPU change. My VM wanted to boot loop and repair with that CPU change.

But the Broadwell changes I suggested should also have worked. My VM starts HyperV VM and Task Manager reports virtualization support:
If you can test kvm64,vmx=on (possibly from new install if needed) of course that seems to be the answer you wanted... im just not sure why Broadwell didn't work when it aught to have.
1691616890185.png
 
Oh well, I'm quite happy with the result now after the change. It's not super slow anymore, but not super fast either. It works, takes a few seconds to open up explorer, but not minutes (as it was before).

I think we solved it, for now.

Thanks!
 
Youve sparked my interest so I am installing a fresh Win 10 vm with the args added to a default kvm64 cpu
args: -cpu kvm64,vmx=on

I'll let you know what I find out.
Because I'm not on PVE 8 I cant tell you if kvm64 = x86-64-v2-[AES]
or if it aught to be args: -cpu x86-64-v2-[AES],vmx=on
 
Last edited:
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back