Win7 Guest Host CPU ?

[wheaties82]

Recently completed new install of PVE 2.0 and had a question regarding CPU type. I have a Win7 x64 guest [KVM] that isn't able to use any cpu type but qemu64. In PVE 1.9 I could pass the 'args: -cpu host' into the config file and it would allow direct access to host cpu. When I try using the GUI selector to choose host, or when I manually update the config file my VM won't boot. It starts to load and then it shows BSOD.

I also tried installing a new VM using Win7 installer disc and leaving host cpu option set and I see same result -- BSOD.

Can someone shed light on this?

root@desmo:~# pveversion -v
pve-manager: 2.0-30 (pve-manager/2.0/af79261b)
running kernel: 2.6.32-7-pve
proxmox-ve-2.6.32: 2.0-60
pve-kernel-2.6.32-7-pve: 2.6.32-60
lvm2: 2.02.88-2pve1
clvm: 2.02.88-2pve1
corosync-pve: 1.4.1-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.8-3
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.7-1
pve-cluster: 1.0-22
qemu-server: 2.0-18
pve-firmware: 1.0-15
libpve-common-perl: 1.0-14
libpve-access-control: 1.0-12
libpve-storage-perl: 2.0-11
vncterm: 1.0-2
vzctl: 3.0.30-2pve1
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.0-3
ksm-control-daemon: 1.1-1

 

[kcalliauw]

Can you try using Nehalem as the cpu model? I've tried switching from qemu64 to host cpu model on one of my Windows 2008 R2 VM's with the same result. Nehalem worked.

To see the other supported models, use one of these commands on your host:

kvm -cpu ?model
kvm -cpu ?dump

With the second command you can see the flags that are exposed to the VM, so you can take your pick from that list according to what you need. I assume you have a reason for wanting anything other than qemu64, so specific CPU flags should be it. Keep in mind that migrations to other hosts require that the same cpu flags are available on the destination host CPU.

Kind regards,
Koen

 

[spirit]

we have added new cpu models to proxmox manager,so you can easily try them now.

(apt-get update && apt-get dist-upgrade to update)

 

[marsboer]

Hi

I have the same problem. -cpu host suddenly bluescreens Windows 7 in RC1.
I did some experimenting using the latest upgraded packages and on my CPU (Intel(R) Xeon(R) CPU E31220) choosing Nehalem as my CPU works, but Westmere doesn't.

 

[kcalliauw]

I remember reading a thread about SSH transfer speeds and an AES cipher to speed that up, but you need the right processor for that. Also a patched version of SSH and a library if I recall correctly.
If your CPU doesn't have aes it's normal that it BSOD's. My CPU's however do support AES (E5645) but also BSOD when booting with "host". Even with "host,-aes" it doesn't boot. Even though the spec sheet says that AES should be supported on that CPU model, I don't see the flag popping up in /proc/cpuinfo.

So perhaps you would need some extra's to get AES working on Debian, I'm not sure. It's also possible this has to be enabled in BIOS, but I can't check that here at this time.

Edit: another thought: perhaps these models needs changes to the Windows HAL?

/K

 

[marsboer]

After having systematically changed parameters in /etc/kvm/targ-x86_64.conf i found out which parameter make Nehalmen work and Westmere not.
It is actually none of the CPU-flags, it's the level parameter. If you set level to 2 instead of 11 for Westmere it boots fine. My physical Xeon E3 1220 CPU has a CPUID level of 13.

I then did some experimenting with different values and sure enough, levels up until 9 works, but 10 and further gives BSOD in Win 7. Seems like a bug to me!
Problems with level values of two numbers or something like that!

EDIT:
As a temporary fix I edit the configuration file for the Windows 7 machine:
-> nano /etc/pve/nodes/vserver/qemu-server/103.conf
and change

cpu: host

to

cpu: host,level=9

This works for me at least.

 

[spirit]

Hi,I'll try to contact qemu dev to see if they have the problem.

Note that Westmere model is not officially release in qemu main tree. (we take conf from redhat kvm version, maybe they have some other patches)


could you post

cat /proc/cpuinfo

from your proxmox host ?

 

[spirit]

Hi again, I have made a custom pve-qemu-kvm package with last redhat patches (from february)

could you test it ? (with westmere and host)

http://odisoweb1.odiso.net/pve-qemu-kvm_1.0-3_amd64.deb

 

[kcalliauw]

I had to enable AES-NI in the BIOS for it to be activated. Will test with Westmere and host now.



/K

 

[kcalliauw]

no luck, not with host or Westmere model. AES is now enabled so that's not the culprit. Also this was tested on a fully upgraded server (aptitude update && aptitude dist-upgrade)

Edit: just noticed something else: I get this on the console of the server:

kvm: 6236: cpu0 unhandled wrmsr: 0x188 data 0

Except for the PID this happens every time I boot the VM, the message appears at the exact same time of the BSOD.

root@node7:~# pveversion -vpve-manager: 2.0-33 (pve-manager/2.0/c598d9e1)
running kernel: 2.6.32-7-pve
proxmox-ve-2.6.32: 2.0-60
pve-kernel-2.6.32-6-pve: 2.6.32-55
pve-kernel-2.6.32-7-pve: 2.6.32-60
lvm2: 2.02.88-2pve1
clvm: 2.02.88-2pve1
corosync-pve: 1.4.1-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.8-3
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.7-1
pve-cluster: 1.0-23
qemu-server: 2.0-19
pve-firmware: 1.0-15
libpve-common-perl: 1.0-14
libpve-access-control: 1.0-15
libpve-storage-perl: 2.0-12
vncterm: 1.0-2
vzctl: 3.0.30-2pve1
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.0-3
ksm-control-daemon: 1.1-1

Searching around I found this thread:
http://forum.proxmox.com/threads/5046-Error-kvm-cpu0-unhandled-wrmsr-amp-unhandled-rdmsr

One suggestion was to use local storage instead of NFS, so to test I copied over the qcow2 file to local storage and tested like that, without any luck. So my best guess is that this bug hasn't been resolved yet.

/K

 

[marsboer]

Hi,I'll try to contact qemu dev to see if they have the problem.

Note that Westmere model is not officially release in qemu main tree. (we take conf from redhat kvm version, maybe they have some other patches)


could you post

cat /proc/cpuinfo

from your proxmox host ?

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Xeon(R) CPU E31220 @ 3.10GHz
stepping        : 7
cpu MHz         : 3093.131
cache size      : 8192 KB
physical id     : 0
siblings        : 4
core id         : 0
cpu cores       : 4
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 x2apic popcnt aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid
bogomips        : 6186.26
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

x4 (quad core)

The AES-part is not a problem, it is only CPUID levels from 10 and up. It would be logical that you need a physical CPU with a cpuid level above 9 for the problem to manifest itself, but not sure.

I will try your package.

EDIT: Installed your package and rebooted the entire server just to be sure. The issue is still the same. It is also important to note that this was working before so something has changed very recently (around RC1, but not 100% sure as I din't use the VM for several days before that)

 

[marsboer]

Did a apt-get update & apt-get upgrade now (did it about 10 hours ago too) and saw there was some new files.
Then shutdown -r now and...

Auch!



voila!

My first complete servercrash with proxmox VE, actually my first crash at all on this hardware.
Not necessarily related at all, but I am beginning to think that there may be a deeper issue with the support for my CPU in either kvm or the rather old kernel version as E3 1220 is a newish CPU model.

 

[wheaties82]

Can you try using Nehalem as the cpu model? I've tried switching from qemu64 to host cpu model on one of my Windows 2008 R2 VM's with the same result. Nehalem worked.

To see the other supported models, use one of these commands on your host:

kvm -cpu ?model
kvm -cpu ?dump

With the second command you can see the flags that are exposed to the VM, so you can take your pick from that list according to what you need. I assume you have a reason for wanting anything other than qemu64, so specific CPU flags should be it. Keep in mind that migrations to other hosts require that the same cpu flags are available on the destination host CPU.

Kind regards,
Koen

Thank you for the suggestion! I upgraded and used the Nehalem option and the VM booted.

In PVE1.9 using the -host flag allowed for a dramatic improvement in guest OS CPU horsepower. Using the qemu64 default I saw a big performance difference than what i had in PVE 1.9. I know it isn't very scientific but I used Handbrake as my benchmark -- encoding is pretty CPU intensive. All other things being equal except the cpu flag, I saw a 50-75% increase in the amount of time to complete the same encode job [PVE 1.9 -host vs PVE2.0 -qemu64]. That is why I was interested in applying the -host command. I also used the Windows 7 performance index (once again not very scientific) -- in PVE 1.9 with the -host flag enabled I was getting benchmarked at 7.5 and with qemu I was seeing 7.2.

After adding the Nehalam flag I am seeing 7.4 in Windows performance benchmark. Running a handbrake job with the Nehalem flag set I am seeing better Avg FPS then with qemu64, but still not up to far with what -host allowed in PVE 1.9.

As this is still RC and not production I anticipate future enhancements, fixes etc to kvm/qemu that will hopefully allow the guest OS to have better I/O with the host CPU.

Thanks for your help!

 

[marsboer]

Given that our issue is the same you can still use host, you just need to override the cpu level: host,level=2

 

[spirit]

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Xeon(R) CPU E31220 @ 3.10GHz
stepping        : 7
cpu MHz         : 3093.131
cache size      : 8192 KB
physical id     : 0
siblings        : 4
core id         : 0
cpu cores       : 4
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 x2apic popcnt aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid
bogomips        : 6186.26
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

x4 (quad core)

The AES-part is not a problem, it is only CPUID levels from 10 and up. It would be logical that you need a physical CPU with a cpuid level above 9 for the problem to manifest itself, but not sure.

I will try your package.

EDIT: Installed your package and rebooted the entire server just to be sure. The issue is still the same. It is also important to note that this was working before so something has changed very recently (around RC1, but not 100% sure as I din't use the VM for several days before that)

Ok, I have send a mail to redhat developper who have made the westmere cpudef.
So I'm waiting for a response from him.

Also maybe before -host doesn't pass cpulevel > 10, and now it does,and qemu doesn't support it yet.
Redhat does not support officially -host, but support cpudefs from target, so maybe they have good reason to lower cpulevel.
I see that Nehalem and other intels cpudefs have cpu level=2, maybe they have good reason to put them to 2 ?

 

[spirit]

using -host, is passing all cpu flags by default.
Normally you must have same performance with Westmere cpudefs.
Nehalem cpudefs is the same than Westmere withtout "aes" cpu flags support.
If aes is use by your encode job, that can explain why -host is faster. (but you normally will have same result with Westmere cpudefs).
Try to edit /etc/kvm/target-x86_64.conf , put level ="2 in Westmere definition and see if it's works.

 

[marsboer]

Ok, I have send a mail to redhat developper who have made the westmere cpudef.
So I'm waiting for a response from him.

Also maybe before -host doesn't pass cpulevel > 10, and now it does,and qemu doesn't support it yet.
Redhat does not support officially -host, but support cpudefs from target, so maybe they have good reason to lower cpulevel.
I see that Nehalem and other intels cpudefs have cpu level=2, maybe they have good reason to put them to 2 ?

It seems that qemu at least has some support as the Linux VMs see the >9 levels fine (the VMs are running Debian Wheezy, kernel 3.2.0-1)

using westmere for vm:

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 44
model name      : Westmere E56xx/L56xx/X56xx (Nehalem-C)
stepping        : 1
microcode       : 0x1
cpu MHz         : 3093.174
cache size      : 4096 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 2
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc arch_perfmon rep_good nopl pni ssse3 cx16 sse4_1 sse4_2 x2apic popcnt aes hypervisor lahf_lm
bogomips        : 6186.34
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

using host for vm:

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Xeon(R) CPU E31220 @ 3.10GHz
stepping        : 7
microcode       : 0x1
cpu MHz         : 3093.174
cache size      : 4096 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 2
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc arch_perfmon rep_good nopl pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 x2apic popcnt aes xsave avx hypervisor lahf_lm xsaveopt
bogomips        : 6186.34
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

Changing level to 2 in the westmere config is also a temporary workaround that works fine for the Windows 7 machines.

 

[spirit]

we have pushed westmere config level=2 in target-x86_64.conf, please apt-get update

I didn't have news from qemu devel for the -host bug, so choose westmere cpu model as workaround .

 

[dups]

Sorry to dig up a slightly older thread, but I'm running PVE 2.0 Final and I'm having pretty much this exact issue. I haven't pinpointed down to the levels yet, but using -host causes a BSOD in my Windows Server 2008R2 x64 VMs while the default -qemu64 does not. Has there been any updates or changes in this bug? We're using the Xeon X5680 Processors in our system. Thanks!

 

[spirit]

Hi, did you have tried Nehalem cpu model ?