VirtIO Win2003 warnings, errors, reboots

[fortechitsolutions]

Hi, I'm having some issues on a ProxVE host deployed a few months ago, onto which I've migrated some windows 2003 servers. Some are p2v migrated; others are 'clean install'. I'm using the latest Virtio driver ISO (drivers signed by redhat) and have paravirt SCSI/disk and paravirt gig nic in the systems.

The problem: Most of the windows systems are logging sporatic errors to event log - typically 'warnings' rather than 'errors'. Frequency varies from .. a few warnings per day -- to more often (dozens of time per day).

I've had a few cases now this week where they windows VM reboots spontaneously. I have events - errors - logged; a sample is pasted below; it appears to relate to the VirtIO based drive.

I'm curious if anyone else has seen this kind of behaviour or has ideas ?

I flipped the system back to IDE based drive, but found disk performance noticably slower - so moved back to VirtIO. I think the problem disappeared when using IDE based controller though.

Note, the system was proxVE 1.8 when originally deployed; and has since been updated to proxve 1.9

Any thoughs are certaily appreciated.

Thanks,


Tim


---paste---

[Header part]
Record number       : 96
Time generated      : Thu Oct 20 02:41:36 2011 local (Thu Oct 20 05:41:36 2011 GMT)
Event ID            : 2147745921 (no description)
Event type          : 2 (Warning)
Event category      : 0 (None)


[Body part]
Event source        : viostor
Computer            : WINWEB
Sid                 : N/A


[Strings part]
String              : \Device\RaidPort0


--
[Header part]
Record number       : 97
Time generated      : Thu Oct 20 03:21:47 2011 local (Thu Oct 20 06:21:47 2011 GMT)
Event ID            : 3221487627 (no description)
Event type          : 1 (Error)
Event category      : 0 (None)


[Body part]
Event source        : Disk
Computer            : WINWEB
Sid                 : N/A


[Strings part]
String              : \Device\Harddisk0

 

[alain]

Hi,

Could you post exactly which Proxmox-ve version you use (pveversion -v) ? I run myself several 2003 R2 (and 2008 R2) with virtio drivers, and have no problem...
Could you post also at least one of your VM configuration (in /etc/qemu-server), to see which configuration they have ?

Alain

 

[fortechitsolutions]

Hi Alain, thanks for the quick followup. Info is below as per your request.

As I paste this I find it odd that the conf file below indicates bootdisk as ide0, which no longer exists. (This VM was clean installed Win2003 to a base KVM VM with IDE drive, first, and then migrated to Virtio after). It did boot - clearly - but I wonder if this setting is 'less than optimal'.

I'm also curious - Alain - do you have to use any custom parameters (ie, not default config) with the virtIO NIC ? I ask because on another project I worked on last month, with Win7Pro as the gues OS, I found the VirtIO nic had issues with default config (system crashes and reboots); and another post in this forum to my query suggested disable-all-features (offloads,etc) in the NIC HW config of the OS. This seems different from your experience, hence my curiosity

Thanks,.

-Tim

prox1:/etc/qemu-server# more 110.conf 
name: winweb2
ide2: none,media=cdrom
vlan0: virtio=EA:BD:26:E4:F2:EF
bootdisk: ide0
ostype: w2k3
memory: 2048
onboot: 1
sockets: 2
description: Clean install Win2003 Web edition Aug-8-11 TDC
cores: 2
virtio0: local:110/vm-110-disk-1.raw
vlan1: virtio=12:3A:DA:05:73:EA


prox1:/etc/qemu-server# pveversion -v
pve-manager: 1.8-18 (pve-manager/1.8/6070)
running kernel: 2.6.35-1-pve
proxmox-ve-2.6.35: 1.8-11
pve-kernel-2.6.35-1-pve: 2.6.35-11
pve-kernel-2.6.18-2-pve: 2.6.18-5
qemu-server: 1.1-30
pve-firmware: 1.0-11
libpve-storage-perl: 1.0-17
vncterm: 0.9-2
vzctl: 3.0.28-1pve1
vzdump: 1.2-14
vzprocps: 2.0.11-2
vzquota: 3.0.11-1
pve-qemu-kvm: 0.14.1-1
ksm-control-daemon: 1.0-6

prox1:/etc/qemu-server#

 

[linum]

Hmm, this sound similar to my problems here: http://forum.proxmox.com/threads/7318-WinXP-virtio-proxmox-1.9-gt-crash The erratic behaviour is not exactly the same I get here but close enough I think.http://forum.proxmox.com/threads/7318-WinXP-virtio-proxmox-1.9-gt-crash

You say you're now using promox 1.9 but your pveversion show older version than I would expect with version 1.9.

I think there's something wrong with virtio drivers and the latest kvm release.

 

[alain]

Hi Fortechit,

Sorry for the delay of my answer. As noted by linum, you say you are using 1.9, but the version shows 1.8. Is it an error ?
And as you said, boot disk seems to be IDE, not virtio. Did you verify ?
Also, I note that you use kernel 2.6.35, it is also my case.

I had a look at several configurations for my w2k3 VMs, and saw that I was overly cautious, stayng with IDE for boot disk (C), and using only virtio for other partitions.

Here is an example :

# cat 119.conf
name: pc-imps
ide2: local:iso/virtio-win-1.1.16.iso,media=cdrom
vlan0: virtio=FE:A8:D9:8A:07:C1
bootdisk: ide0
ostype: w2k3
memory: 512
onboot: 1
sockets: 1
boot: cad
freeze: 0
cpuunits: 1000
acpi: 1
kvm: 1
virtio0: local:119/vm-119-disk-2.raw
virtio1: local:119/vm-119-disk-3.raw
virtio2: local:119/vm-119-disk-4.raw
ide0: local:119/vm-119-disk-1.raw

Here is my pve version :

r# pveversion -v
pve-manager: 1.9-24 (pve-manager/1.9/6542)
running kernel: 2.6.35-2-pve
proxmox-ve-2.6.35: 1.8-13
pve-kernel-2.6.18-2-pve: 2.6.18-5
pve-kernel-2.6.35-2-pve: 2.6.35-13
qemu-server: 1.1-32
pve-firmware: 1.0-13
libpve-storage-perl: 1.0-19
vncterm: 0.9-2
vzctl: 3.0.28-1pve5
vzdump: 1.2-15
vzprocps: 2.0.11-2
vzquota: 3.0.11-1
pve-qemu-kvm: 0.15.0-1
ksm-control-daemon: 1.0-6

So, I am using PVE 1.9 with 2.6.35-2 kernel. And even if I use virtio for NIC (here the drivers version is virtio-win-1.1.16.iso, not signed by RedHat, but I also use virtio-win-0.1-mm34.iso for other VMs, which are signed by RedHat), and virtio for other partition, I don't experience crashes. I could try to use virtio also for boot disk...

I also notice that you use 2048 Mo Ram for your VM. It is a lot for W2K3. I am only using 512 Mo. How many Ram has your host ? Does it use KSM to overcommit Ram ? (I don't)

 

[fortechitsolutions]

Hi, just a few brief followups on this:

- you are right, these hosts were ProxmoxVE version 1.8 at this site - (my mistake in stating otherwise in this thread / original post). However: I've just updated them to version 1.9 yesterday.
- I dropped the 'most problematic win2003 VM' down to a significantly older driver for VirtIO. It got a lot of workout on Friday using this new config; no errors yet. change was as follows:

Before - newer viostor.sys driver file - 61.61.105.68000 built by WinDDK (from driver ISO, virtio-win-0.1-mm34.iso)
After - older viostor.sys driver file - 5.00 buit by WinDDK (from driver ISO, virtio-win-1.1.11-0.iso)

- performance is not apparently affected with this older driver (ie, VirtIO disk performance is still better than non-virtIO / IDE disk within the VM)

- also to comment on your questions:
(a) 2gb ram is not a huge amount IMHO for Win2003 VMs. Many clients I find still like to use Win2003, since it works for them, and Ram is cheap in servers these days so no reason to starve them for Ram. Hardware ProxVE host in this case is a single-socket quadcore HT Xeon CPU, 8 "CPUs" presented to Host in total; and with 16gb ram -- this is a pretty typical config for an 'econobox server' these days IMHO.
(b) I am using the KSM module with KVM / to allow for memory reclamation and better overcommit; but right now it isn't really needed since the physical host has more ram available than I've allocated to VMs so far.

I will post back to this thread later this week regarding the outcome of tests using the old VirtIO driver in this environment.


Tim

 

[linum]

Before - newer viostor.sys driver file - 61.61.105.68000 built by WinDDK (from driver ISO, virtio-win-0.1-mm34.iso)
After - older viostor.sys driver file - 5.00 buit by WinDDK (from driver ISO, virtio-win-1.1.11-0.iso)

Do you use the virtio nic drivers too? You didn't mentioned that in your message but I assume you don't. It may be worth to just try to installed the virtio nic drivers and see if you get wired behaviour with these drivers too. Since my trouble is related to the virtio nic drivers.

 

[fortechitsolutions]

Hi, I am not using VirtIO nic presently. Trying to have things run as stable as possible. (And having problems getting that).

Update for the thread: I've flipped over a second Win2003 VM from the 'latest' to the 'older' Virtio driver. Unfortunately this one seems to still be throwing errors. So the situation is more confused than ever.

Tim