Windows Clients Freeze on Proxmox 3.4-6

[cm350]

Hi all,

A customer of ours has a Proxmox Server (offcourse ) and sometimes a Virtual Machine (he has 4), just freezes. It stops working. The only way to get it working again is to choose Stop - Start (reset doesn't work).

If you need more information, just ask. I don't know what information I should deliver to you.

 

[wolfgang]

Hi,

Please send Type of Vm (OS and Version).
and the output of

pveversion -v

 

[cm350]

Hi,

In fact it are 5 VM's
1 pfSense firewall (never freezed)
4 Windows 2012 R2 VM's (dc, SQL, SQL express & RDS)

output pveversion -v:
proxmox-ve-2.6.32: 3.4-156 (running kernel: 2.6.32-39-pve)
pve-manager: 3.4-6 (running version: 3.4-6/102d4547)
pve-kernel-2.6.32-39-pve: 2.6.32-156
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.7-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.10-2
pve-cluster: 3.0-17
qemu-server: 3.4-6
pve-firmware: 1.1-4
libpve-common-perl: 3.0-24
libpve-access-control: 3.0-16
libpve-storage-perl: 3.0-33
pve-libspice-server1: 0.12.4-3
vncterm: 1.1-8
vzctl: 4.0-1pve6
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 2.2-10
ksm-control-daemon: 1.1-1
glusterfs-client: 3.5.2-1

 

[tom]

Hi,

In fact it are 5 VM's
1 pfSense firewall (never freezed)
4 Windows 2012 R2 VM's (dc, SQL, SQL express & RDS)

...

1. what do you mean with "RDS" exactly and how do you use this in detail?

2. did you disable all CPU energy saving feature in your mainboard bios?

3. post an example VMID.config (qm config VMID)

 

[cm350]

1) RDS == Remote Desktop Service (users connect on this on to work in their applications)
2) Not specific, everything is like normal, don't know how the settings are
3)
bootdisk: virtio0
cores: 8
cpu: host
hotplug: cpu,disk,network,usb
ide2: none,media=cdrom
memory: 16384
name: Server name
net0: virtio=6A:7C:5C:3A2:BA,bridge=vmbr0
numa: 1
ostype: win8
smbios1: uuid=eaea9c97-de83-45b5-9363-d15a0ee5157a
sockets: 1
virtio0: local:104/vm-104-disk-1.qcow2,format=qcow2,cache=writethrough,size=200G
virtio1: local:104/vm-104-disk-2.qcow2,format=qcow2,cache=writethrough,size=100G

 

[MeyrNL]

Please use kernel of pve-kernel-3.10.0. Fixed my problems.

 

[tom]

Please use kernel of pve-kernel-3.10.0. Fixed my problems.

> apt-get install pve-kernel-3.10.0-11-pve
> reboot

 

[cm350]

Sorry for the long wait, you don't install a new kernel without testing and troubleshooting

But we did install it.
root@server:~# uname -a
Linux server3.10.0-11-pve #1 SMP Tue Jul 21 08:59:46 CEST 2015 x86_64 GNU/Linux
root@server:~# pveversion -v
proxmox-ve-2.6.32: 3.4-159 (running kernel: 3.10.0-11-pve)
pve-manager: 3.4-6 (running version: 3.4-6/102d4547)
pve-kernel-2.6.32-40-pve: 2.6.32-159
pve-kernel-2.6.32-39-pve: 2.6.32-157
pve-kernel-3.10.0-11-pve: 3.10.0-35
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.7-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.10-2
pve-cluster: 3.0-18
qemu-server: 3.4-6
pve-firmware: 1.1-4
libpve-common-perl: 3.0-24
libpve-access-control: 3.0-16
libpve-storage-perl: 3.0-33
pve-libspice-server1: 0.12.4-3
vncterm: 1.1-8
vzctl: 4.0-1pve6
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 2.2-10
ksm-control-daemon: 1.1-1
glusterfs-client: 3.5.2-1

But that did not solve our problems, it is much better, but it already did 2 freezes on a 6 day uptime. (with the previous kernel it was almost once a day)

Today we had the chance of having the "freeze" right when we were at the site. It keeps on working but just stops writing/reading things to the disk. Everything you open, does not work. you cannot log out / log in. But the mouse keeps on moving... so in fact it really is not a freeze but more a "istopworking".Now to be honest, a few hours before the "freeze". We adjusted the diskspace on the system disk (of the RDS) without a reboot-shutdown so maybe, because earlier versions of Proxmox really did not like that, that last "freeze" was just because it wanted to reboot

Anymore thoughts? Virtio drivers? Storage driver?

 

[RONIS]

What CPU do you have in server ?

 

[christophe]

Hi,

We had similar problems with RDS farms.
But it was just after an apt-get update ; apt-get dist-upgrade, and never had a crash before. It was in the PVE 3.2 days. Update done in the end of september 2014.

Then, same problems : ping OK, disk dead.

We removed each virtio driver (disk and NIC), and retrieved stability. But the problem did NOT necessarily came within Vrtio drivers. It was triggered by.

Then we upgraded to 3.10 kernel and qemu (genuine PVE)

There are some threads about that problem :

http://forum.proxmox.com/threads/20046-Some-Windows-guests-hanging-every-night

We are now in a stable situation, but can't remember here precise version numbers. Tomorrow!

Christophe.

 

[cm350]

Ronis: Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz, Model 63 Stepping 2

Christophe:
Hi,

That indeed is the problem! Glad it's not a single case

I would love to see your pveversion -v, so I will wait patiently!

What is the performance hit for NOT using the virtio devices?

I await your answer, then we can decide for a maintenance window.

Thanks in advance!

 

[christophe]

Server are IBM X3850 X5.

root@px1-cluster1:~# pveversion -v
proxmox-ve-2.6.32: 3.4-157 (running kernel: 3.10.0-10-pve)
pve-manager: 3.4-6 (running version: 3.4-6/102d4547)
pve-kernel-2.6.32-32-pve: 2.6.32-136
pve-kernel-2.6.32-39-pve: 2.6.32-157
pve-kernel-3.10.0-10-pve: 3.10.0-34
pve-kernel-2.6.32-33-pve: 2.6.32-138
pve-kernel-2.6.32-26-pve: 2.6.32-114
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.7-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.10-2
pve-cluster: 3.0-18
qemu-server: 3.4-6
pve-firmware: 1.1-4
libpve-common-perl: 3.0-24
libpve-access-control: 3.0-16
libpve-storage-perl: 3.0-33
pve-libspice-server1: 0.12.4-3
vncterm: 1.1-8
vzctl: 4.0-1pve6
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 2.2-10
ksm-control-daemon: 1.1-1
glusterfs-client: 3.5.2-1

And RDS servers all have ide disks and e1000 NIC. We tried to re-enable virtio drivers, not without problems it seems since we went back to ide + e1000. I can't remember now. No crash with these drivers.

Intersting : DFS servers (we have roaming profiles of course, with an RDS farm it is not really an option) are fully VirtIO and NEVER had a crash...

Hope this helps...

Christophe.

 

[christophe]

Running two farms as of today, each with 3 RDS servers, and 1 hundred thin clients by farm. Users seems happy.

Christophe.

 

[cm350]

Thanks christophe,

Today we were at another customer with the same setup (other physical server). He doesn't have this problem.

The only difference (exept the physical hardware) is the cache. With the working customer it is default (none) and with the freezing server it is writetrough. So my first thoughts are to disable the cache.

Our second plan will indeed be the disabling of the virtio drivers.

Even if that goes wrong we are going to try to put it to the test proxmox.

But first things first! I'll keep you posted.

 

[lofaz]

Hello, I know this is an old tread, but we just experience the same issue. Just woner if you experience that the windows guest stop logging to eventlogg as well? We find noting in any eventlog during the freeze, making it difficult to further understand what is happaneing.

 

[cm350]

Hello, I know this is an old tread, but we just experience the same issue. Just woner if you experience that the windows guest stop logging to eventlogg as well? We find noting in any eventlog during the freeze, making it difficult to further understand what is happaneing.

Indeed, it just stopped. We just disabled the caching option on the virtual drive. After that it never came back!

 

[christophe]

Hello, I know this is an old tread, but we just experience the same issue. Just woner if you experience that the windows guest stop logging to eventlogg as well? We find noting in any eventlog during the freeze, making it difficult to further understand what is happaneing.

Hi,

Nothing in any evenlog? No ID 129? viostor reset?
We experienced something very similar. It is not due to syslog. It is due to disk on which logs are written becomes unreachable. Logging din't stop anymore once we swapped to a networked syslog.

Christophe.