Windows crashes under heavy I/O load?

Discussion in 'Proxmox VE: Installation and configuration' started by flypig, Oct 3, 2011.

  1. flypig

    flypig New Member

    Joined:
    Sep 1, 2011
    Messages:
    17
    Likes Received:
    0
    Hello,

    I have a test system on Proxmox 2.0 Beta.

    After moving a windows 2008R2 vm from promox 1.9 to 2.0 successfully.
    I notice the windows startup prompted for driver for the new virtio io controller, the old virtio io driver is no longer accepted, therefore I downloaded the latest virtio-win-0.1-15.iso from fedora website.

    Everything works smoothly with the new driver until I ran a IO benchmark / load test.
    The name of the programe is: CrystalDiskMark 3.0.1 x64 (crystalmark.info)

    hddtest1.png
    This screenshot is captured seconds before it crashes. I have ran the same test 10 times and all 10 test crashed at 4k random write test.
    (Hardware LSI RAID 0, Stripe size 64k)

    The first 2 test: sequencial and 512k random read/write went successfully.
    On the 3rd test: random 4k read test went successfully, however on the write test the blue screen of death came out.

    err1.png err2.png

    Below is the screencap of the cpu, io, mem utilization graph.

    proxmox1.png

    I then turn to my other test system (same RAID disk setup) which is using Proxmox 1.9, all the test went successfully without any problems.
    note that on 1.9 windows is using the older virtio io driver.


    It seems that the blue screen only came out under heavy IO write at 4K (since my stripe size is 64k, 4k will only write to 1 disk)
    Could it be issue with the latest virtio io driver?


    root@vps:~# pveversion -v
    pve-manager: 2.0-4 (pve-manager/2.0/52943683)
    running kernel: 2.6.32-6-pve
    proxmox-ve-2.6.32: 2.0-46
    pve-kernel-2.6.32-6-pve: 2.6.32-46
    lvm2: 2.02.86-1pve1
    clvm: 2.02.86-1pve1
    corosync-pve: 1.4.1-1
    openais-pve: 1.1.4-1
    libqb: 0.5.1-1
    redhat-cluster-pve: 3.1.7-1
    pve-cluster: 1.0-7
    qemu-server: 2.0-1
    pve-firmware: 1.0-13
    libpve-common-perl: 1.0-5
    libpve-access-control: 1.0-1
    libpve-storage-perl: 2.0-4
    vncterm: 1.0-2
    vzctl: 3.0.29-3pve1
    vzdump: 1.2.6-1
    vzprocps: 2.0.11-2
    vzquota: 3.0.12-3
    pve-qemu-kvm: 0.15.0-1
    ksm-control-daemon: 1.1-1
     
    #1 flypig, Oct 3, 2011
    Last edited: Oct 3, 2011
  2. tom

    tom Proxmox Staff Member
    Staff Member

    Joined:
    Aug 29, 2006
    Messages:
    13,567
    Likes Received:
    412
    1.9 and 2.0 uses more or less the same kernel and also the same version of KVM. can you test the latest virtio drivers also on 1.9?
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. flypig

    flypig New Member

    Joined:
    Sep 1, 2011
    Messages:
    17
    Likes Received:
    0
    Hi Tom,

    Thank you for your prompt reply,
    I guess I had figured out the problem.

    The blue screen of death dissapears if I set cache: no-cache !

    Earlier I was using "Write-Back" cache, thinking I could use this since my raid card supports write-back with bbu.

    cache.png

    After setting to no-cache, the 4k random wite test went perfectly without any problems.
    However notice that the preformance degraded much compare to the earlier test with write-back cache enabled.

    final.png

    The harddisk cache option seems to be a new feature in proxmox 2.0, under what circumstances should we use the write-back / write-thru cache?
    Should we set to no-cache if using raidcard that already have build in hardware cache?

    Thank you.
     
  4. tom

    tom Proxmox Staff Member
    Staff Member

    Joined:
    Aug 29, 2006
    Messages:
    13,567
    Likes Received:
    412
    yes, for 2.0 we put that option on the gui. on 1.9 you needed to configure this via CLI, in /etc/qemu-server/VMID.conf.

    i prefer cache=no.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  5. tom

    tom Proxmox Staff Member
    Staff Member

    Joined:
    Aug 29, 2006
    Messages:
    13,567
    Likes Received:
    412
    where are your VM disks? which format?
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  6. flypig

    flypig New Member

    Joined:
    Sep 1, 2011
    Messages:
    17
    Likes Received:
    0
    I am using RAW format.
     
  7. flypig

    flypig New Member

    Joined:
    Sep 1, 2011
    Messages:
    17
    Likes Received:
    0
    name: testw2k8
    ide2: none,media=cdrom
    vlan0: rtl8139=AE:87:67:C3:DF:27
    ostype: w2k8
    bootdisk: virtio0
    virtio0: local:103/vm-103-disk-1.raw
    memory: 2048
    sockets: 1
    boot: dc
    freeze: 0
    cpuunits: 1000
    acpi: 1
    kvm: 1
    onboot: 0
    cores: 2
    args: -cpu host

    cpu: host
    net0: virtio=AE:87:67:C3:DF:27,bridge=vmbr0
    net1: rtl8139=26:06:CA:6E:B8:CA,bridge=vmbr0
     
  8. tom

    tom Proxmox Staff Member
    Staff Member

    Joined:
    Aug 29, 2006
    Messages:
    13,567
    Likes Received:
    412
    I run cache=no on all my boxes for this config.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  9. flypig

    flypig New Member

    Joined:
    Sep 1, 2011
    Messages:
    17
    Likes Received:
    0
    ok, noted. shall set cache=no for all my vms.

    Thank you for your advise! =)
     
  10. e100

    e100 Active Member
    Proxmox Subscriber

    Joined:
    Nov 6, 2010
    Messages:
    1,235
    Likes Received:
    24
    I tested writethrough, writeback and none with windows 2008 R2 using virtIO drivers.

    Writethrough worked fine.
    none performed the best.

    Writeback BSOD under heavy IO.
    Writeback can perform better, if you like putting your data at risk, but it is unusable if it causes a BSOD.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice