Backup of Win2012R2 VM failed, but works, if VM is powered off

[Manuel2]

Hi,

I have a Windows 2012 R2 VM. At night during the automated backup this machine comes up with errors - almost every night (two successfull backups in the last two month):

5005: Feb 23 01:00:02 INFO: Starting Backup of VM 5005 (qemu)
5005: Feb 23 01:00:02 INFO: status = running
5005: Feb 23 01:00:03 INFO: update VM 5005: -lock backup
5005: Feb 23 01:00:03 INFO: backup mode: stop
5005: Feb 23 01:00:03 INFO: ionice priority: 7
5005: Feb 23 01:00:04 INFO: stopping vm
5005: Feb 23 01:00:12 INFO: creating archive '/srv/Sicherung_1/dump/vzdump-qemu-5005-2016_02_23-01_00_02.vma.lzo'
5005: Feb 23 01:00:12 INFO: starting kvm to execute backup task
5005: Feb 23 01:00:13 INFO: restarting vm
5005: Feb 23 01:00:14 INFO: vm is online again after 10 seconds
5005: Feb 23 01:00:14 ERROR: Backup of VM 5005 failed - start failed: command '/usr/bin/systemd-run --scope --slice qemu --unit 5005 -p 'KillMode=none' -p 'CPUShares=1000' /usr/bin/kvm -id 5005 -chardev 'socket,id=qmp,path=/var/run/qemu-server/5005.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -vnc unix:/var/run/qemu-server/5005.vnc,x509,password -pidfile /var/run/qemu-server/5005.pid -daemonize -smbios 'type=1,uuid=83f51c2c-f8fc-4950-9855-05fb85a73b34' -name SRV-KWP2 -smp '4,sockets=1,cores=4,maxcpus=4' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000' -vga std -no-hpet -cpu 'kvm64,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_relaxed,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce' -m 12288 -k de -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'virtio-balloon-pci
,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:b73774b6bef' -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2' -drive 'file=/srv/Sicherungsplatten/images/5005/vm-5005-disk-1.qcow2,if=none,id=drive-virtio2,cache=writeback,format=qcow2,aio=threads,detect-zeroes=on' -device 'virtio-blk-pci,drive=drive-virtio2,id=virtio2,bus=pci.0,addr=0xc' -drive 'file=/srv/Images_1/images/5005/vm-5005-disk-1.qcow2,if=none,id=drive-virtio0,cache=writeback,format=qcow2,aio=threads,detect-zeroes=on' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=101' -drive 'file=/srv/Images_1/images/5005/vm-5005-disk-2.qcow2,if=none,id=drive-virtio1,cache=writeback,format=qcow2,aio=threads,detect-zeroes=on' -device 'virtio-blk-pci,drive=drive-virtio1,id=virtio1,bus=pci.0,addr=0xb' -netdev 'type=tap,id=net0,ifname=tap5005i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-se
rver/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=66:38:36:39:63:35,netdev=net0,bus=pci.0,addr=0x12,id=net0' -rtc 'driftfix=slew,base=localtime' -global 'kvm-pit.lost_tick_policy=discard' -S' failed: exit code 1

When I look at the console at backup time, I can see, that the VM is shutting down and at the moment, when it's shutted down, the error occurs. I've tried to shut down the VM befor the backup begins, and the backup is working - but I don't want to shut it down by hand every night and bring it back online after the backup ;-)

I've taken a backup of the VM and restored it on another proxmox server, to see, if it's a VM or a server issue - and, I'm surprised - on the other server, the backup get's the same error - so, this should be an error on the VM - should be, because both servers have almost the same configuration.

The backups of the other VMs are working fine so long.

How can I find out, what the problem is?

 

[wolfgang]

Hi
what do you use as disk bus system and if virtio which driver?

 

[Manuel2]

Hi,

I've used the latest stable virtio (0.1.102) Drivers from Fedora, bus is virtio for 3 HDDs, SCSI Controller Type is LSI 53C895A - on creating the VM I did the same as described here: https://pve.proxmox.com/wiki/Windows_2012_guest_best_practices

 

[Manuel2]

I've tried older Versions of virtio, but the problem is still the same - I have other Windows 2012R2 Servers (with same virtio version) and the backup is successfull - so I looked for the difference and the only one I found was, that the one, where the backup fails, has virtual harddisks on two different storages - on the others, only one storage is used for the vdisks - so yesterday I moved the qcow2-file to the same storage, where the other qcow2 files are - last night the backup was successfull.

I'll have a look on the VM for the next days and tell here.

 

[Manuel2]

Nope, the successfull backup was only once - all next backups failed again.
And for now it's not the only backup that fails this often - I've restored an old 2003 Server, for which the auto-backup worked on older proxmox VEs perfekt, and there I've the same problem on actual VE - backup is failing with such a long failure-text like in my first post. So it's not a 2012R2 issue, but what could it be and how can I find out what's going wrong?

 

[wolfgang]

What sometimes help is to convert the vdisks to raw format.

 

[Manuel2]

Now I've converted the vdisks to raw on both VMs. I'll tell what the automatic backup will do.

 

[Manuel2]

The disks in raw format don't make it much better - only two of the last eight backups are successfully.
Is there an option to buy a proxmox support ticket or is this only available via subscription?

 

[wolfgang]

Is there an option to buy a proxmox support ticket or is this only available via subscription?

You can only update the subscription.