proxmox 4.1 backup fail error code 1

[dragonslayr]

Single host
Proxmox 4.1

Backing up to a usb drive.
All three vms stop to backup.
Other vms seem to backup ok, but one of them keeps getting the error below and I was hopeing you guys might have an idea where to look for the issue.

start failed: command '/usr/bin/systemd-run --scope --slice qemu --unit 101 -p 'KillMode=none' -p 'CPUShares=1000' /usr/bin/kvm -id 101 -chardev 'socket,id=qmp,path=/var/run/qemu-server/101.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -vnc unix:/var/run/qemu-server/101.vnc,x509,password -pidfile /var/run/qemu-server/101.pid -daemonize -smbios 'type=1,uuid=1f1b47ff-c46e-446a-b331-0dd1d702dbb1' -name rdp2008 -smp '2,sockets=1,cores=2,maxcpus=2' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000' -vga std -no-hpet -cpu 'kvm64,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_relaxed,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce' -m 20576 -k en-us -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'virtio-balloon-pci,id=ballo on0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:b6923e29a7e1' -drive 'file=/var/lib/vz/images/101/vm-101-disk-1.qcow2,if=none,id=drive-virtio0,cache=writeback,format=qcow2,aio=threads,detect-zeroes=on' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100' -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2' -netdev 'type=tap,id=net0,ifname=tap101i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=3A:32:32:65:63:35,netdev=net0,bus=pci.0,addr=0x12,id=net0' -rtc 'driftfix=slew,base=localtime' -global 'kvm-pit.lost_tick_policy=discard' -S' failed: exit code 1

 

[dietmar]

Please can you post the backup log for that VM?

 

[dragonslayr]

Please can you post the backup log for that VM?

I did a long smart test and a esfsck -vc /dev/sda1

I do not find errors on the drive.

INFO: Finished Backup of VM 100 (01:58:12)
INFO: Starting Backup of VM 101 (qemu)
INFO: status = running
INFO: update VM 101: -lock backup
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: stopping vm
INFO: creating archive '/mnt/backup_offsite/overman/dump/vzdump-qemu-101-2016_01_24-03_58_15.vma.lzo'
INFO: starting kvm to execute backup task
Failed to create message: Input/output error
INFO: restarting vm
INFO: Running as unit 101.scope.
INFO: vm is online again after 35 seconds
ERROR: Backup of VM 101 failed - start failed: command '/usr/bin/systemd-run --scope --slice qemu --unit 101 -p 'KillMode=none' -p 'CPUShares=1000' /usr/bin/kvm -id 101 -chardev 'socket,id=qmp,path=/var/run/qemu-server/101.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -vnc unix:/var/run/qemu-server/101.vnc,x509,password -pidfile /var/run/qemu-server/101.pid -daemonize -smbios 'type=1,uuid=1f1b47ff-c46e-446a-b331-0dd1d702dbb1' -name rdp2008 -smp '2,sockets=1,cores=2,maxcpus=2' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000' -vga std -no-hpet -cpu 'kvm64,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_relaxed,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce' -m 20576 -k en-us -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:b6923e29a7e1' -drive 'file=/var/lib/vz/images/101/vm-101-disk-1.qcow2,if=none,id=drive-virtio0,cache=writeback,format=qcow2,aio=threads,detect-zeroes=on' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100' -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2' -netdev 'type=tap,id=net0,ifname=tap101i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=3A:32:32:65:63:35,netdev=net0,bus=pci.0,addr=0x12,id=net0' -rtc 'driftfix=slew,base=localtime' -global 'kvm-pit.lost_tick_policy=discard' -S' failed: exit code 1
INFO: Starting Backup of VM 102 (qemu)

 

[gslongo]

Hi,

Same issue for me. The strange thing is it is not happening to all guest (randomly)


Search:
You do not have a valid subscription for this server. Please visit www.proxmox.com to get a list of available options.
INFO: starting new backup job: vzdump 102 --mode stop --compress lzo --storage backups --remove 0 --node pve-build
INFO: Starting Backup of VM 102 (qemu)
INFO: status = running
INFO: update VM 102: -lock backup
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: exclude disk 'virtio1' (backup=no)
libust[39517/39517]: Warning: HOME environment variable not set. Disabling LTTng-UST per-user tracing. (in setup_local_apps() at lttng-ust-comm.c:375)
INFO: stopping vm
INFO: creating archive '/data/pve/backups//dump/vzdump-qemu-102-2016_02_16-11_17_40.vma.lzo'
INFO: starting kvm to execute backup task
Failed to create message: Input/output error
INFO: restarting vm
INFO: Running as unit 102.scope.
INFO: libust[39768/39768]: Warning: HOME environment variable not set. Disabling LTTng-UST per-user tracing. (in setup_local_apps() at lttng-ust-comm.c:375)
INFO: libust[39771/39771]: Error: Error cancelling global ust listener thread: No such process (in lttng_ust_exit() at lttng-ust-comm.c:1592)
INFO: vm is online again after 58 seconds
ERROR: Backup of VM 102 failed - start failed: command '/usr/bin/systemd-run --scope --slice qemu --unit 102 -p 'KillMode=none' -p 'CPUShares=1000' /usr/bin/kvm -id 102 -chardev 'socket,id=qmp,path=/var/run/qemu-server/102.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -vnc unix:/var/run/qemu-server/102.vnc,x509,password -pidfile /var/run/qemu-server/102.pid -daemonize -smbios 'type=1,uuid=8774c839-81f7-4f0a-8970-0d78f6a3b700' -name linux-libc211-x86-64-ifg482-trunk -smp '4,sockets=1,cores=4,maxcpus=4' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000' -vga cirrus -cpu kvm64,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce -m 14000 -object 'memory-backend-ram,size=14000M,id=ram-node0' -numa 'node,nodeid=0,cpus=0-3,memdev=ram-node0' -k fr-be -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:7aff55f753dd' -drive 'file=/data/pve/vg15k.2/fs/images/102/vm-102-disk-2.qcow2,if=none,id=drive-virtio1,format=qcow2,cache=writeback,aio=threads,detect-zeroes=on' -device 'virtio-blk-pci,drive=drive-virtio1,id=virtio1,bus=pci.0,addr=0xb' -drive 'file=/data/pve/vg15k.2/fs/images/102/vm-102-disk-1.raw,if=none,id=drive-virtio0,format=raw,cache=writeback,discard=on,aio=threads,detect-zeroes=unmap' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=201' -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap102i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=00:50:56:2C:87:A4,netdev=net0,bus=pci.0,addr=0x12,id=net0' -S' failed: exit code 1
INFO: Backup job finished with errors
TASK ERROR: job errors


Thanks !

 

[dragonslayr]

I would like to tell you I found an answer to this, but I did not. Happens on multiple Proxmox host. All backing up to usb drive in my case. All doing "shutdown" upon backup. As a temp workaround, I also setup periodic snapshots until a better answer arrives..
However, you seem to have the problem even though your backing up to the local drive and NOT a usb drive. Is that correct?
Like you, it is not always the same vm that it happens to..

 

[gslongo]

However, you seem to have the problem even though your backing up to the local drive and NOT a usb drive. Is that correct?

Correct