Backup failed exit code 1

waranet

New Member
Oct 19, 2017
5
1
3
37
Hi,

Since I set up the automatic backup of one of my VMs, the backup has not been consistent : every two days it will fail with "exit code 1" and no more explanation. And some times (not very often), it will work for 3-4 days in a row and then, fail again.

My configuration :
- vm-100 : disk qcow2 of 500GB (only 350GB used)
- backup on external disk of 1TB
- backup is launched every night at 10pm

Error :
Code:
INFO: starting new backup job: vzdump 100 --quiet 1 --storage usbext --mode stop --node proxmox --mailnotification always --compress lzo
INFO: Starting Backup of VM 100 (qemu)
INFO: status = running
INFO: update VM 100: -lock backup
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: stopping vm
INFO: creating archive '/mnt/usbext/dump/vzdump-qemu-100-2018_01_09-22_00_01.vma.lzo'
INFO: starting kvm to execute backup task
Failed to create message: Input/output error
INFO: restarting vm
INFO: Running as unit 100.scope.
INFO: vm is online again after 18 seconds
ERROR: Backup of VM 100 failed - start failed: command '/usr/bin/systemd-run --scope --slice qemu --unit 100 -p 'KillMode=none' -p 'CPUShares=1000' /usr/bin/kvm -id 100 -chardev 'socket,id=qmp,path=/var/run/qemu-server/100.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -vnc unix:/var/run/qemu-server/100.vnc,x509,password -pidfile /var/run/qemu-server/100.pid -daemonize -smbios 'type=1,uuid=cf96afde-b787-4085-8667-7ad21e7ee966' -name ServeurW -smp '4,sockets=1,cores=4,maxcpus=4' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000' -vga std -no-hpet -cpu 'kvm64,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_relaxed,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce' -m 8192 -k fr -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:f1eed394ec66' -device 'ahci,id=ahci0,multifunction=on,bus=pci.0,addr=0x7' -drive 'file=/var/lib/vz/images/100/vm-100-disk-1.qcow2,if=none,id=drive-sata0,format=qcow2,cache=none,aio=native,detect-zeroes=on' -device 'ide-drive,bus=ahci0.0,drive=drive-sata0,id=sata0,bootindex=100' -drive 'file=/var/lib/vz/template/iso/SW_DVD9_NTRL_Win_Svr_Essentials_2012_R2_64Bit_French_-2_FPP_VL_OEM_X19-83577.ISO,if=none,id=drive-ide2,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -netdev 'type=tap,id=net0,ifname=tap100i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown' -device 'e1000,mac=66:62:64:39:64:31,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' -rtc 'driftfix=slew,base=localtime' -global 'kvm-pit.lost_tick_policy=discard' -S' failed: exit code 1
INFO: Backup job finished with errors
TASK ERROR: job errors

I've read all the 10 pages of search for backup failed but did not find any answer for this problem.

Exit code 1 is not very explicit for me.

I have found that the error might be from "Failed to create message: Input/output error" but cannot find how to correct this.

Do you any ideas for what happen here ?

What can I do to investigate further ?

Thanks for your help !
 
Last edited:
i have a similar inconvenient

somebody please help me, I can use restore backup in this vm

I created a VM CentOS 7, and Back-up using other storage NFS.

I can restore VM, but when I try to start ( using new ID vm) , the vm have problem with boot;

boot failed: not a bootable disk
iPXE -- Open Source Network Boot Firmware -- http://ipxe.org
Features: HTTP iSCSI DNS TFTP AoE FCoE TFTP COMBOOT ELF PXE PXEXT

Press Ctrl-B for the iPXE command line...
net0: A2:B0:75:C0:DA:CE using virtio-net on 0000:00:12.0 (open)
[Link:up, TX:0 TXE:0 RX:0 RXE:0]
Configuring ( net0 A2:B0:75:C0:DA:CE)........... No configuration Methods

This is config of y vm.


agent: 1
bootdisk: sata0
cores: 4
memory: 8200
name: my-server
net0: virtio=A2:B0:75:C0:DA:CE,bridge=vmbr208
numa: 0
onboot: 1
ostype: l26
sata0: Storage-RBD:vm-115-disk-1,cache=writethrough,size=100G
scsihw: virtio-scsi-pci
smbios1: uuid=01c7cde1-d3cf-4c37-99de-7358853ab9c9
sockets: 1



*******show configuration backup ****************

agent: 1
bootdisk: sata0
cores: 4
memory: 8200
name: my-server
net0: virtio=CE:64:8B:52:E9:63,bridge=vmbr208
numa: 0
onboot: 1
ostype: l26
sata0: ceph-vm:vm-105-disk-1,size=100G
scsihw: virtio-scsi-pci
smbios1: uuid=01c7cde1-d3cf-4c37-99de-7358853ab9c9
sockets: 1
#qmdump#map:sata0:drive-sata0:ceph-vm::


pveversion -v
proxmox-ve: 5.1-36 (running kernel: 4.13.13-5-pve)
pve-manager: 5.1-42 (running version: 5.1-42/724a6cb3)
pve-kernel-4.13.13-4-pve: 4.13.13-35
pve-kernel-4.13.13-2-pve: 4.13.13-33
pve-kernel-4.10.17-2-pve: 4.10.17-20
pve-kernel-4.13.13-5-pve: 4.13.13-36
libpve-http-server-perl: 2.0-8
lvm2: 2.02.168-pve6
corosync: 2.4.2-pve3
libqb0: 1.0.1-1
pve-cluster: 5.0-19
qemu-server: 5.0-19
pve-firmware: 2.0-3
libpve-common-perl: 5.0-25
libpve-guest-common-perl: 2.0-14
libpve-access-control: 5.0-7
libpve-storage-perl: 5.0-17
pve-libspice-server1: 0.12.8-3
vncterm: 1.5-3
pve-docs: 5.1-16
pve-qemu-kvm: 2.9.1-6
pve-container: 2.0-18
pve-firewall: 3.0-5
pve-ha-manager: 2.0-4
ksm-control-daemon: 1.2-2
glusterfs-client: 3.8.8-1
lxc-pve: 2.1.1-2
lxcfs: 2.0.8-1
criu: 2.11.1-1~bpo90
novnc-pve: 0.6-4
smartmontools: 6.5+svn4324-1
zfsutils-linux: 0.7.3-pve1~bpo9
ceph: 12.2.2-pve1
 
Hi,

I don't think that your problem is related to mine. I think you should open your own thread to have the help specific to your problem.

My problem is during backup.
 
  • Like
Reactions: Miguel Moreira
No one has any idea about this problem ?

I am kind of lost about this, everything I try do not work and my backup is still randomly working...
 
Hi,

I would like to add, that I have the same/a similar problem.

Please find below the error message I have here:

start failed: command '/usr/bin/kvm -id 400 -name Win10Pro -chardev 'socket,id=qmp,path=/var/run/qemu-server/400.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -pidfile /var/run/qemu-server/400.pid -daemonize -smbios 'type=1,uuid=290337e2-22bd-4041-ac29-1f3ec6251baf' -smp '2,sockets=1,cores=2,maxcpus=2' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vga qxl -vnc unix:/var/run/qemu-server/400.vnc,x509,password -no-hpet -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed' -m 2048 -object 'memory-backend-ram,id=ram-node0,size=2048M' -numa 'node,nodeid=0,cpus=0-1,memdev=ram-node0' -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -chardev 'socket,path=/var/run/ qemu-server/400.qga,server,nowait,id=qga0' -device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' -device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' -spice 'tls-port=61002,addr=127.0.0.1,tls-ciphers=HIGH,seamless-migration=on' -device 'virtio-serial,id=spice,bus=pci.0,addr=0x9' -chardev 'spicevmc,id=vdagent,name=vdagent' -device 'virtserialport,chardev=vdagent,name=com.redhat.spice.0' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:353f3e30457a' -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -device 'virtio-scsi-pci,id=scsihw0,bus=pci.0,addr=0x5' -drive 'file=/dev/zvol/pool_ZFS0/vm-400-disk-1,if=none,id=drive-scsi0,discard=on,format=raw,cache=none,aio=native,detect-zeroes=unmap' -device 'scsi-hd,bus=scsihw0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap400i0,script=/var/lib /qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=DE:CE:6D:9A:D8:36,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' -rtc 'driftfix=slew,base=localtime' -global 'kvm-pit.lost_tick_policy=discard' -S' failed: exit code 1

Can soone please hint in the right direction? There are also Linux VMs which are not backed up.

Interesting: All running VMs are backuped normally, I didnt test this further and dont know if its related.

Many thanks,
vdm
 
Hi,

I found an almost solution : the backup still failed but not as often as before (1 time a week now against 3/4 times before).

I edit /etc/vzdump.conf and change the size parameter by setting a larger value.

I read that we should be careful about the size we set, the size should be available (I suppose in RAM).

Hope this can help.
 
Thank you,

I tried with 3GB, 2GB and 1GB in vzdump.conf, but the backup does fail quite often.

In fact, 99% it fails only with powered off VMs.

The running VMs just get backed up fine.

Is this some kind of configuration mistake, or is it something like a bug somewhere?

The server has 32G on memory and a cache SSD with 200G.

Every help welcome.
 
Hi,

Sorry that this tip didn't improve the number of successfull backup.

Which mode do you use for the backup ? (snapshot, suspend or stop)

I use "Stop" for running VMs.

What I observe is that Linux VM (only command line) are always backup successfully but Windows VM backup is unstable.

Unfortunately, it seems that no one answers to this thread with a solution :(
 
I think I have similar symptoms to you. I have 7 LXC containers running, a few are as old as the LXC gui mods, the others are the last 6 months or so and they all get backed up every morning at 0200. However I noticed that some were failing, after looking into it the ones that are failing have the same error type of 1.

The only thing that seems similar with me is they are unprivileged CentOS containers.
 
Just ran the command from a commandline via SSH and it will give you more information about the failure:

Code:
/usr/bin/systemd-run --scope --slice qemu --unit 100 -p 'KillMode=none' -p 'CPUShares=1000' /usr/bin/kvm -id 100 -chardev 'socket,id=qmp,path=/var/run/qemu-server/100.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -vnc unix:/var/run/qemu-server/100.vnc,x509,password -pidfile /var/run/qemu-server/100.pid -daemonize -smbios 'type=1,uuid=cf96afde-b787-4085-8667-7ad21e7ee966' -name ServeurWaranet -smp '4,sockets=1,cores=4,maxcpus=4' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000' -vga std -no-hpet -cpu 'kvm64,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_relaxed,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce' -m 8192 -k fr -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:f1eed394ec66' -device 'ahci,id=ahci0,multifunction=on,bus=pci.0,addr=0x7' -drive 'file=/var/lib/vz/images/100/vm-100-disk-1.qcow2,if=none,id=drive-sata0,format=qcow2,cache=none,aio=native,detect-zeroes=on' -device 'ide-drive,bus=ahci0.0,drive=drive-sata0,id=sata0,bootindex=100' -drive 'file=/var/lib/vz/template/iso/SW_DVD9_NTRL_Win_Svr_Essentials_2012_R2_64Bit_French_-2_FPP_VL_OEM_X19-83577.ISO,if=none,id=drive-ide2,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -netdev 'type=tap,id=net0,ifname=tap100i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown' -device 'e1000,mac=66:62:64:39:64:31,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' -rtc 'driftfix=slew,base=localtime' -global 'kvm-pit.lost_tick_policy=discard' -S
 
I've tried manually backing up and looks like snapshot and suspend methods dont work but stop seems to.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!