Hello,
I have a high memory usage during backup on two of my servers (other are fines).
Sometimes it's lead to VM crash
This is standalone install with soft raid (i know it's not recommended), one use ZFS and the other ext4.
Backup log
State of memory after a failed backug wich lead to vm crash.
On Munin Graph the memory go high as soon as backup start
Any advices?
I have a high memory usage during backup on two of my servers (other are fines).
Sometimes it's lead to VM crash
This is standalone install with soft raid (i know it's not recommended), one use ZFS and the other ext4.
Backup log
Code:
INFO: transferred 17179 MB in 36 seconds (477 MB/s)
INFO: archive file size: 3.92GB
INFO: delete old backup '/var/lib/vz/dump/vzdump-qemu-104-2017_07_29-03_04_34.vma.lzo'
INFO: Finished Backup of VM 104 (00:00:37)
INFO: Starting Backup of VM 105 (qemu)
INFO: status = stopped
INFO: update VM 105: -lock backup
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: VM Name: qm-105
INFO: include disk 'virtio0' 'local:105/vm-105-disk-1.qcow2' 8G
INFO: creating archive '/var/lib/vz/dump/vzdump-qemu-105-2017_08_01-03_05_10.vma.lzo'
INFO: starting kvm to execute backup task
kvm: cannot set up guest memory 'pc.ram': Cannot allocate memory
ERROR: Backup of VM 105 failed - start failed: command '/usr/bin/kvm -id 105 -chardev 'socket,id=qmp,path=/var/run/qemu-server/105.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -pidfile /var/run/qemu-server/105.pid -daemonize -smbios 'type=1,uuid=205a6221-d6e2-4ea3-a0b9-e6d45c15953b' -name ase-mono01 -smp '2,sockets=1,cores=2,maxcpus=2' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vga std -vnc unix:/var/run/qemu-server/105.vnc,x509,password -cpu kvm64,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce -m 1024 -k fr -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:47185d13f4a' -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2' -drive 'file=/var/lib/vz/images/105/vm-105-disk-1.qcow2,if=none,id=drive-virtio0,cache=writeback,format=qcow2,aio=threads,detect-zeroes=on' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap105i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=02:00:00:51:f1:b9,netdev=net0,bus=pci.0,addr=0x12,id=net0' -S' failed: exit code 1
INFO: Backup job finished with errors
TASK ERROR: job errors
State of memory after a failed backug wich lead to vm crash.
Code:
root@proxmox:~# qm start 105
kvm: cannot set up guest memory 'pc.ram': Cannot allocate memory
start failed: command '/usr/bin/kvm -id 105 -chardev 'socket,id=qmp,path=/var/run/qemu-server/105.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -pidfile /var/run/qemu-server/105.pid -daemonize -smbios 'type=1,uuid=205a6221-d6e2-4ea3-a0b9-e6d45c15953b' -name ase-mono01 -smp '2,sockets=1,cores=2,maxcpus=2' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vga std -vnc unix:/var/run/qemu-server/105.vnc,x509,password -cpu kvm64,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce -m 1024 -k fr -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:47185d13f4a' -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2' -drive 'file=/var/lib/vz/images/105/vm-105-disk-1.qcow2,if=none,id=drive-virtio0,cache=writeback,format=qcow2,aio=threads,detect-zeroes=on' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap105i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=02:00:00:51:f1:b9,netdev=net0,bus=pci.0,addr=0x12,id=net0'' failed: exit code 1
root@proxmox:~# qm list
VMID NAME STATUS MEM(MB) BOOTDISK(GB) PID
100 qm-101 running 3072 32.00 5857
102 qm-102 running 6144 20.00 3644
103 qm-103 running 4096 16.00 7214
104 qm-104 running 1024 16.00 4820
105 qm-105 stopped 1024 8.00 0
root@proxmox:~# free -m
total used free shared buff/cache available
Mem: 31859 30451 1214 123 192 986
Swap: 0 0 0
root@proxmox:~# qm stop 104
root@proxmox:~# qm start 104
root@proxmox:~# qm start 105
root@proxmox:~# free -m
total used free shared buff/cache available
Mem: 31859 19748 11916 123 194 11689
Swap: 0 0 0
On Munin Graph the memory go high as soon as backup start
Any advices?