After update to 3.2: VM crashing during backup

dietmar said:
Just want to note that this is a totally different error (timeout instead of crash).
Click to expand...

Thanks for notice, BTW the result is the same: no backup and crashed VM since update 3.1->3.2.

Log from yesterday ;)
Code: 
Mar 17 00:09:14 INFO: Starting Backup of VM 302 (qemu)
Mar 17 00:09:14 INFO: status = running
Mar 17 00:09:15 INFO: update VM 302: -lock backup
Mar 17 00:09:15 INFO: backup mode: snapshot
Mar 17 00:09:15 INFO: ionice priority: 7
Mar 17 00:09:15 INFO: creating archive '/srv/dump/vzdump-qemu-302-2014_03_17-00_09_14.vma.gz'
Mar 17 00:09:15 INFO: started backup task 'a7a94481-79bd-486d-b77b-095e78f6d9e4'
Mar 17 00:09:18 INFO: status: 1% (266141696/16110321664), sparse 1% (209334272), duration 3, 88/18 MB/s
Mar 17 00:09:21 INFO: status: 7% (1186004992/16110321664), sparse 6% (1038827520), duration 6, 306/30 MB/s
Mar 17 00:09:24 INFO: status: 8% (1315176448/16110321664), sparse 6% (1081188352), duration 9, 43/28 MB/s
Mar 17 00:09:31 INFO: status: 9% (1471021056/16110321664), sparse 6% (1082675200), duration 16, 22/22 MB/s
Mar 17 00:09:38 INFO: status: 10% (1626865664/16110321664), sparse 6% (1083707392), duration 23, 22/22 MB/s
Mar 17 00:09:46 INFO: status: 11% (1790312448/16110321664), sparse 6% (1086218240), duration 31, 20/20 MB/s
Mar 17 00:09:54 INFO: status: 12% (1946157056/16110321664), sparse 6% (1095159808), duration 39, 19/18 MB/s
Mar 17 00:10:03 INFO: status: 13% (2105802752/16110321664), sparse 6% (1097342976), duration 48, 17/17 MB/s
Mar 17 00:10:12 INFO: status: 14% (2265448448/16110321664), sparse 6% (1097420800), duration 57, 17/17 MB/s
Mar 17 00:10:19 INFO: status: 15% (2425094144/16110321664), sparse 6% (1098801152), duration 64, 22/22 MB/s
Mar 17 00:10:28 INFO: status: 16% (2592342016/16110321664), sparse 6% (1107537920), duration 73, 18/17 MB/s
Mar 17 00:10:36 INFO: status: 17% (2740584448/16110321664), sparse 6% (1123172352), duration 81, 18/16 MB/s
Mar 17 00:10:39 ERROR: VM 302 not running
Mar 17 00:10:39 INFO: aborting backup job
Mar 17 00:10:39 ERROR: VM 302 not running
Mar 17 00:10:40 ERROR: Backup of VM 302 failed - VM 302 not running
 
Hello together,

last night I had the following Situation:

3 Hosts
6 VM's
2 VM's stopped suddenly during the backup

HOST1
VM1 Ubuntu 12.04 Crash
INFO: Starting Backup of VM 100 (qemu)
INFO: status = running
INFO: update VM 100: -lock backup
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating archive '/mnt/backup-server/iBackOffice/dump/vzdump-qemu-100-2014_03_18-00_00_02.vma.gz'
INFO: started backup task '290961f4-516c-491a-aebc-09df718a6bbc'
INFO: status: 0% (111804416/64424509440), sparse 0% (8003584), duration 3, 37/34 MB/s
INFO: status: 1% (759824384/64424509440), sparse 0% (411312128), duration 11, 81/30 MB/s
INFO: status: 2% (1303773184/64424509440), sparse 1% (742797312), duration 20, 60/23 MB/s
INFO: status: 3% (1995571200/64424509440), sparse 2% (1302581248), duration 26, 115/22 MB/s
INFO: status: 4% (2717777920/64424509440), sparse 2% (1872957440), duration 29, 240/50 MB/s
INFO: status: 5% (3382968320/64424509440), sparse 3% (2430328832), duration 33, 166/26 MB/s
INFO: status: 7% (4903403520/64424509440), sparse 5% (3815112704), duration 36, 506/45 MB/s
INFO: status: 10% (6994001920/64424509440), sparse 9% (5804183552), duration 39, 696/33 MB/s
INFO: status: 11% (7222788096/64424509440), sparse 9% (5839634432), duration 42, 76/64 MB/s
INFO: status: 12% (7739015168/64424509440), sparse 9% (6120386560), duration 49, 73/33 MB/s
INFO: status: 13% (8382578688/64424509440), sparse 10% (6471688192), duration 56, 91/41 MB/s
INFO: status: 14% (9058451456/64424509440), sparse 10% (6769160192), duration 67, 61/34 MB/s
INFO: status: 15% (9711779840/64424509440), sparse 10% (7005376512), duration 80, 50/32 MB/s
INFO: status: 16% (10312351744/64424509440), sparse 10% (7055798272), duration 102, 27/25 MB/s
INFO: status: 17% (10973741056/64424509440), sparse 11% (7105454080), duration 123, 31/29 MB/s
INFO: status: 18% (11623727104/64424509440), sparse 11% (7225573376), duration 139, 40/33 MB/s
INFO: status: 19% (12262309888/64424509440), sparse 11% (7266762752), duration 158, 33/31 MB/s
INFO: status: 20% (12912295936/64424509440), sparse 11% (7333105664), duration 177, 34/30 MB/s
INFO: status: 21% (13562281984/64424509440), sparse 11% (7641001984), duration 189, 54/28 MB/s
INFO: status: 22% (14185660416/64424509440), sparse 11% (7703629824), duration 213, 25/23 MB/s
INFO: status: 23% (14828044288/64424509440), sparse 11% (7721385984), duration 244, 20/20 MB/s
INFO: status: 24% (15531245568/64424509440), sparse 12% (7832801280), duration 269, 28/23 MB/s
INFO: status: 25% (16116613120/64424509440), sparse 12% (7889600512), duration 288, 30/27 MB/s
INFO: status: 26% (16751984640/64424509440), sparse 12% (8004296704), duration 307, 33/27 MB/s
INFO: status: 27% (17404198912/64424509440), sparse 12% (8074612736), duration 327, 32/29 MB/s
INFO: status: 28% (18039963648/64424509440), sparse 12% (8162406400), duration 350, 27/23 MB/s
INFO: status: 29% (18697551872/64424509440), sparse 12% (8200192000), duration 375, 26/24 MB/s
INFO: status: 30% (19362742272/64424509440), sparse 12% (8273248256), duration 395, 33/29 MB/s
INFO: status: 31% (19977338880/64424509440), sparse 12% (8354426880), duration 419, 25/22 MB/s
INFO: status: 32% (20628504576/64424509440), sparse 12% (8366137344), duration 442, 28/27 MB/s
INFO: status: 33% (21263286272/64424509440), sparse 12% (8370499584), duration 468, 24/24 MB/s
INFO: status: 34% (21962686464/64424509440), sparse 13% (8533970944), duration 490, 31/24 MB/s
INFO: status: 35% (22574661632/64424509440), sparse 13% (8550928384), duration 516, 23/22 MB/s
INFO: status: 36% (23201841152/64424509440), sparse 13% (8610680832), duration 540, 26/23 MB/s
INFO: status: 37% (23870832640/64424509440), sparse 13% (8781332480), duration 559, 35/26 MB/s
INFO: status: 38% (24494211072/64424509440), sparse 13% (8922251264), duration 576, 36/28 MB/s
INFO: status: 39% (25140396032/64424509440), sparse 13% (8964857856), duration 599, 28/26 MB/s
INFO: status: 40% (25805586432/64424509440), sparse 14% (9084706816), duration 618, 35/28 MB/s
INFO: status: 41% (26444169216/64424509440), sparse 14% (9247731712), duration 634, 39/29 MB/s
ERROR: VM 100 not running
INFO: aborting backup job
ERROR: VM 100 not running
ERROR: Backup of VM 100 failed - VM 100 not running

VM2 WIN2012R2 OK
VM3 CentOS 5.6 OK

HOST2
VM1 Ubuntu 12.04 Crash
INFO: Starting Backup of VM 100 (qemu)
INFO: status = running
INFO: update VM 100: -lock backup
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating archive '/mnt/backup-server/WD/dump/vzdump-qemu-100-2014_03_18-03_00_01.vma.gz'
INFO: started backup task '28869d90-4118-4340-a69d-956eb522536a'
INFO: status: 0% (97845248/64424509440), sparse 0% (6754304), duration 3, 32/30 MB/s
INFO: status: 1% (702808064/64424509440), sparse 0% (189382656), duration 18, 40/28 MB/s
ERROR: VM 100 not running
INFO: aborting backup job
ERROR: VM 100 not running
ERROR: Backup of VM 100 failed - VM 100 not running

VM2 WIN2008R2 OK

HOST3
VM1 Ubuntu 12.04 OK

The Problem does't belongs to a specific VM, the backup in snapshot mode with good compression runs fine one night, the other night the vm stops suddenly during the backup.

Best regards Stephan
 
cesarpk said:
@gbr, @kaya, @oer2001, @jens.kuespert, @shartenauer:
@dietmar (if i am in the correct way):

I believe that the problem is a common scenery for all you, and please correct me if I'm wrong:

When the Backup are in process (Snapshot or other mode), and two or more PVE Hosts are doing backup to the same destination (HDD or RAID of disks), so the destination NAS/CIFS is saturated of many inputs, and PVE (vzdump) don't know how negotiate this situation.
Click to expand...

I manually started my backup to local storage. In my case, the VM was on the same storage as the backup.

Gerald
 
shartenauer said:
The Problem does't belongs to a specific VM, the backup in snapshot mode with good compression runs fine one night, the other night the vm stops suddenly during the backup.
Click to expand...

What kind of backup storage do you use?
 
johnplv said:
My observations show that the crushed VM with high disk IO
Click to expand...

Then, for get a error reproducible, we can do:
1- Can be copying a dvd full to the VM while a backup snapshot is in progress.
2- Or may be copying several gigabytes of files into the VM to other folder or other PC in network.

Best regards
Cesar
 
cesarpk said:
Then, for get a error reproducible, we can do:
1- Can be copying a dvd full to the VM while a backup snapshot is in progress.
2- Or may be copying several gigabytes of files into the VM to other folder or other PC in network.
Click to expand...

We have already done such tests, but it does not trigger the bug here.
 
I have two crushed VMs:
Zimbra 8 on Debian 7.1 kernel 2.6.32-27-pve
Zabbix 2.2.1 on Debian 7.1 kernel 3.2.0-4-amd64
both use virtio and have high disk IO

Other 10 use diffrent OSes (Debian, Ubuntu, Win2k8, Win2k12)
and have lower disk IO and backup successfully.

May be is important not high BW but high IOPS ?
 
dietmar said:
What kind of backup storage do you use?
Click to expand...

Hello Dietmar,

my backuptarget is the Hetzner FTP Space that is provided with my Hetzner Rootserver.
The backuptarget is mountet via CIFS.
I have 3 hosts with different FTP Spaces, only one Snapshot from a Proxmox host is running simultaneously to the FTP Space.
These situation is running fine since 11 months with one host and 7 months with the other 2 hosts.

Last night 2 VM's crashed again, but not the same as last night.

Host1
VM Ubuntu 12.04 OK

VM WIN2012R2 OK

VM CentOS 5.6 crashes
INFO: Starting Backup of VM 110 (qemu)
INFO: status = running
INFO: update VM 110: -lock backup
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating archive '/mnt/backup-server/iBackOffice/dump/vzdump-qemu-110-2014_03_19-00_37_41.vma.gz'
INFO: started backup task '04b9bb9a-ed4e-4bc9-bb09-0a5044d55e85'
INFO: status: 0% (125435904/53687091200), sparse 0% (25870336), duration 3, 41/33 MB/s
INFO: status: 1% (756416512/53687091200), sparse 1% (583897088), duration 6, 210/24 MB/s
INFO: status: 2% (1117519872/53687091200), sparse 1% (684093440), duration 11, 72/52 MB/s
ERROR: VM 110 not running
INFO: aborting backup job
ERROR: VM 110 not running
ERROR: Backup of VM 110 failed - VM 110 not running
INFO: Backup job finished with errors
TASK ERROR: job errors

Host2
VM Ubuntu 12.04 crashes
INFO: Starting Backup of VM 100 (qemu)
INFO: status = running
INFO: update VM 100: -lock backup
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating archive '/mnt/backup-server/WD/dump/vzdump-qemu-100-2014_03_18-03_00_01.vma.gz'
INFO: started backup task '28869d90-4118-4340-a69d-956eb522536a'
INFO: status: 0% (97845248/64424509440), sparse 0% (6754304), duration 3, 32/30 MB/s
INFO: status: 1% (702808064/64424509440), sparse 0% (189382656), duration 18, 40/28 MB/s
ERROR: VM 100 not running
INFO: aborting backup job
ERROR: VM 100 not running
ERROR: Backup of VM 100 failed - VM 100 not running

VM WIN2008R2 OK

HOST3
VM Ubuntu 12.04 OK

Best regards Stephan
 
This just happened on a production box that's been running 3.2 for quite awhile.

Backup was to NFS mounted USB drive. These backups happen everynight, so it's something pretty random.

Gerald
 
Back on kernel proxmox-ve-2.6.32: 3.2-121 (running kernel: 2.6.32-26-pve) and it still crashes on the same VM. Backup is on local storage, 2 others KVM with the same configuration are OK, 10 oVZ without any problem:
Code: 
Mar 20 00:09:31 INFO: Starting Backup of VM 302 (qemu)
Mar 20 00:09:31 INFO: status = running
Mar 20 00:09:31 INFO: update VM 302: -lock backup
Mar 20 00:09:31 INFO: backup mode: snapshot
Mar 20 00:09:31 INFO: ionice priority: 7
Mar 20 00:09:31 INFO: creating archive '/srv/dump/vzdump-qemu-302-2014_03_20-00_09_31.vma.gz'
Mar 20 00:09:32 INFO: started backup task '3d263fc1-d54c-4570-a171-c62fcd1780bf'
Mar 20 00:09:35 INFO: status: 1% (283377664/16110321664), sparse 1% (226570240), duration 3, 94/18 MB/s
Mar 20 00:09:38 INFO: status: 7% (1193541632/16110321664), sparse 6% (1040908288), duration 6, 303/31 MB/s
Mar 20 00:09:41 INFO: status: 8% (1315176448/16110321664), sparse 6% (1081188352), duration 9, 40/27 MB/s
Mar 20 00:09:48 INFO: status: 9% (1471021056/16110321664), sparse 6% (1082675200), duration 16, 22/22 MB/s
Mar 20 00:09:55 INFO: status: 10% (1626865664/16110321664), sparse 6% (1083645952), duration 23, 22/22 MB/s
Mar 20 00:10:03 INFO: status: 11% (1786511360/16110321664), sparse 6% (1086156800), duration 31, 19/19 MB/s
Mar 20 00:10:11 INFO: status: 12% (1938554880/16110321664), sparse 6% (1094537216), duration 39, 19/17 MB/s
Mar 20 00:10:20 INFO: status: 13% (2105802752/16110321664), sparse 6% (1097281536), duration 48, 18/18 MB/s
Mar 20 00:10:29 INFO: status: 14% (2269249536/16110321664), sparse 6% (1097334784), duration 57, 18/18 MB/s
Mar 20 00:10:36 INFO: status: 15% (2425094144/16110321664), sparse 6% (1099358208), duration 64, 22/21 MB/s
Mar 20 00:10:37 ERROR: VM 302 not running
Mar 20 00:10:37 INFO: aborting backup job
Mar 20 00:10:37 ERROR: VM 302 not running
Mar 20 00:10:37 ERROR: Backup of VM 302 failed - VM 302 not running
 
dietmar said:
Are there any hints in /var/log/syslog?

Hello Dietmar,

last night 1 VM (VM 110) fails during backup.
I found this in syslog.

Mar 20 00:39:08 proxmox002 vzdump[525113]: INFO: Starting Backup of VM 110 (qemu)
Mar 20 00:39:09 proxmox002 qm[526093]: <root@pam> update VM 110: -lock backup
Mar 20 00:39:55 proxmox002 postfix/smtpd[525616]: connect from localhost[127.0.0.1]
Mar 20 00:39:55 proxmox002 postfix/smtpd[525616]: disconnect from localhost[127.0.0.1]
Mar 20 00:40:52 proxmox002 kernel: vmbr0: port 3(tap110i0) entering disabled state
Mar 20 00:40:52 proxmox002 kernel: vmbr0: port 3(tap110i0) entering disabled state
Mar 20 00:40:52 proxmox002 vzdump[525113]: VM 110 qmp command failed - VM 110 not running
Mar 20 00:40:52 proxmox002 vzdump[525113]: VM 110 qmp command failed - VM 110 not running
Mar 20 00:40:52 proxmox002 vzdump[525113]: ERROR: Backup of VM 110 failed - VM 110 not running
Mar 20 00:40:52 proxmox002 vzdump[525113]: INFO: Backup job finished with errors
Mar 20 00:40:52 proxmox002 vzdump[525113]: job Errors

Best regards Stephan
Click to expand...
 
adauc said:
Back on kernel proxmox-ve-2.6.32: 3.2-121 (running kernel: 2.6.32-26-pve) and it still crashes on the same VM. Backup is on local storage, 2 others KVM with the same configuration are OK, 10 oVZ without any problem:
Click to expand...

Please can you test with a more recent kernel.
 
dietmar said:
Please can you test with a more recent kernel.
Click to expand...

Hello Dietmar,
yesterday I upgraded to the new kernel 2.6.32-28-pve, but the result ist not looking good.
During the backup my VM100 freezes.
I could open the console for the VM but the VM's clock stopped at 00:25.
The VM was locked

Detailed backup logs:
vzdump 100 110 101 --quiet 1 --mailto info@xyz.de --mode snapshot --compress gzip --storage backup
100: Mar 21 00:00:01 INFO: Starting Backup of VM 100 (qemu)
100: Mar 21 00:00:01 INFO: status = running
100: Mar 21 00:00:01 INFO: update VM 100: -lock backup
100: Mar 21 00:00:02 INFO: backup mode: snapshot
100: Mar 21 00:00:02 INFO: ionice priority: 7
100: Mar 21 00:00:02 INFO: creating archive '/mnt/backup-server/iBackOffice/dump/vzdump-qemu-100-2014_03_21-00_00_01.vma.gz'
100: Mar 21 00:00:02 INFO: started backup task '7f076408-3e25-44df-897a-77749155eac3'
100: Mar 21 00:00:05 INFO: status: 0% (129236992/64424509440), sparse 0% (9162752), duration 3, 43/40 MB/s
100: Mar 21 00:00:13 INFO: status: 1% (653787136/64424509440), sparse 0% (250462208), duration 11, 65/35 MB/s
100: Mar 21 00:00:23 INFO: status: 2% (1531838464/64424509440), sparse 1% (832204800), duration 21, 87/29 MB/s
100: Mar 21 00:00:27 INFO: status: 3% (1949958144/64424509440), sparse 1% (1161310208), duration 25, 104/22 MB/s
100: Mar 21 00:00:30 INFO: status: 4% (2683043840/64424509440), sparse 2% (1740365824), duration 28, 244/51 MB/s
100: Mar 21 00:00:34 INFO: status: 6% (4274847744/64424509440), sparse 4% (3208503296), duration 32, 397/30 MB/s
100: Mar 21 00:00:37 INFO: status: 10% (6898188288/64424509440), sparse 8% (5657825280), duration 35, 874/58 MB/s
100: Mar 21 00:00:40 INFO: status: 11% (7089029120/64424509440), sparse 8% (5694308352), duration 38, 63/51 MB/s
100: Mar 21 00:00:48 INFO: status: 12% (7834042368/64424509440), sparse 9% (6043840512), duration 46, 93/49 MB/s
100: Mar 21 00:00:53 INFO: status: 13% (8400207872/64424509440), sparse 9% (6353723392), duration 51, 113/51 MB/s
100: Mar 21 00:01:02 INFO: status: 14% (9046589440/64424509440), sparse 10% (6607945728), duration 60, 71/43 MB/s
100: Mar 21 00:01:12 INFO: status: 15% (9669967872/64424509440), sparse 10% (6833528832), duration 70, 62/39 MB/s
100: Mar 21 00:01:31 INFO: status: 16% (10338959360/64424509440), sparse 10% (6890340352), duration 89, 35/32 MB/s
100: Mar 21 00:01:47 INFO: status: 17% (10985144320/64424509440), sparse 10% (6934982656), duration 105, 40/37 MB/s
100: Mar 21 00:02:00 INFO: status: 18% (11619926016/64424509440), sparse 10% (7046836224), duration 118, 48/40 MB/s
100: Mar 21 00:02:16 INFO: status: 19% (12266110976/64424509440), sparse 11% (7088123904), duration 134, 40/37 MB/s
100: Mar 21 00:02:31 INFO: status: 20% (12893290496/64424509440), sparse 11% (7154585600), duration 149, 41/37 MB/s
100: Mar 21 00:02:41 INFO: status: 21% (13550878720/64424509440), sparse 11% (7461253120), duration 159, 65/35 MB/s
100: Mar 21 00:02:59 INFO: status: 22% (14189461504/64424509440), sparse 11% (7522238464), duration 177, 35/32 MB/s
100: Mar 21 00:03:18 INFO: status: 23% (14828044288/64424509440), sparse 11% (7539773440), duration 196, 33/32 MB/s
100: Mar 21 00:03:35 INFO: status: 24% (15508439040/64424509440), sparse 11% (7649042432), duration 213, 40/33 MB/s
100: Mar 21 00:03:51 INFO: status: 25% (16143810560/64424509440), sparse 11% (7695712256), duration 229, 39/36 MB/s

100:
Mar 21 08:37:43 ERROR: VM 100 qmp command 'query-backup' failed - client closed Connection
100: Mar 21 08:37:43 INFO: aborting backup job

100:
Mar 21 08:37:44 ERROR: Backup of VM 100 failed - VM 100 qmp command 'query-backup' failed - client closed Connection

101: Mar 21 08:37:44 INFO: Starting Backup of VM 101 (qemu)
101: Mar 21 08:37:44 INFO: status = running
101: Mar 21 08:37:45 INFO: update VM 101: -lock backup
101: Mar 21 08:37:45 INFO: exclude disk 'virtio1' (backup=no)
101: Mar 21 08:37:45 INFO: backup mode: snapshot
101: Mar 21 08:37:45 INFO: ionice priority: 7
101: Mar 21 08:37:45 INFO: creating archive '/mnt/backup-server/iBackOffice/dump/vzdump-qemu-101-2014_03_21-08_37_44.vma.gz'
101: Mar 21 08:37:45 INFO: started backup task '51e14699-483f-4509-a235-040cc279bb28'
101: Mar 21 08:37:48 INFO: status: 0% (224264192/64424509440), sparse 0% (113680384), duration 3, 74/36 MB/s
101: Mar 21 08:38:01 INFO: status: 1% (676593664/64424509440), sparse 0% (135360512), duration 16, 34/33 MB/s
101: Mar 21 08:38:22 INFO: status: 2% (1315176448/64424509440), sparse 0% (171171840), duration 37, 30/28 MB/s
101: Mar 21 08:38:45 INFO: status: 3% (1946157056/64424509440), sparse 0% (178192384), duration 60, 27/27 MB/s
101: Mar 21 08:39:10 INFO: status: 4% (2592342016/64424509440), sparse 0% (179208192), duration 85, 25/25 MB/s
101: Mar 21 08:39:34 INFO: status: 5% (3245408256/64424509440), sparse 0% (189190144), duration 109, 27/26 MB/s
101: Mar 21 08:39:50 INFO: status: 6% (3877109760/64424509440), sparse 0% (205983744), duration 125, 39/38 MB/s
101: Mar 21 08:40:11 INFO: status: 7% (4538499072/64424509440), sparse 0% (233639936), duration 146, 31/30 MB/s
101: Mar 21 08:40:31 INFO: status: 8% (5173280768/64424509440), sparse 0% (254513152), duration 166, 31/30 MB/s
101: Mar 21 08:40:53 INFO: status: 9% (5811863552/64424509440), sparse 0% (262455296), duration 188, 29/28 MB/s
101: Mar 21 08:41:17 INFO: status: 10% (6458048512/64424509440), sparse 0% (273510400), duration 212, 26/26 MB/s
101: Mar 21 08:41:44 INFO: status: 11% (7092830208/64424509440), sparse 0% (280158208), duration 239, 23/23 MB/s
101: Mar 21 08:42:05 INFO: status: 12% (7754219520/64424509440), sparse 0% (295604224), duration 260, 31/30 MB/s
101: Mar 21 08:42:32 INFO: status: 13% (8396603392/64424509440), sparse 0% (301154304), duration 287, 23/23 MB/s
101: Mar 21 08:42:49 INFO: status: 15% (10114695168/64424509440), sparse 2% (1541148672), duration 304, 101/28 MB/s
101: Mar 21 08:42:54 INFO: status: 16% (10331357184/64424509440), sparse 2% (1568800768), duration 309, 43/37 MB/s
101: Mar 21 08:43:17 INFO: status: 17% (10954735616/64424509440), sparse 2% (1593049088), duration 332, 27/26 MB/s
101: Mar 21 08:43:37 INFO: status: 18% (11600920576/64424509440), sparse 2% (1594929152), duration 352, 32/32 MB/s

101:
Mar 21 08:52:23 ERROR: VM 101 qmp command 'query-backup' failed - client closed Connection
101: Mar 21 08:52:23 INFO: aborting backup job
101:
Mar 21 08:52:23 ERROR: VM 101 not running
101:
Mar 21 08:52:24 ERROR: Backup of VM 101 failed - VM 101 qmp command 'query-backup' failed - client closed Connection
110: Mar 21 08:52:24 INFO: Starting Backup of VM 110 (qemu)
110: Mar 21 08:52:24 INFO: status = stopped
110: Mar 21 08:52:24 INFO: update VM 110: -lock backup
110: Mar 21 08:52:24 INFO: backup mode: stop
110: Mar 21 08:52:24 INFO: ionice priority: 7
110: Mar 21 08:52:24 INFO: creating archive '/mnt/backup-server/iBackOffice/dump/vzdump-qemu-110-2014_03_21-08_52_24.vma.gz'
110: Mar 21 08:52:24 INFO: starting kvm to execute backup task
110: Mar 21 08:52:24 INFO: started backup task 'cae161f5-9dc9-4f65-bc28-a6c833762b3a'
110: Mar 21 08:52:27 INFO: status: 0% (148242432/53687091200), sparse 0% (38789120), duration 3, 49/36 MB/s
110: Mar 21 08:52:30 INFO: status: 1% (828637184/53687091200), sparse 1% (598437888), duration 6, 226/40 MB/s
110: Mar 21 08:52:33 INFO: status: 2% (1089339392/53687091200), sparse 1% (681742336), duration 9, 86/59 MB/s
110: Mar 21 08:52:59 INFO: status: 3% (1619263488/53687091200), sparse 1% (739655680), duration 35, 20/18 MB/s
110:
Mar 21 08:53:24 ERROR: VM 110 not running
110: Mar 21 08:53:24 INFO: aborting backup job
110:
Mar 21 08:53:24 ERROR: VM 110 not running
110:
Mar 21 08:53:24 ERROR: Backup of VM 110 failed - VM 110 not running


Best regards Stephan
 
We need a way to reproduce this if we want it fixed.
Maybe more information will lead to a way to reliably reproduce the problem.

Maybe someone having this issue could do this?:
1. Stop your VM
2. At the command line run:
Code: 
qm showcmd <vmid>
3. Copy the command that was output and run it to start the VM.
4. Perform a backup, wait for crash to occur
5. Hopefully some error messages are spit out when the KVM process crashes and you can copy/paste them here.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back