ZFS and Suspend does NOT work - I understood it did?

It's not that I don't want to it's that it doesn't work as a replacement for the previous vzdump

Please be more specific about what exactly "our qemu backup and restore" is. I.e. A CLI command as requested. Using english words leaves too much room for ambigutity. A CLI copy an paste is both instant and unambiguous and I think anyone who uses Proxmox is proficient enough to understand this.

I think more and more that I'm correct and you're just not understanding me (or you are) despite me giving CLI examples that are unambiguous.

Your further suggestion above does not address my need (indeed I'd already found and experimented with it) - it is not a replacement for the previous vzdump.

Thank you at least for your ongoing assistance.
 
please post the configuration of the VM in question ("qm config 102"), and run a manual backup with "vzdump 102 --mode snapshot --storage mnt_backup_daily" and post the complete output here.

also, what kind of storage is your backup storage really? you once write
The backup -storage is an ext4 partition on a locally attached disk.
but then you write
This is the target (iSCSI) and this is taken whilst it's being read from

it might be a good idea to monitor system usage with atop before and while the backup is running, and also check the system logs around the time of the backup for any warnings or error messages.
 
How do we establish that the guest did or did not lock... ? In the example below I could open a console to the 2008r2 VPS, but could not set the focus on any of the windows using the console.

If you feel the storage is relevant state how, I've tried both the options given above. My own solution works equally well with both, thus I'm happy I've ruled out any capacity issues you might have with the system.

Let me know if you suspect a config issue, I'm happy to pursue that.

]# qm config 102
agent: 1
bootdisk: virtio0
cores: 2
ide0: none,media=cdrom
memory: 8192
name: XXXXXXXXXXXXXXX
net0: virtio=1E:C7:C0:2D:20:45,bridge=vmbr1
onboot: 1
ostype: win7
sockets: 1
virtio0: cis01zfs:vm-102-disk-1,size=50G

# vzdump 102 --mode snapshot --storage mnt_backup_daily
INFO: starting new backup job: vzdump 102 --mode snapshot --storage mnt_backup_daily
INFO: Starting Backup of VM 102 (qemu)
INFO: status = running
INFO: update VM 102: -lock backup
INFO: VM Name: XXXXXXXXXXXXXXXXXXXX
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating archive '/mnt/backup/daily/dump/vzdump-qemu-102-2016_09_23-12_41_36.vma'
ERROR: VM 102 qmp command 'guest-fsfreeze-freeze' failed - unable to connect to VM 102 qga socket - timeout after 101 retries
ERROR: VM 102 qmp command 'guest-fsfreeze-thaw' failed - unable to connect to VM 102 qga socket - timeout after 101 retries
INFO: started backup task 'a66bb1f2-53ed-460f-aabd-042f5de46aa2'
INFO: status: 0% (383909888/53687091200), sparse 0% (94871552), duration 4, 95/72 MB/s
INFO: status: 1% (560791552/53687091200), sparse 0% (95784960), duration 11, 25/25 MB/s
INFO: status: 2% (1083441152/53687091200), sparse 0% (133791744), duration 34, 22/21 MB/s
INFO: status: 3% (1630797824/53687091200), sparse 0% (140488704), duration 57, 23/23 MB/s
INFO: status: 4% (2165178368/53687091200), sparse 0% (140607488), duration 81, 22/22 MB/s
INFO: status: 5% (2691432448/53687091200), sparse 0% (147005440), duration 107, 20/19 MB/s
INFO: status: 6% (3227320320/53687091200), sparse 0% (157200384), duration 129, 24/23 MB/s
INFO: status: 7% (3759472640/53687091200), sparse 0% (245313536), duration 150, 25/21 MB/s
INFO: status: 8% (4310564864/53687091200), sparse 0% (248745984), duration 174, 22/22 MB/s
INFO: status: 9% (4848025600/53687091200), sparse 0% (255332352), duration 201, 19/19 MB/s
INFO: status: 10% (5371133952/53687091200), sparse 0% (257757184), duration 225, 21/21 MB/s
INFO: status: 11% (5914755072/53687091200), sparse 0% (263405568), duration 250, 21/21 MB/s
INFO: status: 12% (6443106304/53687091200), sparse 0% (270290944), duration 273, 22/22 MB/s
INFO: status: 13% (6982795264/53687091200), sparse 0% (284098560), duration 297, 22/21 MB/s
INFO: status: 14% (7533887488/53687091200), sparse 0% (286797824), duration 323, 21/21 MB/s
INFO: status: 15% (8066105344/53687091200), sparse 0% (290217984), duration 347, 22/22 MB/s
INFO: status: 16% (8849784832/53687091200), sparse 1% (1026097152), duration 352, 156/9 MB/s
INFO: status: 18% (9826009088/53687091200), sparse 3% (2002321408), duration 355, 325/0 MB/s
INFO: status: 20% (10931666944/53687091200), sparse 5% (3107979264), duration 358, 368/0 MB/s
INFO: status: 21% (11632246784/53687091200), sparse 7% (3808559104), duration 361, 233/0 MB/s
INFO: status: 23% (12520390656/53687091200), sparse 8% (4696702976), duration 364, 296/0 MB/s
INFO: status: 24% (13010337792/53687091200), sparse 9% (5186650112), duration 367, 163/0 MB/s
INFO: status: 26% (14060355584/53687091200), sparse 11% (6236667904), duration 370, 350/0 MB/s
INFO: status: 28% (15119155200/53687091200), sparse 13% (7295467520), duration 373, 352/0 MB/s
INFO: status: 30% (16138960896/53687091200), sparse 15% (8315273216), duration 376, 339/0 MB/s
INFO: status: 31% (16680157184/53687091200), sparse 16% (8833384448), duration 380, 135/5 MB/s
INFO: status: 32% (17203920896/53687091200), sparse 16% (8838356992), duration 403, 22/22 MB/s
INFO: status: 33% (17717133312/53687091200), sparse 16% (8841019392), duration 432, 17/17 MB/s
INFO: status: 34% (18256887808/53687091200), sparse 16% (8841977856), duration 456, 22/22 MB/s
INFO: status: 35% (18807848960/53687091200), sparse 16% (8845365248), duration 483, 20/20 MB/s
INFO: status: 36% (19343867904/53687091200), sparse 16% (8853078016), duration 508, 21/21 MB/s
INFO: status: 37% (19871432704/53687091200), sparse 16% (8858013696), duration 530, 23/23 MB/s
INFO: status: 38% (20430913536/53687091200), sparse 16% (8858976256), duration 555, 22/22 MB/s
INFO: status: 39% (20963131392/53687091200), sparse 16% (8895569920), duration 579, 22/20 MB/s
INFO: status: 40% (21491613696/53687091200), sparse 16% (8899739648), duration 601, 24/23 MB/s
INFO: status: 41% (22027501568/53687091200), sparse 16% (8937922560), duration 624, 23/21 MB/s
INFO: status: 42% (22573023232/53687091200), sparse 16% (8940560384), duration 651, 20/20 MB/s
INFO: status: 43% (23091871744/53687091200), sparse 16% (8941780992), duration 676, 20/20 MB/s
INFO: status: 44% (23627759616/53687091200), sparse 16% (8942055424), duration 702, 20/20 MB/s
INFO: status: 45% (24159846400/53687091200), sparse 16% (8949645312), duration 727, 21/20 MB/s
INFO: status: 46% (24726405120/53687091200), sparse 16% (9065177088), duration 747, 28/22 MB/s
INFO: status: 47% (25258622976/53687091200), sparse 17% (9322803200), duration 763, 33/17 MB/s
INFO: status: 48% (25791234048/53687091200), sparse 17% (9587761152), duration 776, 40/20 MB/s
INFO: status: 49% (26311524352/53687091200), sparse 18% (9748049920), duration 793, 30/21 MB/s
INFO: status: 50% (26866417664/53687091200), sparse 18% (9751138304), duration 818, 22/22 MB/s
INFO: status: 51% (27398504448/53687091200), sparse 18% (9776750592), duration 842, 22/21 MB/s
INFO: status: 52% (27919384576/53687091200), sparse 18% (9778110464), duration 865, 22/22 MB/s
INFO: status: 53% (28477882368/53687091200), sparse 18% (9782095872), duration 889, 23/23 MB/s
INFO: status: 54% (29002563584/53687091200), sparse 18% (9786363904), duration 914, 20/20 MB/s
INFO: status: 55% (29539696640/53687091200), sparse 18% (9791827968), duration 937, 23/23 MB/s
INFO: status: 56% (30086332416/53687091200), sparse 18% (9793277952), duration 961, 22/22 MB/s
INFO: status: 57% (30602952704/53687091200), sparse 18% (9797378048), duration 986, 20/20 MB/s
INFO: status: 58% (31172722688/53687091200), sparse 18% (9800908800), duration 1011, 22/22 MB/s
INFO: status: 59% (31688622080/53687091200), sparse 18% (9802547200), duration 1034, 22/22 MB/s
INFO: status: 60% (32214614016/53687091200), sparse 18% (9935773696), duration 1054, 26/19 MB/s
INFO: status: 61% (33041285120/53687091200), sparse 19% (10416214016), duration 1070, 51/21 MB/s
INFO: status: 63% (34027667456/53687091200), sparse 21% (11402596352), duration 1073, 328/0 MB/s
INFO: status: 65% (35005661184/53687091200), sparse 23% (12380590080), duration 1076, 325/0 MB/s
INFO: status: 66% (35964190720/53687091200), sparse 24% (13339119616), duration 1079, 319/0 MB/s
INFO: status: 68% (36669161472/53687091200), sparse 26% (14044090368), duration 1082, 234/0 MB/s
INFO: status: 69% (37433376768/53687091200), sparse 27% (14808305664), duration 1085, 254/0 MB/s
INFO: status: 71% (38300024832/53687091200), sparse 29% (15674953728), duration 1089, 216/0 MB/s
INFO: status: 73% (39216021504/53687091200), sparse 30% (16590950400), duration 1092, 305/0 MB/s
INFO: status: 74% (40166359040/53687091200), sparse 32% (17541287936), duration 1095, 316/0 MB/s
INFO: status: 76% (40930377728/53687091200), sparse 34% (18305306624), duration 1098, 254/0 MB/s
INFO: status: 77% (41777954816/53687091200), sparse 35% (19152883712), duration 1101, 282/0 MB/s
INFO: status: 79% (42530635776/53687091200), sparse 37% (19905564672), duration 1104, 250/0 MB/s
INFO: status: 80% (43446697984/53687091200), sparse 38% (20821626880), duration 1107, 305/0 MB/s
INFO: status: 82% (44347555840/53687091200), sparse 40% (21722484736), duration 1111, 225/0 MB/s
INFO: status: 84% (45122977792/53687091200), sparse 41% (22497906688), duration 1114, 258/0 MB/s
INFO: status: 85% (45842300928/53687091200), sparse 43% (23217229824), duration 1117, 239/0 MB/s
INFO: status: 86% (46658093056/53687091200), sparse 44% (24033021952), duration 1120, 271/0 MB/s
INFO: status: 88% (47675408384/53687091200), sparse 46% (25050337280), duration 1123, 339/0 MB/s
INFO: status: 90% (48745152512/53687091200), sparse 48% (26120081408), duration 1126, 356/0 MB/s
INFO: status: 92% (49699422208/53687091200), sparse 50% (27074351104), duration 1129, 318/0 MB/s
INFO: status: 94% (50611748864/53687091200), sparse 52% (27986677760), duration 1132, 304/0 MB/s
INFO: status: 95% (51216711680/53687091200), sparse 53% (28591640576), duration 1135, 201/0 MB/s
INFO: status: 97% (52202962944/53687091200), sparse 55% (29577891840), duration 1138, 328/0 MB/s
INFO: status: 99% (53348532224/53687091200), sparse 57% (30723461120), duration 1141, 381/0 MB/s
INFO: status: 100% (53687091200/53687091200), sparse 57% (31062016000), duration 1142, 338/0 MB/s
INFO: transferred 53687 MB in 1142 seconds (47 MB/s)
INFO: archive file size: 21.08GB
INFO: delete old backup '/mnt/backup/daily/dump/vzdump-qemu-102-2016_09_22-13_50_02.vma'
INFO: Finished Backup of VM 102 (00:19:26)
INFO: Backup job finished successfully
 
so it seems the backup works without issues? except for the fact that you tell PVE that the guest has the agent installed, which is not true..
 
This is such a time waste, scroll up to Tom writing "our qemu backup does NOT need a special guest OS tool".

So now we're agreed: ZFS and Suspend does NOT work (unless you install the agent on the guest os) ?
 
This is such a time waste, scroll up to Tom writing "our qemu backup does NOT need a special guest OS tool".

So now we're agreed: ZFS and Suspend does NOT work (unless you install the agent on the guest os) ?

I'll try to make this 100% clear to prevent further misunderstandings (all of this applies to VMs only, the backup mechanism for containers is completely different under the hood):
  • our backup mechanism for KVM guests does not require any support from the guest operating system, it is built into our version of Qemu
  • stop mode works completely independent from the guest agent or the used storage technology, vzdump basically tells Qemu to shutdown/stop the VM before the backup and restart it afterwards (if it was running)
  • suspend mode works completely independent from the guest agent or the used storage technology, vzdump basically tells Qemu to pause the VM for the duration of the backup, and resume it afterwards
  • snapshot mode has nothing to do with storage snapshots, and is also independent from the guest agent
  • if the VM is running, and you set the configuration flag telling PVE that the Qemu agent is available in the guest, then (and only then) vzdump will try to freeze the guest file systems (for improved consistency) before starting the actual backup operation, and thaw them after it's finished

In your case, you set the flag in the configuration telling PVE that the agent is available, but you did not install it in the guest (in other words, you made a configuration error). vzdump now tries to freeze and thaw the file systems, and these two operations fail (naturally, as the agent is not actually available!). vzdump warns you about this (you should either install the agent or remove the flag), but the backup itself works (just without the additional consistency that the freeze/thaw operations would provide).

All of this information is contained in the two links Tom posted in this thread in his first reply..
 
Right, so if I change "agent: 1" to "agent: 0" and run "vzdump 102 --mode snapshot --storage mnt_backup_daily" (this is what I originally did before starting this thread) do you beleive the guest OS will lock or not? This is with the VM Running. No Agent.
 
yes, that is expected to work. I am not sure what you mean by "the guest OS will locK", so please correct me if I am wrong - I assume you mean it freezes/hangs somehow? if so, than it is very likely that the reason is that your storage cannot sustain both the VM running and the backup job running at the same time, and you will need to limit one of them (or upgrade your storage). vzdump offers a "--bwlimit" option which you can use to slow down the backup job, to keep the total I/O load down.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!