[SOLVED] Backup with Snapshot & Qemu Guest running but VM never starts after finish ?

Spirog

Member
Jan 31, 2022
230
24
18
Chicago, IL
Hello,
I was hoping to find some clarity. I am new to Proxmox and setup a backup of 1 VM to test, I setup the backup using Snapshot mode and every 2 hours, But as soon as I started the backup, the VM shut down and so were all my websites and when the backup was finished the Websites were still down. I had to manually shut down the VM and start it again for my sites to be back online.

Is this a normal process? I did not think it was but wondering what went wrong?

Should I just use Stop mode and it should shut down the VM and then start it and my websites should work again?

I find it weird that I had to shut it down and start it to get my websites working again :(

Could this be a bug?

Code:
INFO: starting new backup job: vzdump 104 --prune-backups 'keep-daily=1,keep-last=2' --mailto **@gmail.com --mailnotification always --storage backup_drive --compress zstd --mode snapshot --all 0 --node proxmox
INFO: Starting Backup of VM 104 (qemu)
INFO: Backup started at 2022-01-31 05:44:58
INFO: status = running
INFO: VM Name: AlmaLinux
INFO: include disk 'scsi0' 'local-lvm:vm-104-disk-0' 500G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating vzdump archive '/mnt/backup_drive/dump/vzdump-qemu-104-2022_01_31-05_44_58.vma.zst'
INFO: issuing guest-agent 'fs-freeze' command



I am using the latest Proxmox Virtual Environment 7.1-10

Code:
proxmox-ve: 7.1-1 (running kernel: 5.13.19-3-pve)
pve-manager: 7.1-10 (running version: 7.1-10/6ddebafe)
pve-kernel-helper: 7.1-8
pve-kernel-5.13: 7.1-6
pve-kernel-5.4: 6.4-11
pve-kernel-5.13.19-3-pve: 5.13.19-7
pve-kernel-5.13.19-2-pve: 5.13.19-4
pve-kernel-5.4.157-1-pve: 5.4.157-1
pve-kernel-5.4.73-1-pve: 5.4.73-1
ceph-fuse: 15.2.15-pve1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: 0.8.36+pve1
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.1
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-6
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.1-2
libpve-guest-common-perl: 4.0-3
libpve-http-server-perl: 4.1-1
libpve-storage-perl: 7.0-15
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.11-1
lxcfs: 4.0.11-pve1
novnc-pve: 1.3.0-1
proxmox-backup-client: 2.1.4-1
proxmox-backup-file-restore: 2.1.4-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.4-5
pve-cluster: 7.1-3
pve-container: 4.1-3
pve-docs: 7.1-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.3-4
pve-ha-manager: 3.3-3
pve-i18n: 2.6-2
pve-qemu-kvm: 6.1.0-3
pve-xtermjs: 4.16.0-1
qemu-server: 7.1-4
smartmontools: 7.2-pve2
spiceterm: 3.2-2
swtpm: 0.7.0~rc1+2
vncterm: 1.7-1
zfsutils-linux: 2.1.2-pve1

Thank you in advance to anyone who is able to help me!
Kind Regards,
Spiro
 
Last edited:

Spirog

Member
Jan 31, 2022
230
24
18
Chicago, IL
OK just tested again with backup mode - Snapshot and after the backup has finished, the VM shows it's running in PVE but my websites are still down and I have to stop and start the VM for my sites to work again.

- I did another test SELECTED backup mode - STOP and after it finished it started and my sites were up without manually having to stop and start the VM.

So it may be a bug while using Qemu Guest Agent

I will just use backup mode -Stop for now
until I can get better direction from someone who is well versed with this

Thanks again for anyone's assistance

Kind Regards,
Spiro
 

Spirog

Member
Jan 31, 2022
230
24
18
Chicago, IL
Ok tested Manual backup of 1 VM using Snapshot mode and its been 50 mins so far
all i see is below
Code:
INFO: starting new backup job: vzdump 104 --node proxmox --storage backup_drive --remove 0 --compress zstd --mode snapshot
INFO: Starting Backup of VM 104 (qemu)
INFO: Backup started at 2022-01-31 23:15:07
INFO: status = running
INFO: VM Name: AlmaLinux
INFO: include disk 'scsi0' 'local-lvm:vm-104-disk-0' 500G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating vzdump archive '/mnt/backup_drive/dump/vzdump-qemu-104-2022_01_31-23_15_07.vma.zst'
INFO: issuing guest-agent 'fs-freeze' command

and here is VM Specs

Code:
 Status running
 HA State none
 Node proxmox
 CPU usage 0.33% of 50 CPU(s)
 Memory usage 3.23% (2.91 GiB of 90.00 GiB)
 Bootdisk size 500.00 GiB
 IPs
Guest Agent not running

last backup was only 3.79GB
I did a scheduled backup using Stop mode and it only took about 3-5 mins and was completed and VM was back up and all websites running

So I think the Snapshot mode for backing up Via Manually or Via Scheduled is a BUG..


When I mouse over the VM there is a popup shows
----- Status Running Config Locked (backup) ------


OK Now it finished but Freeze Failed and so did Thaw see errors below. backup finished and in PVE shows its ruining but the VM is really not, cause all my websites are down took total 1 hr 5 min 30.4 sec

Guest Agent Not Running.

ONLY: archive file size: 3.47GB small but took over 1 hour. there is something wrong here for the specs on my server this is way to long. and VM is still down.



Code:
INFO: starting new backup job: vzdump 104 --node proxmox --storage backup_drive --remove 0 --compress zstd --mode snapshot
INFO: Starting Backup of VM 104 (qemu)
INFO: Backup started at 2022-01-31 23:15:07
INFO: status = running
INFO: VM Name: AlmaLinux
INFO: include disk 'scsi0' 'local-lvm:vm-104-disk-0' 500G
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating vzdump archive '/mnt/backup_drive/dump/vzdump-qemu-104-2022_01_31-23_15_07.vma.zst'
INFO: issuing guest-agent 'fs-freeze' command
ERROR: VM 104 qmp command 'guest-fsfreeze-freeze' failed - got timeout
INFO: issuing guest-agent 'fs-thaw' command
ERROR: VM 104 qmp command 'guest-fsfreeze-thaw' failed - got timeout
INFO: started backup task '7c1bafde-a78a-4e9a-8b90-2b9c4d0fa40d'
INFO: resuming VM again
INFO:   2% (11.4 GiB of 500.0 GiB) in 3s, read: 3.8 GiB/s, write: 383.8 MiB/s
INFO:   3% (18.0 GiB of 500.0 GiB) in 6s, read: 2.2 GiB/s, write: 0 B/s
INFO:   4% (22.8 GiB of 500.0 GiB) in 9s, read: 1.6 GiB/s, write: 0 B/s
INFO:   7% (37.0 GiB of 500.0 GiB) in 12s, read: 4.7 GiB/s, write: 1.8 MiB/s
INFO:   8% (43.0 GiB of 500.0 GiB) in 15s, read: 2.0 GiB/s, write: 190.7 KiB/s
INFO:   9% (49.5 GiB of 500.0 GiB) in 18s, read: 2.2 GiB/s, write: 0 B/s
INFO:  11% (56.0 GiB of 500.0 GiB) in 21s, read: 2.2 GiB/s, write: 0 B/s
INFO:  12% (62.7 GiB of 500.0 GiB) in 24s, read: 2.2 GiB/s, write: 0 B/s
INFO:  16% (80.8 GiB of 500.0 GiB) in 27s, read: 6.0 GiB/s, write: 12.0 KiB/s
INFO:  19% (99.7 GiB of 500.0 GiB) in 30s, read: 6.3 GiB/s, write: 0 B/s
INFO:  22% (111.3 GiB of 500.0 GiB) in 33s, read: 3.9 GiB/s, write: 30.8 MiB/s
INFO:  23% (116.4 GiB of 500.0 GiB) in 41s, read: 651.2 MiB/s, write: 83.2 MiB/s
INFO:  24% (123.5 GiB of 500.0 GiB) in 44s, read: 2.4 GiB/s, write: 0 B/s
INFO:  26% (130.6 GiB of 500.0 GiB) in 47s, read: 2.4 GiB/s, write: 0 B/s
INFO:  27% (137.7 GiB of 500.0 GiB) in 50s, read: 2.4 GiB/s, write: 0 B/s
INFO:  28% (144.7 GiB of 500.0 GiB) in 53s, read: 2.3 GiB/s, write: 2.3 MiB/s
INFO:  30% (151.9 GiB of 500.0 GiB) in 56s, read: 2.4 GiB/s, write: 0 B/s
INFO:  32% (163.5 GiB of 500.0 GiB) in 59s, read: 3.8 GiB/s, write: 0 B/s
INFO:  36% (182.2 GiB of 500.0 GiB) in 1m 2s, read: 6.3 GiB/s, write: 16.0 KiB/s
INFO:  37% (189.7 GiB of 500.0 GiB) in 1m 5s, read: 2.5 GiB/s, write: 0 B/s
INFO:  39% (196.9 GiB of 500.0 GiB) in 1m 8s, read: 2.4 GiB/s, write: 0 B/s
INFO:  40% (204.1 GiB of 500.0 GiB) in 1m 11s, read: 2.4 GiB/s, write: 0 B/s
INFO:  42% (211.3 GiB of 500.0 GiB) in 1m 14s, read: 2.4 GiB/s, write: 0 B/s
INFO:  43% (217.5 GiB of 500.0 GiB) in 1m 17s, read: 2.1 GiB/s, write: 13.3 MiB/s
INFO:  44% (220.6 GiB of 500.0 GiB) in 1m 24s, read: 454.0 MiB/s, write: 93.3 MiB/s
INFO:  45% (227.7 GiB of 500.0 GiB) in 1m 27s, read: 2.4 GiB/s, write: 0 B/s
INFO:  46% (234.9 GiB of 500.0 GiB) in 1m 30s, read: 2.4 GiB/s, write: 0 B/s
INFO:  48% (242.1 GiB of 500.0 GiB) in 1m 33s, read: 2.4 GiB/s, write: 0 B/s
INFO:  49% (249.2 GiB of 500.0 GiB) in 1m 36s, read: 2.4 GiB/s, write: 0 B/s
INFO:  51% (256.4 GiB of 500.0 GiB) in 1m 39s, read: 2.4 GiB/s, write: 600.0 KiB/s
INFO:  52% (263.6 GiB of 500.0 GiB) in 1m 42s, read: 2.4 GiB/s, write: 0 B/s
INFO:  54% (270.8 GiB of 500.0 GiB) in 1m 45s, read: 2.4 GiB/s, write: 0 B/s
INFO:  55% (278.0 GiB of 500.0 GiB) in 1m 48s, read: 2.4 GiB/s, write: 0 B/s
INFO:  57% (285.2 GiB of 500.0 GiB) in 1m 51s, read: 2.4 GiB/s, write: 13.3 KiB/s
INFO:  58% (292.4 GiB of 500.0 GiB) in 1m 54s, read: 2.4 GiB/s, write: 0 B/s
INFO:  59% (299.8 GiB of 500.0 GiB) in 1m 57s, read: 2.5 GiB/s, write: 0 B/s
INFO:  61% (307.0 GiB of 500.0 GiB) in 2m, read: 2.4 GiB/s, write: 0 B/s
INFO:  62% (314.2 GiB of 500.0 GiB) in 2m 3s, read: 2.4 GiB/s, write: 0 B/s
INFO:  63% (319.0 GiB of 500.0 GiB) in 2m 6s, read: 1.6 GiB/s, write: 0 B/s
INFO:  64% (323.8 GiB of 500.0 GiB) in 2m 9s, read: 1.6 GiB/s, write: 23.5 MiB/s
INFO:  65% (326.9 GiB of 500.0 GiB) in 2m 16s, read: 450.9 MiB/s, write: 127.4 MiB/s
INFO:  66% (334.0 GiB of 500.0 GiB) in 2m 19s, read: 2.4 GiB/s, write: 0 B/s
INFO:  68% (341.1 GiB of 500.0 GiB) in 2m 22s, read: 2.4 GiB/s, write: 0 B/s
INFO:  69% (348.3 GiB of 500.0 GiB) in 2m 25s, read: 2.4 GiB/s, write: 0 B/s
INFO:  71% (355.4 GiB of 500.0 GiB) in 2m 28s, read: 2.4 GiB/s, write: 0 B/s
INFO:  72% (362.5 GiB of 500.0 GiB) in 2m 31s, read: 2.4 GiB/s, write: 665.3 KiB/s
INFO:  73% (369.5 GiB of 500.0 GiB) in 2m 34s, read: 2.4 GiB/s, write: 0 B/s
INFO:  75% (376.6 GiB of 500.0 GiB) in 2m 37s, read: 2.4 GiB/s, write: 0 B/s
INFO:  76% (383.7 GiB of 500.0 GiB) in 2m 40s, read: 2.4 GiB/s, write: 0 B/s
INFO:  78% (391.3 GiB of 500.0 GiB) in 2m 43s, read: 2.5 GiB/s, write: 22.7 KiB/s
INFO:  79% (399.0 GiB of 500.0 GiB) in 2m 46s, read: 2.6 GiB/s, write: 0 B/s
INFO:  81% (406.3 GiB of 500.0 GiB) in 2m 49s, read: 2.4 GiB/s, write: 0 B/s
INFO:  82% (413.5 GiB of 500.0 GiB) in 2m 52s, read: 2.4 GiB/s, write: 0 B/s
INFO:  84% (420.9 GiB of 500.0 GiB) in 2m 55s, read: 2.5 GiB/s, write: 0 B/s
INFO:  85% (428.5 GiB of 500.0 GiB) in 2m 58s, read: 2.5 GiB/s, write: 0 B/s
INFO:  86% (430.2 GiB of 500.0 GiB) in 3m 1s, read: 574.0 MiB/s, write: 57.0 MiB/s
INFO:  87% (435.8 GiB of 500.0 GiB) in 3m 19s, read: 320.7 MiB/s, write: 95.1 MiB/s
INFO:  88% (443.0 GiB of 500.0 GiB) in 3m 22s, read: 2.4 GiB/s, write: 0 B/s
INFO:  89% (447.6 GiB of 500.0 GiB) in 3m 25s, read: 1.5 GiB/s, write: 25.2 MiB/s
INFO:  90% (450.9 GiB of 500.0 GiB) in 3m 38s, read: 261.3 MiB/s, write: 87.4 MiB/s
INFO:  91% (459.0 GiB of 500.0 GiB) in 3m 41s, read: 2.7 GiB/s, write: 0 B/s
INFO:  93% (465.1 GiB of 500.0 GiB) in 3m 44s, read: 2.0 GiB/s, write: 17.5 MiB/s
INFO:  94% (471.0 GiB of 500.0 GiB) in 4m 8s, read: 255.0 MiB/s, write: 88.7 MiB/s
INFO:  95% (478.5 GiB of 500.0 GiB) in 4m 11s, read: 2.5 GiB/s, write: 0 B/s
INFO:  96% (482.6 GiB of 500.0 GiB) in 4m 14s, read: 1.4 GiB/s, write: 30.9 MiB/s
INFO:  97% (485.1 GiB of 500.0 GiB) in 4m 30s, read: 162.6 MiB/s, write: 103.8 MiB/s
INFO:  98% (493.3 GiB of 500.0 GiB) in 4m 33s, read: 2.7 GiB/s, write: 0 B/s
INFO: 100% (500.0 GiB of 500.0 GiB) in 4m 36s, read: 2.2 GiB/s, write: 0 B/s
INFO: backup is sparse: 489.64 GiB (97%) total zero data
INFO: transferred 500.00 GiB in 276 seconds (1.8 GiB/s)
INFO: archive file size: 3.47GB
INFO: Finished Backup of VM 104 (01:04:46)
INFO: Backup finished at 2022-02-01 00:19:53
INFO: Backup job finished successfully
TASK OK

I also had SSH opened and saw some messages not a pro what so ever so I have no Idea what it means but I could not copy and paste so I uploaded a pdf so you can see if that is an actual error ?

Any Help would be appreciated how to fix this or what the issue may be

Thanks again
Spiro
 

Attachments

  • QEMU (AlmaLinux) - noVNC.pdf
    63.6 KB · Views: 2

dcsapak

Proxmox Staff Member
Staff member
Feb 1, 2016
8,062
993
163
34
Vienna
did you install/start the guest agent in the vm ?
ERROR: VM 104 qmp command 'guest-fsfreeze-freeze' failed - got timeout
Code:
Guest Agent not running
these lines would indicate that you did not.

we try to freeze/thaw the guests fs with the guest agent, but if it's not installed that can lead to problems.. so either install/start it in the guest, or deactivate it in the config (under options)
 

Spirog

Member
Jan 31, 2022
230
24
18
Chicago, IL
did you install/start the guest agent in the vm ?


these lines would indicate that you did not.

we try to freeze/thaw the guests fs with the guest agent, but if it's not installed that can lead to problems.. so either install/start it in the guest, or deactivate it in the config (under options)
Hello and thanks for reply.
I have installed it on that vm prior when first setting it up. Also prior to setting up scheduled backups. It is running. Then I setup schedule backup snapshot mode and when it Goes into backup mode. I get those errors. So it was installed and running prior to the backups.

It for some reason it doesn’t work with snapshot mode manually or scheduled

Thanks
SPIRO
 

Spirog

Member
Jan 31, 2022
230
24
18
Chicago, IL
did you install/start the guest agent in the vm ?
@dcsapak Yes was already installed and running prior tp scheduled or manual backup in mode - Snapshot

It shuts down when the backup starts.. and it takes for ever for the backup to start and never starts backup so leaves the VM down until you stop and start it again.

there has to be something wrong.

Thanks Spiro
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!