Folder with images is on Gluster.
Folder with backup is on Gluster.
Gluster version: glusterfs 9.3
pveversion -v
proxmox-ve: 7.0-2 (running kernel: 5.11.22-4-pve)
pve-manager: 7.0-11 (running version: 7.0-11/63d82f4e)
pve-kernel-5.11: 7.0-7
pve-kernel-helper: 7.0-7
pve-kernel-5.11.22-4-pve: 5.11.22-8
pve-kernel-5.11.22-1-pve: 5.11.22-2
ceph-fuse: 15.2.13-pve1
corosync: 3.1.2-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.21-pve1
libproxmox-acme-perl: 1.3.0
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.0-4
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.0-6
libpve-guest-common-perl: 4.0-2
libpve-http-server-perl: 4.0-2
libpve-storage-perl: 7.0-10
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.9-4
lxcfs: 4.0.8-pve2
novnc-pve: 1.2.0-3
proxmox-backup-client: 2.0.9-2
proxmox-backup-file-restore: 2.0.9-2
proxmox-mini-journalreader: 1.2-1
proxmox-widget-toolkit: 3.3-6
pve-cluster: 7.0-3
pve-container: 4.0-9
pve-docs: 7.0-5
pve-edk2-firmware: 3.20200531-1
pve-firewall: 4.2-2
pve-firmware: 3.3-1
pve-ha-manager: 3.3-1
pve-i18n: 2.4-1
pve-qemu-kvm: 6.0.0-3
pve-xtermjs: 4.12.0-1
qemu-server: 7.0-13
smartmontools: 7.2-pve2
spiceterm: 3.2-2
vncterm: 1.7-1
zfsutils-linux: 2.0.5-pve1
Backing up file:
Task viewer: VM/CT 100 - Backup
OutputStatus
Stop
INFO: starting new backup job: vzdump 100 --remove 0 --storage gluster-backup --node bvmVm80 --mode suspend --compress zstd
INFO: Starting Backup of VM 100 (qemu)
INFO: Backup started at 2021-09-10 11:36:42
INFO: status = stopped
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: VM Name: DNS-slave
INFO: include disk 'sata0' 'gluster-image:100/vm-100-disk-0.qcow2' 10G
INFO: creating vzdump archive '/mnt/pve/gluster-backup/dump/vzdump-qemu-100-2021_09_10-11_36_41.vma.zst'
INFO: starting kvm to execute backup task
INFO: started backup task 'eff352ca-b863-4ae8-8c37-6a83c3d9da41'
INFO: 3% (525.6 MiB of 14.0 GiB) in 3s, read: 175.2 MiB/s, write: 163.9 MiB/s
INFO: 6% (909.9 MiB of 14.0 GiB) in 6s, read: 128.1 MiB/s, write: 125.2 MiB/s
INFO: 9% (1.4 GiB of 14.0 GiB) in 9s, read: 169.2 MiB/s, write: 159.9 MiB/s
INFO: 30% (4.3 GiB of 14.0 GiB) in 12s, read: 1001.7 MiB/s, write: 154.0 MiB/s
INFO: 100% (14.0 GiB of 14.0 GiB) in 14s, read: 4.8 GiB/s, write: 22.0 MiB/s
INFO: backup is sparse: 12.19 GiB (87%) total zero data
INFO: transferred 14.00 GiB in 14 seconds (1023.9 MiB/s)
INFO: stopping kvm after backup task
INFO: archive file size: 663MB
INFO: Finished Backup of VM 100 (00:00:23)
INFO: Backup finished at 2021-09-10 11:37:04
INFO: Backup job finished successfully
TASK OK
Restore:
Task viewer: VM 101 - Restore
OutputStatus
Stop
restore vma archive: zstd -q -d -c /mnt/pve/gluster-backup/dump/vzdump-qemu-100-2021_09_10-11_21_14.vma.zst | vma extract -v -r /var/tmp/vzdumptmp2485.fifo - /var/tmp/vzdumptmp2485
CFG: size: 390 name: qemu-server.conf
DEV: dev_id=1 size: 15030747136 devname: drive-sata0
CTIME: Fri Sep 10 11:21:15 2021
Formatting 'gluster://10.8.8.241/bvm_vmimage/images/101/vm-101-disk-0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=metadata compression_type=zlib size=15030747136 lazy_refcounts=off refcount_bits=16
[2021-09-10 09:35:13.414971 +0000] I [io-stats.c:3706:ios_sample_buf_size_configure] 0-bvm_vmimage: Configure ios_sample_buf size is 1024 because ios_sample_interval is 0
[2021-09-10 09:35:13.573057 +0000] E [MSGID: 108006] [afr-common.c:6140:__afr_handle_child_down_event] 0-bvm_vmimage-replicate-0: All subvolumes are down. Going offline until at least one of them comes back up.
no lock found trying to remove 'create' lock
error before or during data restore, some or all disks were not completely restored. VM 101 state is NOT cleaned up.
TASK ERROR: command 'set -o pipefail && zstd -q -d -c /mnt/pve/gluster-backup/dump/vzdump-qemu-100-2021_09_10-11_21_14.vma.zst | vma extract -v -r /var/tmp/vzdumptmp2485.fifo - /var/tmp/vzdumptmp2485' failed: unable to create image: 'storage-gluster-image'-locked command timed out - aborting
Folder with backup is on Gluster.
Gluster version: glusterfs 9.3
pveversion -v
proxmox-ve: 7.0-2 (running kernel: 5.11.22-4-pve)
pve-manager: 7.0-11 (running version: 7.0-11/63d82f4e)
pve-kernel-5.11: 7.0-7
pve-kernel-helper: 7.0-7
pve-kernel-5.11.22-4-pve: 5.11.22-8
pve-kernel-5.11.22-1-pve: 5.11.22-2
ceph-fuse: 15.2.13-pve1
corosync: 3.1.2-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.21-pve1
libproxmox-acme-perl: 1.3.0
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.0-4
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.0-6
libpve-guest-common-perl: 4.0-2
libpve-http-server-perl: 4.0-2
libpve-storage-perl: 7.0-10
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.9-4
lxcfs: 4.0.8-pve2
novnc-pve: 1.2.0-3
proxmox-backup-client: 2.0.9-2
proxmox-backup-file-restore: 2.0.9-2
proxmox-mini-journalreader: 1.2-1
proxmox-widget-toolkit: 3.3-6
pve-cluster: 7.0-3
pve-container: 4.0-9
pve-docs: 7.0-5
pve-edk2-firmware: 3.20200531-1
pve-firewall: 4.2-2
pve-firmware: 3.3-1
pve-ha-manager: 3.3-1
pve-i18n: 2.4-1
pve-qemu-kvm: 6.0.0-3
pve-xtermjs: 4.12.0-1
qemu-server: 7.0-13
smartmontools: 7.2-pve2
spiceterm: 3.2-2
vncterm: 1.7-1
zfsutils-linux: 2.0.5-pve1
Backing up file:
Task viewer: VM/CT 100 - Backup
OutputStatus
Stop
INFO: starting new backup job: vzdump 100 --remove 0 --storage gluster-backup --node bvmVm80 --mode suspend --compress zstd
INFO: Starting Backup of VM 100 (qemu)
INFO: Backup started at 2021-09-10 11:36:42
INFO: status = stopped
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: VM Name: DNS-slave
INFO: include disk 'sata0' 'gluster-image:100/vm-100-disk-0.qcow2' 10G
INFO: creating vzdump archive '/mnt/pve/gluster-backup/dump/vzdump-qemu-100-2021_09_10-11_36_41.vma.zst'
INFO: starting kvm to execute backup task
INFO: started backup task 'eff352ca-b863-4ae8-8c37-6a83c3d9da41'
INFO: 3% (525.6 MiB of 14.0 GiB) in 3s, read: 175.2 MiB/s, write: 163.9 MiB/s
INFO: 6% (909.9 MiB of 14.0 GiB) in 6s, read: 128.1 MiB/s, write: 125.2 MiB/s
INFO: 9% (1.4 GiB of 14.0 GiB) in 9s, read: 169.2 MiB/s, write: 159.9 MiB/s
INFO: 30% (4.3 GiB of 14.0 GiB) in 12s, read: 1001.7 MiB/s, write: 154.0 MiB/s
INFO: 100% (14.0 GiB of 14.0 GiB) in 14s, read: 4.8 GiB/s, write: 22.0 MiB/s
INFO: backup is sparse: 12.19 GiB (87%) total zero data
INFO: transferred 14.00 GiB in 14 seconds (1023.9 MiB/s)
INFO: stopping kvm after backup task
INFO: archive file size: 663MB
INFO: Finished Backup of VM 100 (00:00:23)
INFO: Backup finished at 2021-09-10 11:37:04
INFO: Backup job finished successfully
TASK OK
Restore:
Task viewer: VM 101 - Restore
OutputStatus
Stop
restore vma archive: zstd -q -d -c /mnt/pve/gluster-backup/dump/vzdump-qemu-100-2021_09_10-11_21_14.vma.zst | vma extract -v -r /var/tmp/vzdumptmp2485.fifo - /var/tmp/vzdumptmp2485
CFG: size: 390 name: qemu-server.conf
DEV: dev_id=1 size: 15030747136 devname: drive-sata0
CTIME: Fri Sep 10 11:21:15 2021
Formatting 'gluster://10.8.8.241/bvm_vmimage/images/101/vm-101-disk-0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=metadata compression_type=zlib size=15030747136 lazy_refcounts=off refcount_bits=16
[2021-09-10 09:35:13.414971 +0000] I [io-stats.c:3706:ios_sample_buf_size_configure] 0-bvm_vmimage: Configure ios_sample_buf size is 1024 because ios_sample_interval is 0
[2021-09-10 09:35:13.573057 +0000] E [MSGID: 108006] [afr-common.c:6140:__afr_handle_child_down_event] 0-bvm_vmimage-replicate-0: All subvolumes are down. Going offline until at least one of them comes back up.
no lock found trying to remove 'create' lock
error before or during data restore, some or all disks were not completely restored. VM 101 state is NOT cleaned up.
TASK ERROR: command 'set -o pipefail && zstd -q -d -c /mnt/pve/gluster-backup/dump/vzdump-qemu-100-2021_09_10-11_21_14.vma.zst | vma extract -v -r /var/tmp/vzdumptmp2485.fifo - /var/tmp/vzdumptmp2485' failed: unable to create image: 'storage-gluster-image'-locked command timed out - aborting