Fleecing - "Orphaned" fleece files

Indy-Admin

New Member
May 9, 2024
2
0
1
Hello all,
So I was testing the new fleece function in my home lab.

During a backup operation of a large VM, I was forced to hit "stop" in the middle of the backup job because the VM itself was performing some un-anticipated write operations that were ultimately going to fill up the storage before the backup job could finish.

The backup job terminated, but left the fleece files in place and they are consuming significant disk capacity. I can see them under local-lvm -> vm Disks tab, but cannot remove them from there.

Is there a command line option to detach and delete these fleece disk files from the VM? The feature seems to be so new that I can't find any existing info.
 
Last edited:
which kind of storage was the fleecing image put on?
can you post the task log? (i could not reproduce here an orphaned fleecing image)


you can remove volumes with

Code:
pvesm free <volume>

so that should also work for the fleecing image
 
Here is the portion of the log covering the affected VM. Fleece was pointed at the first of two LVM-Thin volumes. Storage is SSD mirror on top of hardware Dell raid controller. I cannot remember what filesystem I chose, but it was not ZFS.

INFO: Starting Backup of VM 106 (qemu)
INFO: Backup started at 2024-05-08 07:54:12
INFO: status = running
INFO: VM Name: GCRFS02
INFO: include disk 'scsi1' 'local-lvm:vm-106-disk-1' 200G
INFO: include disk 'scsi2' 'SATA:vm-106-disk-2' 3584G
INFO: include disk 'scsi3' 'SATA:vm-106-disk-3' 500G
INFO: include disk 'efidisk0' 'local-lvm:vm-106-disk-0' 4M
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: skip unused drive 'SATA:vm-106-disk-0' (not included into backup)
INFO: creating Proxmox Backup Server archive 'vm/106/2024-05-08T11:54:12Z'
WARNING: You have not turned on protection against thin pools running out of space.
WARNING: Set activation/thin_pool_autoextend_threshold below 100 to trigger automatic extension of thin pools before they get full.
Logical volume "vm-106-fleece-0" created.
WARNING: Sum of all thin volume sizes (<2.89 TiB) exceeds the size of thin pool pve/data and the size of whole volume group (1.86 TiB).
WARNING: You have not turned on protection against thin pools running out of space.
WARNING: Set activation/thin_pool_autoextend_threshold below 100 to trigger automatic extension of thin pools before they get full.
Logical volume "vm-106-fleece-1" created.
WARNING: Sum of all thin volume sizes (<6.39 TiB) exceeds the size of thin pool pve/data and the size of whole volume group (1.86 TiB).
WARNING: You have not turned on protection against thin pools running out of space.
WARNING: Set activation/thin_pool_autoextend_threshold below 100 to trigger automatic extension of thin pools before they get full.
Logical volume "vm-106-fleece-2" created.
WARNING: Sum of all thin volume sizes (<6.88 TiB) exceeds the size of thin pool pve/data and the size of whole volume group (1.86 TiB).
INFO: drive-scsi1: attaching fleecing image local-lvm:vm-106-fleece-0 to QEMU
INFO: drive-scsi2: attaching fleecing image local-lvm:vm-106-fleece-1 to QEMU
INFO: drive-scsi3: attaching fleecing image local-lvm:vm-106-fleece-2 to QEMU
INFO: issuing guest-agent 'fs-freeze' command
INFO: issuing guest-agent 'fs-thaw' command
INFO: started backup task '2196b3b6-1279-452d-be59-4d77df1a6bf8'
INFO: resuming VM again
INFO: efidisk0: dirty-bitmap status: created new
INFO: scsi1: dirty-bitmap status: created new
INFO: scsi2: dirty-bitmap status: created new
INFO: scsi3: dirty-bitmap status: created new
INFO: 0% (3.7 GiB of 4.2 TiB) in 3s, read: 1.2 GiB/s, write: 32.0 MiB/s
INFO: 1% (43.0 GiB of 4.2 TiB) in 41s, read: 1.0 GiB/s, write: 538.9 KiB/s
INFO: 2% (86.9 GiB of 4.2 TiB) in 1m 16s, read: 1.3 GiB/s, write: 3.2 MiB/s
INFO: 3% (129.3 GiB of 4.2 TiB) in 1m 52s, read: 1.2 GiB/s, write: 4.3 MiB/s
INFO: 4% (172.8 GiB of 4.2 TiB) in 2m 27s, read: 1.2 GiB/s, write: 4.9 MiB/s
INFO: 5% (214.9 GiB of 4.2 TiB) in 3m 2s, read: 1.2 GiB/s, write: 4.8 MiB/s
INFO: 6% (257.3 GiB of 4.2 TiB) in 3m 39s, read: 1.1 GiB/s, write: 4.6 MiB/s
INFO: 7% (300.7 GiB of 4.2 TiB) in 4m 17s, read: 1.1 GiB/s, write: 4.5 MiB/s
INFO: 8% (343.5 GiB of 4.2 TiB) in 4m 53s, read: 1.2 GiB/s, write: 4.8 MiB/s
INFO: 9% (386.6 GiB of 4.2 TiB) in 5m 30s, read: 1.2 GiB/s, write: 4.6 MiB/s
INFO: 10% (429.1 GiB of 4.2 TiB) in 6m 8s, read: 1.1 GiB/s, write: 4.5 MiB/s
INFO: 11% (472.1 GiB of 4.2 TiB) in 6m 44s, read: 1.2 GiB/s, write: 4.7 MiB/s
INFO: 12% (514.1 GiB of 4.2 TiB) in 12m 15s, read: 129.9 MiB/s, write: 43.1 MiB/s
INFO: 13% (556.9 GiB of 4.2 TiB) in 26m 22s, read: 51.8 MiB/s, write: 51.4 MiB/s
INFO: 14% (599.8 GiB of 4.2 TiB) in 39m 13s, read: 56.9 MiB/s, write: 56.7 MiB/s
INFO: 15% (642.6 GiB of 4.2 TiB) in 52m 32s, read: 54.9 MiB/s, write: 54.8 MiB/s
INFO: 16% (685.5 GiB of 4.2 TiB) in 1h 6m 15s, read: 53.3 MiB/s, write: 53.3 MiB/s
INFO: 17% (728.3 GiB of 4.2 TiB) in 1h 20m 47s, read: 50.3 MiB/s, write: 50.2 MiB/s
INFO: 18% (771.2 GiB of 4.2 TiB) in 1h 36m 6s, read: 47.8 MiB/s, write: 47.2 MiB/s
INFO: 19% (814.0 GiB of 4.2 TiB) in 1h 50m 43s, read: 50.1 MiB/s, write: 49.6 MiB/s
INFO: 20% (856.8 GiB of 4.2 TiB) in 2h 3m 58s, read: 55.1 MiB/s, write: 53.7 MiB/s
INFO: 21% (899.7 GiB of 4.2 TiB) in 2h 18m 39s, read: 49.8 MiB/s, write: 48.1 MiB/s
INFO: 22% (942.5 GiB of 4.2 TiB) in 2h 34m 57s, read: 44.8 MiB/s, write: 44.1 MiB/s
INFO: 23% (985.4 GiB of 4.2 TiB) in 2h 50m 55s, read: 45.8 MiB/s, write: 45.4 MiB/s
INFO: 24% (1.0 TiB of 4.2 TiB) in 3h 6m 44s, read: 46.2 MiB/s, write: 45.5 MiB/s
INFO: 25% (1.0 TiB of 4.2 TiB) in 3h 20m 50s, read: 51.9 MiB/s, write: 51.6 MiB/s
INFO: 26% (1.1 TiB of 4.2 TiB) in 3h 35m 40s, read: 49.3 MiB/s, write: 48.9 MiB/s
INFO: 27% (1.1 TiB of 4.2 TiB) in 3h 49m 53s, read: 51.4 MiB/s, write: 51.1 MiB/s
INFO: 28% (1.2 TiB of 4.2 TiB) in 4h 4m 42s, read: 49.4 MiB/s, write: 49.1 MiB/s
INFO: 29% (1.2 TiB of 4.2 TiB) in 4h 17m 53s, read: 55.5 MiB/s, write: 55.4 MiB/s
INFO: 30% (1.3 TiB of 4.2 TiB) in 4h 32m 59s, read: 48.4 MiB/s, write: 48.0 MiB/s
INFO: 31% (1.3 TiB of 4.2 TiB) in 4h 47m 37s, read: 50.0 MiB/s, write: 49.8 MiB/s
INFO: 32% (1.3 TiB of 4.2 TiB) in 5h 1m 32s, read: 52.5 MiB/s, write: 52.4 MiB/s
INFO: 33% (1.4 TiB of 4.2 TiB) in 5h 16m 26s, read: 49.1 MiB/s, write: 48.7 MiB/s
INFO: 34% (1.4 TiB of 4.2 TiB) in 5h 32m 20s, read: 46.0 MiB/s, write: 45.9 MiB/s
INFO: 35% (1.5 TiB of 4.2 TiB) in 5h 50m 53s, read: 39.4 MiB/s, write: 39.1 MiB/s
INFO: 36% (1.5 TiB of 4.2 TiB) in 6h 10m 37s, read: 37.1 MiB/s, write: 37.0 MiB/s
INFO: 37% (1.5 TiB of 4.2 TiB) in 6h 31m 37s, read: 34.8 MiB/s, write: 34.8 MiB/s
INFO: 38% (1.6 TiB of 4.2 TiB) in 6h 53m 16s, read: 33.8 MiB/s, write: 33.7 MiB/s
INFO: 39% (1.6 TiB of 4.2 TiB) in 7h 14m 49s, read: 33.9 MiB/s, write: 33.9 MiB/s
INFO: 40% (1.7 TiB of 4.2 TiB) in 7h 37m 28s, read: 32.3 MiB/s, write: 32.1 MiB/s
INFO: 41% (1.7 TiB of 4.2 TiB) in 8h 1m 41s, read: 30.2 MiB/s, write: 30.1 MiB/s
INFO: 42% (1.8 TiB of 4.2 TiB) in 8h 23m 49s, read: 33.0 MiB/s, write: 33.0 MiB/s
INFO: 43% (1.8 TiB of 4.2 TiB) in 8h 46m 59s, read: 31.6 MiB/s, write: 30.7 MiB/s
INFO: 44% (1.8 TiB of 4.2 TiB) in 9h 11m 29s, read: 29.8 MiB/s, write: 29.7 MiB/s
INFO: 45% (1.9 TiB of 4.2 TiB) in 9h 34m 55s, read: 31.2 MiB/s, write: 30.9 MiB/s
INFO: 46% (1.9 TiB of 4.2 TiB) in 9h 58m 49s, read: 30.6 MiB/s, write: 30.5 MiB/s
INFO: 47% (2.0 TiB of 4.2 TiB) in 10h 24m 12s, read: 28.8 MiB/s, write: 28.8 MiB/s
INFO: 48% (2.0 TiB of 4.2 TiB) in 10h 49m 15s, read: 29.2 MiB/s, write: 28.7 MiB/s
INFO: 49% (2.0 TiB of 4.2 TiB) in 11h 13m 7s, read: 30.7 MiB/s, write: 30.5 MiB/s
INFO: 50% (2.1 TiB of 4.2 TiB) in 11h 38m 6s, read: 29.3 MiB/s, write: 29.1 MiB/s
INFO: 51% (2.1 TiB of 4.2 TiB) in 12h 4m 18s, read: 27.9 MiB/s, write: 27.7 MiB/s
INFO: 52% (2.2 TiB of 4.2 TiB) in 12h 28m 17s, read: 30.5 MiB/s, write: 30.1 MiB/s
INFO: 53% (2.2 TiB of 4.2 TiB) in 12h 51m 10s, read: 32.0 MiB/s, write: 31.2 MiB/s
ERROR: interrupted by signal
INFO: aborting backup job
INFO: resuming VM again
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!