Proxmox Cant remove image after online migration

[paul_b]

Hi 2 all!

I have a problem, Proxmox Cant remove image after migration.

pveversion -v

proxmox-ve: 6.2-2 (running kernel: 5.4.65-1-pve)
pve-manager: 6.2-15 (running version: 6.2-15/48bd51b6)
pve-kernel-5.4: 6.2-7
pve-kernel-helper: 6.2-7
pve-kernel-5.3: 6.1-6
pve-kernel-5.0: 6.0-11
pve-kernel-5.4.65-1-pve: 5.4.65-1
pve-kernel-5.3.18-3-pve: 5.3.18-3
pve-kernel-5.3.13-1-pve: 5.3.13-1
pve-kernel-5.0.21-5-pve: 5.0.21-10
ceph-fuse: 12.2.13-pve1
corosync: 3.0.4-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 3.0.0-1+pve3
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.16-pve1
libproxmox-acme-perl: 1.0.5
libpve-access-control: 6.1-3
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.2-4
libpve-guest-common-perl: 3.1-3
libpve-http-server-perl: 3.0-6
libpve-storage-perl: 6.2-9
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.3-1
lxcfs: 4.0.3-pve3
novnc-pve: 1.1.0-1
openvswitch-switch: 2.10.0+2018.08.28+git.8ca7c82b7d+ds1-12+deb10u1
proxmox-backup-client: 0.9.7-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.3-8
pve-cluster: 6.2-1
pve-container: 3.2-2
pve-docs: 6.2-6
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-3
pve-firmware: 3.1-3
pve-ha-manager: 3.1-1
pve-i18n: 2.2-2
pve-qemu-kvm: 5.1.0-6
pve-xtermjs: 4.7.0-2
qemu-server: 6.2-19
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 0.8.4-pve2

When I do an online migration from thin to thin, after the migration the disk can't be deleted:

2021-04-27 09:04:15 stopping NBD storage migration server on target.
2021-04-27 09:04:20 ERROR: removing local copy of 'thin0:vm-520002-disk-0' failed - lvremove 'VGLOCAL/vm-520002-disk-0' error:   Logical volume VGLOCAL/vm-520002-disk-0 is used by another device.
2021-04-27 09:04:21 ERROR: migration finished with problems (duration 00:01:25)
TASK ERROR: migration problems

lvremove does not give a result:
Logical volume VGLOCAL/vm-520002-disk-0 is used by another device.

lsblk shows that each image is divided into many parts:

└─VGLOCAL-TP0_tdata                      253:1    0    19T  0 lvm
  └─VGLOCAL-TP0-tpool                    253:2    0    19T  0 lvm
    ├─VGLOCAL-TP0                        253:3    0    19T  0 lvm
    ├─VGLOCAL-vm--20022--disk--0         253:5    0    64G  0 lvm
    │ ├─VGLOCAL-vm--20022--disk--0p1     253:201  0   350M  0 part
    │ └─VGLOCAL-vm--20022--disk--0p2     253:202  0  63.7G  0 part
    ├─VGLOCAL-vm--101--disk--0           253:6    0    20G  0 lvm
    │ ├─VGLOCAL-vm--101--disk--0p1       253:163  0   476M  0 part
    │ ├─VGLOCAL-vm--101--disk--0p2       253:164  0   954M  0 part
    │ └─VGLOCAL-vm--101--disk--0p3       253:165  0  18.6G  0 part
    ├─VGLOCAL-vm--102--cloudinit         253:7    0     4M  0 lvm
    ├─VGLOCAL-vm--9518--disk--0          253:8    0    80G  0 lvm
    │ ├─VGLOCAL-vm--9518--disk--0p1      253:160  0   476M  0 part
    │ ├─VGLOCAL-vm--9518--disk--0p2      253:161  0   954M  0 part
    │ └─VGLOCAL-vm--9518--disk--0p3      253:162  0  78.6G  0 part
    ├─VGLOCAL-vm--102--disk--0           253:9    0    20G  0 lvm
    │ ├─VGLOCAL-vm--102--disk--0p1       253:275  0   476M  0 part
    │ ├─VGLOCAL-vm--102--disk--0p2       253:276  0  18.6G  0 part
    │ └─VGLOCAL-vm--102--disk--0p3       253:277  0   953M  0 part
    ├─VGLOCAL-vm--20022--disk--1         253:10   0   740G  0 lvm
    │ └─VGLOCAL-vm--20022--disk--1p1     253:200  0   740G  0 part
    ├─VGLOCAL-vm--2020010--disk--0       253:11   0    65G  0 lvm
    │ ├─VGLOCAL-vm--2020010--disk--0p1   253:122  0   579M  0 part
    │ ├─VGLOCAL-vm--2020010--disk--0p2   253:123  0    64G  0 part
    │ └─VGLOCAL-vm--2020010--disk--0p3   253:124  0   513M  0 part
    ├─VGLOCAL-vm--9799--disk--0          253:12   0    20G  0 lvm
    │ ├─VGLOCAL-vm--9799--disk--0p1      253:157  0   476M  0 part
    │ ├─VGLOCAL-vm--9799--disk--0p2      253:158  0   954M  0 part
    │ └─VGLOCAL-vm--9799--disk--0p3      253:159  0  18.6G  0 part
    ├─VGLOCAL-vm--1820001--disk--0       253:13   0    20G  0 lvm
    ├─VGLOCAL-base--9409--disk--0        253:15   0   100G  1 lvm
    ├─VGLOCAL-vm--9804--disk--0          253:17   0    20G  0 lvm
    │ ├─VGLOCAL-vm--9804--disk--0p1      253:109  0   476M  0 part
    │ ├─VGLOCAL-vm--9804--disk--0p2      253:110  0   954M  0 part
    │ └─VGLOCAL-vm--9804--disk--0p3      253:111  0  18.6G  0 part

and I can delete the image after the migration if I do
dmsetup remove /dev/dm-{part-id}

I have 7 nodes in a cluster with the same configuration, but this problem with parts appeared only on one of them. On other nodes, lsblk shows only full images without parts, and there is no such problem.

Please tell me how I can fix this? Thanx

 

[paul_b]

I also noticed that there are no parts on the newly created VMs, and if I do a migration to this host and back, the migration goes fine and the disk is deleted. That is, the splitting into parts occurred earlier at some time and I do not yet know how to fix it. Need to understand the reason for this behavior