"Insecure dependency in exec" During OSD Destroy

[need2gcm]

I was removing a problematic OSD and had these errors at the end of the command execution before it returned "TASK OK":

command '/sbin/pvremove /dev/sdd' failed: Insecure dependency in exec while running with -T switch at /usr/share/perl/5.28/IPC/Open3.pm line 178.
command '/sbin/pvremove /dev/sdf' failed: Insecure dependency in exec while running with -T switch at /usr/share/perl/5.28/IPC/Open3.pm line 178.

Full log:

destroy OSD osd.10
Remove osd.10 from the CRUSH map
Remove the osd.10 authentication key.
Remove OSD osd.10
--> Zapping: /dev/ceph-3472e276-592f-4ec3-8d4a-ff7edcd5e61d/osd-block-24f58e8e-5862-4eaf-801b-b3b7e3051f5d
--> Unmounting /var/lib/ceph/osd/ceph-10
Running command: /bin/umount -v /var/lib/ceph/osd/ceph-10
 stderr: umount: /var/lib/ceph/osd/ceph-10 unmounted
Running command: /bin/dd if=/dev/zero of=/dev/ceph-3472e276-592f-4ec3-8d4a-ff7edcd5e61d/osd-block-24f58e8e-5862-4eaf-801b-b3b7e3051f5d bs=1M count=10 conv=fsync
 stderr: 10+0 records in
10+0 records out
 stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0906829 s, 116 MB/s
--> Only 1 LV left in VG, will proceed to destroy volume group ceph-3472e276-592f-4ec3-8d4a-ff7edcd5e61d
Running command: /sbin/vgremove -v -f ceph-3472e276-592f-4ec3-8d4a-ff7edcd5e61d
 stderr: Removing ceph--3472e276--592f--4ec3--8d4a--ff7edcd5e61d-osd--block--24f58e8e--5862--4eaf--801b--b3b7e3051f5d (253:8)
 stderr: Archiving volume group "ceph-3472e276-592f-4ec3-8d4a-ff7edcd5e61d" metadata (seqno 19).
 stderr: Releasing logical volume "osd-block-24f58e8e-5862-4eaf-801b-b3b7e3051f5d"
 stderr: Creating volume group backup "/etc/lvm/backup/ceph-3472e276-592f-4ec3-8d4a-ff7edcd5e61d" (seqno 20).
 stdout: Logical volume "osd-block-24f58e8e-5862-4eaf-801b-b3b7e3051f5d" successfully removed
 stderr: Removing physical volume "/dev/sdd" from volume group "ceph-3472e276-592f-4ec3-8d4a-ff7edcd5e61d"
 stdout: Volume group "ceph-3472e276-592f-4ec3-8d4a-ff7edcd5e61d" successfully removed
--> Zapping: /dev/ceph-35b54ad9-d1c0-4ca7-969a-1844b96a54b3/osd-db-f29e2db4-9617-4565-840a-87b523480d0c
Running command: /bin/dd if=/dev/zero of=/dev/ceph-35b54ad9-d1c0-4ca7-969a-1844b96a54b3/osd-db-f29e2db4-9617-4565-840a-87b523480d0c bs=1M count=10 conv=fsync
 stderr: 10+0 records in
10+0 records out
 stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.035031 s, 299 MB/s
--> More than 1 LV left in VG, will proceed to destroy LV only
--> Removing LV because --destroy was given: /dev/ceph-35b54ad9-d1c0-4ca7-969a-1844b96a54b3/osd-db-f29e2db4-9617-4565-840a-87b523480d0c
Running command: /sbin/lvremove -v -f /dev/ceph-35b54ad9-d1c0-4ca7-969a-1844b96a54b3/osd-db-f29e2db4-9617-4565-840a-87b523480d0c
 stdout: Logical volume "osd-db-f29e2db4-9617-4565-840a-87b523480d0c" successfully removed
 stderr: Removing ceph--35b54ad9--d1c0--4ca7--969a--1844b96a54b3-osd--db--f29e2db4--9617--4565--840a--87b523480d0c (253:3)
 stderr: Archiving volume group "ceph-35b54ad9-d1c0-4ca7-969a-1844b96a54b3" metadata (seqno 71).
 stderr: Releasing logical volume "osd-db-f29e2db4-9617-4565-840a-87b523480d0c"
 stderr: Creating volume group backup "/etc/lvm/backup/ceph-35b54ad9-d1c0-4ca7-969a-1844b96a54b3" (seqno 72).
--> Zapping successful for OSD: 10
command '/sbin/pvremove /dev/sdd' failed: Insecure dependency in exec while running with -T switch at /usr/share/perl/5.28/IPC/Open3.pm line 178.
command '/sbin/pvremove /dev/sdf' failed: Insecure dependency in exec while running with -T switch at /usr/share/perl/5.28/IPC/Open3.pm line 178.
TASK OK

Versions:

proxmox-ve: 6.2-2 (running kernel: 5.4.65-1-pve)
pve-manager: 6.2-15 (running version: 6.2-15/48bd51b6)
pve-kernel-5.4: 6.2-7
pve-kernel-helper: 6.2-7
pve-kernel-5.3: 6.1-6
pve-kernel-5.4.65-1-pve: 5.4.65-1
pve-kernel-5.4.60-1-pve: 5.4.60-2
pve-kernel-5.3.18-3-pve: 5.3.18-3
pve-kernel-5.3.10-1-pve: 5.3.10-1
ceph: 14.2.11-pve1
ceph-fuse: 14.2.11-pve1
corosync: 3.0.4-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: not correctly installed
ifupdown2: 3.0.0-1+pve3
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.16-pve1
libproxmox-acme-perl: 1.0.5
libpve-access-control: 6.1-3
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.2-4
libpve-guest-common-perl: 3.1-3
libpve-http-server-perl: 3.0-6
libpve-storage-perl: 6.2-10
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.3-1
lxcfs: 4.0.3-pve3
novnc-pve: 1.1.0-1
proxmox-backup-client: 1.0.1-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.3-10
pve-cluster: 6.2-1
pve-container: 3.2-2
pve-docs: 6.2-6
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-3
pve-firmware: 3.1-3
pve-ha-manager: 3.1-1
pve-i18n: 2.2-2
pve-qemu-kvm: 5.1.0-6
pve-xtermjs: 4.7.0-2
qemu-server: 6.2-19
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 0.8.4-pve2

 

[Stoiko Ivanov]

Thanks for reporting this - I'll try to reproduce it locally.

I assume you removed/destroyed the OSD via the GUI?

 

[need2gcm]

Correct

EDIT: Also for clarity, I had the associated DB/WAL on an SSD.

 

[Stoiko Ivanov]

Thanks - figured that while reproducing

I sent an initial patch for discussion to the pve-devel list:
https://lists.proxmox.com/pipermail/pve-devel/2020-November/046160.html