Did I ruin my cluster?

U

uSlackr

New Member
Nov 26, 2024
2
0
1
I have a 5 node pve cluster. I just added the 5th node yesterday. Today while looking at storage I noticed that the 500gb disk was only showing 325GB for vm space. Did some searching and wound up on NetworkChucks video where he has you remove the local-lvm volume and resize the local. I did that without issue. Perhaps I was hasty.

I then tried moving a vm to that node and get this error:
```
Code: 
2025-10-31 11:42:18 found local volume 'local-lvm:vm-103-disk-0' (in current VM config)
2025-10-31 11:42:19 can't activate LV 'pve/vm-103-disk-0' to zero-out its data:   Failed to find logical volume "pve/vm-103-disk-0"
2025-10-31 11:42:19 not enough free space (0 < 16777216)
2025-10-31 11:42:19 command 'dd 'if=/dev/pve/vm-103-disk-0' 'bs=64k' 'status=progress'' failed: got signal 13
2025-10-31 11:42:19 ERROR: storage migration for 'local-lvm:vm-103-disk-0' to storage 'local-lvm' failed - command 'set -o pipefail && pvesm export local-lvm:vm-103-disk-0 raw+size - -with-snapshots 0 | /usr/bin/ssh -e none -o 'BatchMode=yes' -o 'HostKeyAlias=prox5' -o 'UserKnownHostsFile=/etc/pve/nodes/prox5/ssh_known_hosts' -o 'GlobalKnownHostsFile=none' root@192.168.1.2 -- pvesm import local-lvm:vm-103-disk-0 raw+size - -with-snapshots 0 -allow-rename 1' failed: exit code 5
2025-10-31 11:42:19 aborting phase 1 - cleanup resources
2025-10-31 11:42:19 ERROR: found stale volume copy 'local-lvm:vm-103-disk-0' on node 'prox5'
2025-10-31 11:42:19 start final cleanup
2025-10-31 11:42:19 start container on source node
2025-10-31 11:42:21 ERROR: migration aborted (duration 00:00:06): storage migration for 'local-lvm:vm-103-disk-0' to storage 'local-lvm' failed - command 'set -o pipefail && pvesm export local-lvm:vm-103-disk-0 raw+size - -with-snapshots 0 | /usr/bin/ssh -e none -o 'BatchMode=yes' -o 'HostKeyAlias=prox5' -o 'UserKnownHostsFile=/etc/pve/nodes/prox5/ssh_known_hosts' -o 'GlobalKnownHostsFile=none' root@192.168.1.2 -- pvesm import local-lvm:vm-103-disk-0 raw+size - -with-snapshots 0 -allow-rename 1' failed: exit code 5
TASK ERROR: migration aborted
And, in fact, do not appear to be able to move any vm to any node. I did restart one of the nodes and the containers came back.

What can I do to recover? (I do have backups)

Thanks!

\\Greg
 
See vgs, vgdisplay, lvs, lvdisplay on other nodes and create a local-lvm in fivth node again out of your remaining space after your resize of local.
 
You must log in or register to reply here.
Top Bottom