[SOLVED] ERROR: found stale volume copy - but stale volume exists?

D

dekiesel

Member
Apr 30, 2023
91
12
13
Hi,

I am trying to use PDM to move a container from one host g3 to another host proxmox-g4
Code: 
2026-02-25 14:02:54 remote: started tunnel worker 'UPID:proxmox-g4:000018C4:0002971E:699EF2FE:vzmtunnel:103:root@pam!pdm-admin-7379cff5bd13:'
tunnel: -> sending command "version" to remote
tunnel: <- got reply
2026-02-25 14:02:54 local WS tunnel version: 2
2026-02-25 14:02:54 remote WS tunnel version: 2
2026-02-25 14:02:54 minimum required WS tunnel version: 2
2026-02-25 14:02:54 websocket tunnel started
2026-02-25 14:02:54 shutdown CT 103
2026-02-25 14:03:04 starting migration of CT 103 to node 'proxmox-g4' (10.1.100.12)
tunnel: -> sending command "bwlimit" to remote
tunnel: <- got reply
2026-02-25 14:03:04 found local volume 'local-zfs:subvol-103-disk-0' (in current VM config)
tunnel: -> sending command "disk-import" to remote
tunnel: <- got reply
tunnel: accepted new connection on '/run/pve/103.storage'
tunnel: requesting WS ticket via tunnel
tunnel: established new WS for forwarding '/run/pve/103.storage'
full send of rpool/data/subvol-103-disk-0@__migration__ estimated size is 2.61G
total estimated size is 2.61G
TIME        SENT   SNAPSHOT rpool/data/subvol-103-disk-0@__migration__
tunnel: -> sending command "query-disk-import" to remote
tunnel: done handling forwarded connection from '/run/pve/103.storage'
tunnel: <- got reply
2026-02-25 14:03:33 volume 'local-zfs:subvol-103-disk-0' is 'local-zfs:subvol-103-disk-0' on the target
2026-02-25 14:03:33 mapped: net1 from vmbr0 to vmbr0
tunnel: -> sending command "config" to remote
tunnel: <- got reply
2026-02-25 14:03:33 ERROR: error - tunnel command '{"firewall-config":"[OPTIONS]\n\nenable: 1\n\n[RULES]\n\nGROUP services_ssh\nIN ACCEPT -source 10.1.99.2 -p tcp -dport 2222 -log nolog # allow pihole1's gravity sync ssh access\nIN ACCEPT -source +management -p tcp -dport 80 -log nolog # Allow management access to admin interface\nIN ACCEPT -p udp -dport 53 -log nolog # allow udp 53\nIN DROP -p udp -log nolog # drop all udp traffic\nIN DROP -p tcp -log nolog # drop all tcp traffic\n\n","conf":"#SLAVE!!!\n#Alle %C3%84nderungen m%C3%BCssen am Master vorgenommen und mit \"docker exec -it xxxx gravity-sync\" zu diesem slave gepusht werden (falls m%C3%B6glich).\narch: amd64\ncores: 1\nfeatures: keyctl=1,nesting=1\nhostname: pihole2\nlock: migrate\nmemory: 256\nnet1: name=eth101,bridge=vmbr0,firewall=1,gw=10.1.99.1,hwaddr=16:D8:E5:41:13:4D,ip=10.1.99.3/24,tag=101,type=veth\nonboot: 1\nostype: alpine\nrootfs: local-zfs:subvol-103-disk-0,size=3G\nstartup: order=1,up=15\nswap: 0\ntags: git;prod\nunprivileged: 1\n","cmd":"config"}' failed - failed to handle 'config' command - 403 Permission check failed (changing feature flags (except nesting) is only allowed for root@pam)
2026-02-25 14:03:33 aborting phase 1 - cleanup resources
2026-02-25 14:03:33 ERROR: found stale volume copy 'local-zfs:subvol-103-disk-0' on node 'proxmox-g4'
tunnel: -> sending command "quit" to remote
tunnel: <- got reply
2026-02-25 14:03:34 start final cleanup
2026-02-25 14:03:34 start container on source node
2026-02-25 14:03:37 ERROR: migration aborted (duration 00:00:43): error - tunnel command '{"firewall-config":"[OPTIONS]\n\nenable: 1\n\n[RULES]\n\nGROUP services_ssh\nIN ACCEPT -source 10.1.99.2 -p tcp -dport 2222 -log nolog # allow pihole1's gravity sync ssh access\nIN ACCEPT -source +management -p tcp -dport 80 -log nolog # Allow management access to admin interface\nIN ACCEPT -p udp -dport 53 -log nolog # allow udp 53\nIN DROP -p udp -log nolog # drop all udp traffic\nIN DROP -p tcp -log nolog # drop all tcp traffic\n\n","conf":"#SLAVE!!!\n#Alle %C3%84nderungen m%C3%BCssen am Master vorgenommen und mit \"docker exec -it xxxx gravity-sync\" zu diesem slave gepusht werden (falls m%C3%B6glich).\narch: amd64\ncores: 1\nfeatures: keyctl=1,nesting=1\nhostname: pihole2\nlock: migrate\nmemory: 256\nnet1: name=eth101,bridge=vmbr0,firewall=1,gw=10.1.99.1,hwaddr=16:D8:E5:41:13:4D,ip=10.1.99.3/24,tag=101,type=veth\nonboot: 1\nostype: alpine\nrootfs: local-zfs:subvol-103-disk-0,size=3G\nstartup: order=1,up=15\nswap: 0\ntags: git;prod\nunprivileged: 1\n","cmd":"config"}' failed - failed to handle 'config' command - 403 Permission check failed (changing feature flags (except nesting) is only allowed for root@pam)
TASK ERROR: migration aborted

The issue is, that no such subvol exists to the best of my knowledge:
Source node:
Code: 
root@proxmox-g3:~# zfs list|grep 103
rpool/data/subvol-103-disk-0  1.07G  1.93G  1.07G  /rpool/data/subvol-103-disk-0
Target node:
Code: 
root@proxmox-g4:~# zfs list|grep 103
root@proxmox-g4:~#
I also tried pct rescan --vmid 103 but nothing changes.

How can I find the missing volume?
 
Ok, so the issue is, that the keyctl=1 was set on the ct.
I was able to make it work by turning it off before moving the CT and then turning it back on again after.

Is there a way to increase the API tokens rights?
 
You must log in or register to reply here.
Top Bottom
Back