Migration failed - changing feature flags is only allowed for root@pam

P

pr0j3ctx

Renowned Member
May 4, 2016
9
0
66
44
Hello,

today I tried the new Datacenter Manager. It looks very good and seems to work.

I`ve tested migration. Basically it works great. I migrated the running Datacenter Manager VM to another Remote. No lag. Great!

But If i try to migrate some Containers with features like nfs or something else. I get this message.

2025-01-29 15:04:31 ERROR: migration aborted (duration 00:00:05): error - tunnel command '{"firewall-config":null,"cmd":"config","conf":"arch: amd64\ncores: 2\nfeatures: nesting=0\nhostname: testwan1\nlock: migrate\nmemory: 1024\nnameserver: 9.9.9.9\nnet0: name=eth0,bridge=vmbr0,gw=10.0.21.2,hwaddr=0A:00:00:21:00:10,ip=10.0.21.10/24,ip6=dhcp,link_down=1,tag=21,type=veth\nonboot: 1\nostype: ubuntu\nrootfs: prxpool:subvol-21010-disk-0,size=4G\nsearchdomain: dlan.site\nswap: 0\nunprivileged: 1\n"}' failed - failed to handle 'config' command - 403 Permission check failed (changing feature flags (except nesting) is only allowed for root@pam)
Click to expand...

I added the remotes with root login. But it seems, that the tunnel uses another user?

Is this an issue or i am doing something wrong?

Greats pr0
 
smueller said:
Hi @pr0j3ctx,

you did nothing wrong, could you please share your CT config and Logs of the Migration?
Did you change the storage destination to a existing one?
Click to expand...
Good morning,

I´ve used detailed mapping and configured the correct storage and network.

Code: 
2025-01-30 06:13:13 remote: started tunnel worker 'UPID:kvm12:00053CA2:0054A67B:679B0A69:vzmtunnel:21010:root@pam!pdm-admin:'
tunnel: -> sending command "version" to remote
tunnel: <- got reply
2025-01-30 06:13:13 local WS tunnel version: 2
2025-01-30 06:13:13 remote WS tunnel version: 2
2025-01-30 06:13:13 minimum required WS tunnel version: 2
2025-01-30 06:13:13 websocket tunnel started
2025-01-30 06:13:13 shutdown CT 21010
2025-01-30 06:13:14 starting migration of CT 21010 to node 'kvm12' (192.168.5.12)
tunnel: -> sending command "bwlimit" to remote
tunnel: <- got reply
2025-01-30 06:13:14 found local volume 'prxpool:subvol-21010-disk-0' (in current VM config)
tunnel: -> sending command "disk-import" to remote
tunnel: <- got reply
tunnel: accepted new connection on '/run/pve/21010.storage'
tunnel: requesting WS ticket via tunnel
tunnel: established new WS for forwarding '/run/pve/21010.storage'
full send of prxpool/subvol-21010-disk-0@__migration__ estimated size is 1.00G
total estimated size is 1.00G
TIME        SENT   SNAPSHOT prxpool/subvol-21010-disk-0@__migration__
tunnel: -> sending command "query-disk-import" to remote
tunnel: done handling forwarded connection from '/run/pve/21010.storage'
tunnel: <- got reply
2025-01-30 06:13:16 volume 'prxpool:subvol-21010-disk-0' is 'prxpool:subvol-21010-disk-0' on the target
2025-01-30 06:13:16 mapped: net0 from vmbr0 to vmbr0
tunnel: -> sending command "config" to remote
tunnel: <- got reply
2025-01-30 06:13:16 ERROR: error - tunnel command '{"firewall-config":null,"cmd":"config","conf":"arch: amd64\ncores: 2\nfeatures: nesting=0\nhostname: testwan1\nlock: migrate\nmemory: 1024\nnameserver: 9.9.9.9\nnet0: name=eth0,bridge=vmbr0,gw=10.0.21.2,hwaddr=0A:00:00:21:00:10,ip=10.0.21.10/24,ip6=dhcp,link_down=1,tag=21,type=veth\nonboot: 1\nostype: ubuntu\nrootfs: prxpool:subvol-21010-disk-0,size=4G\nsearchdomain: xxx.site\nswap: 0\nunprivileged: 1\n"}' failed - failed to handle 'config' command - 403 Permission check failed (changing feature flags (except nesting) is only allowed for root@pam)
2025-01-30 06:13:16 aborting phase 1 - cleanup resources
2025-01-30 06:13:16 ERROR: found stale volume copy 'prxpool:subvol-21010-disk-0' on node 'kvm12'
tunnel: -> sending command "quit" to remote
tunnel: <- got reply
2025-01-30 06:13:17 start final cleanup
2025-01-30 06:13:17 start container on source node
2025-01-30 06:13:18 ERROR: migration aborted (duration 00:00:05): error - tunnel command '{"firewall-config":null,"cmd":"config","conf":"arch: amd64\ncores: 2\nfeatures: nesting=0\nhostname: testwan1\nlock: migrate\nmemory: 1024\nnameserver: 9.9.9.9\nnet0: name=eth0,bridge=vmbr0,gw=10.0.21.2,hwaddr=0A:00:00:21:00:10,ip=10.0.21.10/24,ip6=dhcp,link_down=1,tag=21,type=veth\nonboot: 1\nostype: ubuntu\nrootfs: prxpool:subvol-21010-disk-0,size=4G\nsearchdomain: xxx.site\nswap: 0\nunprivileged: 1\n"}' failed - failed to handle 'config' command - 403 Permission check failed (changing feature flags (except nesting) is only allowed for root@pam)
TASK ERROR: migration aborted

Code: 
arch: amd64
cores: 2
features: nesting=0
hostname: testwan1
memory: 1024
nameserver: 9.9.9.9
net0: name=eth0,bridge=vmbr0,gw=10.0.21.2,hwaddr=0A:00:00:21:00:10,ip=10.0.21.10/24,ip6=dhcp,link_down=1,tag=21,type=veth
onboot: 1
ostype: ubuntu
rootfs: prxpool:subvol-21010-disk-0,size=4G
searchdomain: xxx.site
swap: 0
unprivileged: 1

The logs says thats only allowed except nesting . But onliy nesting is used. Maybe nesting=0 is not needed and could be deleted?

Greats pr0
 
For my understanding:
You have created the remotes with their appropriate root user, if this is not the case I would try that again with the root user.
I have also read that the migration is carried out with the same user (via API token).

I would also look at your target, are there any sings of your CT?
Volumes and Configs? If so try to delete those :)
 
  • Like
Reactions: smueller and andyjayh
I had nesting=1 but did not work. BUT, from some other unrelated post, I unchecked the keyctl option on the LXC today and then the migration worked. Looking at that patch, I guess I would have bypassed that part of the code. Which PDM release will this be coming in?
 
Last edited:
ggee said:
I had nesting=1 but did not work. BUT, from some other unrelated post, I unchecked the keyctl option on the LXC today and then the migration worked. Looking at that patch, I guess I would have bypassed that part of the code. Which PDM release will this be coming in?
Click to expand...
For setting keyctl you still need to be the root user for security reasons. You can unset the feature, do the migration and then set the feature again. The stuff with nesting is not a PDM issue, the fix is included in PVE package pve-container >= 5.2.4 currently available on the pve testing repository.
 
fiona said:
For setting keyctl you still need to be the root user for security reasons. You can unset the feature, do the migration and then set the feature again. The stuff with nesting is not a PDM issue, the fix is included in PVE package pve-container >= 5.2.4 currently available on the pve testing repository.
Click to expand...
I am running into this issue. Not sure what you mean by the root user, I am the root@pam when running the migration. I also installed the pve-container >= 5.2.4 and am still running into the same issues.
Code: 
2025-02-13 12:20:38 remote: started tunnel worker 'UPID:prxmx2:0000C029:000A5C22:67AE29E6:vzmtunnel:102:root@pam!pdm-admin:'
tunnel: -> sending command "version" to remote
tunnel: <- got reply
2025-02-13 12:20:38 local WS tunnel version: 2
2025-02-13 12:20:38 remote WS tunnel version: 2
2025-02-13 12:20:38 minimum required WS tunnel version: 2
2025-02-13 12:20:38 websocket tunnel started
2025-02-13 12:20:38 shutdown CT 102
2025-02-13 12:20:40 starting migration of CT 102 to node 'prxmx2' (192.168.150.3)
tunnel: -> sending command "bwlimit" to remote
tunnel: <- got reply
2025-02-13 12:20:40 found local volume 'NVME_Data_0:subvol-102-disk-0' (in current VM config)
tunnel: -> sending command "disk-import" to remote
tunnel: <- got reply
tunnel: accepted new connection on '/run/pve/102.storage'
tunnel: requesting WS ticket via tunnel
tunnel: established new WS for forwarding '/run/pve/102.storage'
full send of NVME_Data_0/subvol-102-disk-0@__migration__ estimated size is 774M
total estimated size is 774M
TIME        SENT   SNAPSHOT NVME_Data_0/subvol-102-disk-0@__migration__
tunnel: -> sending command "query-disk-import" to remote
tunnel: done handling forwarded connection from '/run/pve/102.storage'
tunnel: <- got reply
2025-02-13 12:20:44 volume 'NVME_Data_0:subvol-102-disk-0' is 'ZFS_1TB:subvol-102-disk-0' on the target
2025-02-13 12:20:44 mapped: net0 from vmbr1 to vmbr1
tunnel: -> sending command "config" to remote
tunnel: <- got reply
2025-02-13 12:20:44 ERROR: error - tunnel command '{"conf":"#http%3A//192.168.255.225%3A8090\n#\n#<div align='center'>\n#  <a href='https%3A//Helper-Scripts.com' target='_blank' rel='noopener noreferrer'>\n#    <img src='https%3A//raw.githubusercontent.com/community-scripts/ProxmoxVE/main/misc/images/logo-81x112.png' alt='Logo' style='width%3A81px;height%3A112px;'/>\n#  </a>\n#\n#  <h2 style='font-size%3A 24px; margin%3A 20px 0;'>Beszel LXC</h2>\n#\n#  <p style='margin%3A 16px 0;'>\n#    <a href='https%3A//ko-fi.com/community_scripts' target='_blank' rel='noopener noreferrer'>\n#      <img src='https%3A//img.shields.io/badge/&#x2615;-Buy us a coffee-blue' alt='spend Coffee' />\n#    </a>\n#  </p>\n#  \n#  <span style='margin%3A 0 10px;'>\n#    <i class=\"fa fa-github fa-fw\" style=\"color%3A #f5f5f5;\"></i>\n#    <a href='https%3A//github.com/community-scripts/ProxmoxVE' target='_blank' rel='noopener noreferrer' style='text-decoration%3A none; color%3A #00617f;'>GitHub</a>\n#  </span>\n#  <span style='margin%3A 0 10px;'>\n#    <i class=\"fa fa-comments fa-fw\" style=\"color%3A #f5f5f5;\"></i>\n#    <a href='https%3A//github.com/community-scripts/ProxmoxVE/discussions' target='_blank' rel='noopener noreferrer' style='text-decoration%3A none; color%3A #00617f;'>Discussions</a>\n#  </span>\n#  <span style='margin%3A 0 10px;'>\n#    <i class=\"fa fa-exclamation-circle fa-fw\" style=\"color%3A #f5f5f5;\"></i>\n#    <a href='https%3A//github.com/community-scripts/ProxmoxVE/issues' target='_blank' rel='noopener noreferrer' style='text-decoration%3A none; color%3A #00617f;'>Issues</a>\n#  </span>\n#</div>\narch: amd64\ncores: 1\nfeatures: keyctl=1,nesting=1\nhostname: beszel-lxc\nlock: migrate\nmemory: 512\nnet0: name=eth0,bridge=vmbr1,hwaddr=BC:24:11:CA:48:2F,ip=dhcp,tag=255,type=veth\nonboot: 1\nostype: debian\nrootfs: ZFS_1TB:subvol-102-disk-0,size=5G\nsearchdomain: dmz.local\nswap: 512\ntags: community-script;monitoring\nunprivileged: 1\n","firewall-config":null,"cmd":"config"}' failed - failed to handle 'config' command - 403 Permission check failed (changing feature flags (except nesting) is only allowed for root@pam)
2025-02-13 12:20:44 aborting phase 1 - cleanup resources
2025-02-13 12:20:44 ERROR: found stale volume copy 'ZFS_1TB:subvol-102-disk-0' on node 'prxmx2'
tunnel: -> sending command "quit" to remote
tunnel: <- got reply
2025-02-13 12:20:46 start final cleanup
2025-02-13 12:20:46 start container on source node
2025-02-13 12:20:47 ERROR: migration aborted (duration 00:00:09): error - tunnel command '{"conf":"#http%3A//192.168.255.225%3A8090\n#\n#<div align='center'>\n#  <a href='https%3A//Helper-Scripts.com' target='_blank' rel='noopener noreferrer'>\n#    <img src='https%3A//raw.githubusercontent.com/community-scripts/ProxmoxVE/main/misc/images/logo-81x112.png' alt='Logo' style='width%3A81px;height%3A112px;'/>\n#  </a>\n#\n#  <h2 style='font-size%3A 24px; margin%3A 20px 0;'>Beszel LXC</h2>\n#\n#  <p style='margin%3A 16px 0;'>\n#    <a href='https%3A//ko-fi.com/community_scripts' target='_blank' rel='noopener noreferrer'>\n#      <img src='https%3A//img.shields.io/badge/&#x2615;-Buy us a coffee-blue' alt='spend Coffee' />\n#    </a>\n#  </p>\n#  \n#  <span style='margin%3A 0 10px;'>\n#    <i class=\"fa fa-github fa-fw\" style=\"color%3A #f5f5f5;\"></i>\n#    <a href='https%3A//github.com/community-scripts/ProxmoxVE' target='_blank' rel='noopener noreferrer' style='text-decoration%3A none; color%3A #00617f;'>GitHub</a>\n#  </span>\n#  <span style='margin%3A 0 10px;'>\n#    <i class=\"fa fa-comments fa-fw\" style=\"color%3A #f5f5f5;\"></i>\n#    <a href='https%3A//github.com/community-scripts/ProxmoxVE/discussions' target='_blank' rel='noopener noreferrer' style='text-decoration%3A none; color%3A #00617f;'>Discussions</a>\n#  </span>\n#  <span style='margin%3A 0 10px;'>\n#    <i class=\"fa fa-exclamation-circle fa-fw\" style=\"color%3A #f5f5f5;\"></i>\n#    <a href='https%3A//github.com/community-scripts/ProxmoxVE/issues' target='_blank' rel='noopener noreferrer' style='text-decoration%3A none; color%3A #00617f;'>Issues</a>\n#  </span>\n#</div>\narch: amd64\ncores: 1\nfeatures: keyctl=1,nesting=1\nhostname: beszel-lxc\nlock: migrate\nmemory: 512\nnet0: name=eth0,bridge=vmbr1,hwaddr=BC:24:11:CA:48:2F,ip=dhcp,tag=255,type=veth\nonboot: 1\nostype: debian\nrootfs: ZFS_1TB:subvol-102-disk-0,size=5G\nsearchdomain: dmz.local\nswap: 512\ntags: community-script;monitoring\nunprivileged: 1\n","firewall-config":null,"cmd":"config"}' failed - failed to handle 'config' command - 403 Permission check failed (changing feature flags (except nesting) is only allowed for root@pam)
TASK ERROR: migration aborted
 
Hi,
crumb4life said:
I am running into this issue. Not sure what you mean by the root user, I am the root@pam when running the migration. I also installed the pve-container >= 5.2.4 and am still running into the same issues.
Click to expand...
the remote migration happens with an API token which has more limited privileges than root@pam. Because of security implications, you will need to (temporarily) unset the keyctl feature for the remote migration to work. In general, it is recommended to set up docker in a VM in Proxmox VE, not in an LXC. (Just search the forum a bit and you will quickly find issues, the docs also mention it near the beginning: https://pve.proxmox.com/pve-docs/chapter-pct.html).
 
You must log in or register to reply here.
Top Bottom