Cluster Join failed This host already contains virtual guests

[DocSpiegel]

Hello all, I will keep this brief and to the point. I have moved my cluster to a permanent home on a new network and am recreating my cluster. In order to do so I followed the process of removing corosync.conf and deleting all the config files. I also removed all ceph config and all vm's are removed from all nodes.

I can create a cluster fine, but whenever I attempt to join a node to it I get the error 'this host already contains virtual guests' but they are all gone.

Any ideas?

 

[mira]

Are there any files left in '/etc/pve/qemu-server' or '/etc/pve/lxc' on the node you want to join?

 

[DocSpiegel]

Are there any files left in '/etc/pve/qemu-server' or '/etc/pve/lxc' on the node you want to join?

In /etc/pve/nodes/ I found that there was a directory for each of the old nodes(i.e. /etc/pve/nodes/node1/qemu-server/ and inside found the vm .conf files i.e. 101.conf. I removed those and then the join worked properly.

Thank you for the quick response and help!

 

[SwaggerRO19]

I have the same issue. I removed the 101.conf and 102.conf and when I try again to add the server to the cluster I receive the same error * this host already contains virtual guests

Any ideeas?

 

[mira]

Please post the output of ls /etc/pve/qemu-server and ls /etc/pve/lxc as well as ls /etc/pve/nodes

 

[SwaggerRO19]

@mira
The server that will join the cluster:

root@pmx2:~# ls /etc/pve/qemu-server root@pmx2:~# ls /etc/pve/lxc 100.conf root@pmx2:~# ls /etc/pve/nodes pmx2 root@pmx2:~#

The server that hosts the cluster:
root@proxmox:~# ls /etc/pve/qemu-server root@proxmox:~# ls /etc/pve/lxc 101.conf 102.conf 105.conf 106.conf 107.conf 108.conf 110.conf root@proxmox:~# ls /etc/pve/nodes proxmox root@proxmox:~#

 

[mira]

You already have a container on that node. If you want to keep it, make a backup. Then delete the container, join the node to the cluster and restore the container.

 

[SwaggerRO19]

@mira may I ask on what node? On the pmx2 one? Or on the host node.

 

[mira]

On the one you want to join, as the error message mentions. You can't join a node to a cluster if the node has guests.

 

[wifi75]

hello I have some problem,

I have two node the first have a lot VM and the host name is Proxmox
On the proxmox host I have make a cluster.
now I'd like to join another node named proxmox2 but when i try there is error:
* this host already contains virtual guests


below result of command:

root@proxmox:~# ls /etc/pve/qemu-server 100.conf 101.conf 102.conf 103.conf 104.conf 105.conf 106.conf 110.conf 112.conf root@proxmox:~# ls /etc/pve/lxc root@proxmox:~# ls /etc/pve/nodes proxmox root@proxmox:~#


root@proxmox2:~# ls /etc/pve/qemu-server root@proxmox2:~# ls /etc/pve/lxc root@proxmox2:~# ls /etc/pve/nodes proxmox2 root@proxmox2:~#

what can I d for fix ?

 

[mira]

You can backup all your VMs on that node. Afterwards remove them from the node and join the node to the cluster. Once everything is working you can restore the VMs on the node.
Please make sure the backups are working before you delete the VMs.

 

[wifi75]

each night node backs up to nas

 

[voarsh]

On the one you want to join, as the error message mentions. You can't join a node to a cluster if the node has guests.

This is a bad design. Consider making it possible to join when guests exist on the joining node.

 

[thiagotgc]

This is a bad design. Consider making it possible to join when guests exist on the joining node.

I also don't understand why this limitation.

Will this ever be overcome?

 

[thiagotgc]

News?

 

[mira]

No, this won't be changed.
One of the reasons why this limitation exists, is that there can be a conflict of VM/CT IDs.

 

[voarsh]

No, this won't be changed.
One of the reasons why this limitation exists, is that there can be a conflict of VM/CT IDs.

yes, I know this, but PVE team can compare what's in the cluster and what's in the new joining node, rename VM/CT IDs and disk names to not be what's in the cluster, so when joining it's unique.

Only workaround is to backup every VM/CT and restore after joining (oh, and dealing with VM/CT IDs)

 

[mokaz]

That's what i did to sort this out:

- stop all the VMs on the joining node
- move the VMs descriptors within /etc/pve/nodes/[NODENAME]/qemu-server to /home/_cluster_bck
- join the cluster
- copy the backup VMs descriptors back to /etc/pve/nodes/[NODENAME]/qemu-server

done =)

 

[fabian]

yes, I know this, but PVE team can compare what's in the cluster and what's in the new joining node, rename VM/CT IDs and disk names to not be what's in the cluster, so when joining it's unique.

Only workaround is to backup every VM/CT and restore after joining (oh, and dealing with VM/CT IDs)

it's not that easy - changing guest IDs involves storage operations, storage config entries can also conflict, as could stuff like CPU models, firewall groups, backup jobs - and that's just from the top of my head without actually thinking through all the corner cases that might pop up.

you can already trivially work around this limitation manually if you know what you are doing (backup relevant content from /etc/pve, remove guest configs, join, restore content as needed), but there is so much that can go wrong that this will not be automated. 99% of the time a node is freshly installed, then joined to an existing cluster while empty.

 

[mokaz]

Sure thing Fabian, I wouldn't do this in production that's for sure.. But see here I've just made some stupid mistakes (rm -r /etc/pve/nodes/) while reconfiguring the cluster etc and well all the VMs were locally hosted or on NFS.. And I needed a few VMs to control underneath hardware, hence not much of a choice... in my home lab setup this hasn't gave me any issues really..

And sure thing, backup, backup backup..