The question: I see from the documentation at https://pve.proxmox.com/wiki/Cluster_Manager that "All existing configuration in /etc/pve is overwritten when joining a cluster. In particular, a joining node cannot hold any guests, since guest IDs could otherwise conflict, and the node will inherit the cluster’s storage configuration."
I have a node with a very large (3.2TB) guest, and I need to add the node to our main cluster. I know I could back it up, join, and then restore it however that's going to cause a significant outage, so am wondering if doing this would work - note the guest's VIMID is unique and won't conflict with anything in the cluster.
1. Stop the guest.
2. Remove its config file from /etc/pve/qemu-server, after copying and pasting it to Notepad here on my command machine first.
3. Join the node to the cluster.
4. Recreate the config file in /etc/pve/qemu-server, which should cause it to reappear in the GUI
5. Boot the guest.
Would this work?
Full details:
We had a three node cluster, two large SuperMicro machines with 50TB of disk and over 300 gig of memory each, and a smaller blade node with 10TB of disk and 94 gig of memory.
We were upgrading the whole cluster to v7.4-16 (from 7.0-11) in preparation for moving to v8.
The blade server was upgraded with no problem, and rebooted.
Then attempted to upgrade the first of the large machines, initial upgrade went OK but when we rebooted it, the boot SSD failed, node lost.
I've now rebuilt it (with hardware raid1 boot disks, after attempting to use ZFS raidz1 boot proved impossible), and managed to re-import its main ZFS pool, tank1, which meant that we were able to get the large guest running on it by copying the VM's config file back from our backups - as well as backing up the VMs, we also back up the /etc/pve/qemu-server directories for each machine so we have the configs. Placing the config file in /etc/pve/qemu-server made the guest appear in the GUI and it then booted normally in response to the start command.
I've removed the old version of this node from the cluster using pvecm delnode and then deleting its directory structure under /etc/pve/nodes/<name>, so now I want to re-add it to the cluster, so that I can start migrating production VMs off the second large machine to it, in order to then upgrade that host to 7.4-16, ready to upgrade the whole cluster to v8 in a week or two.
The very large guest is a virtual XenServer running multiple guests of its own, hence the huge disk allocation. Long term we plan to replace these Xen VMs with new hosts built directly on Proxmox, and abolish the virtual hypervisor, but for now they are needed, and we'd prefer to avoid excessive downtime.
I have a node with a very large (3.2TB) guest, and I need to add the node to our main cluster. I know I could back it up, join, and then restore it however that's going to cause a significant outage, so am wondering if doing this would work - note the guest's VIMID is unique and won't conflict with anything in the cluster.
1. Stop the guest.
2. Remove its config file from /etc/pve/qemu-server, after copying and pasting it to Notepad here on my command machine first.
3. Join the node to the cluster.
4. Recreate the config file in /etc/pve/qemu-server, which should cause it to reappear in the GUI
5. Boot the guest.
Would this work?
Full details:
We had a three node cluster, two large SuperMicro machines with 50TB of disk and over 300 gig of memory each, and a smaller blade node with 10TB of disk and 94 gig of memory.
We were upgrading the whole cluster to v7.4-16 (from 7.0-11) in preparation for moving to v8.
The blade server was upgraded with no problem, and rebooted.
Then attempted to upgrade the first of the large machines, initial upgrade went OK but when we rebooted it, the boot SSD failed, node lost.
I've now rebuilt it (with hardware raid1 boot disks, after attempting to use ZFS raidz1 boot proved impossible), and managed to re-import its main ZFS pool, tank1, which meant that we were able to get the large guest running on it by copying the VM's config file back from our backups - as well as backing up the VMs, we also back up the /etc/pve/qemu-server directories for each machine so we have the configs. Placing the config file in /etc/pve/qemu-server made the guest appear in the GUI and it then booted normally in response to the start command.
I've removed the old version of this node from the cluster using pvecm delnode and then deleting its directory structure under /etc/pve/nodes/<name>, so now I want to re-add it to the cluster, so that I can start migrating production VMs off the second large machine to it, in order to then upgrade that host to 7.4-16, ready to upgrade the whole cluster to v8 in a week or two.
The very large guest is a virtual XenServer running multiple guests of its own, hence the huge disk allocation. Long term we plan to replace these Xen VMs with new hosts built directly on Proxmox, and abolish the virtual hypervisor, but for now they are needed, and we'd prefer to avoid excessive downtime.
