Mixed Cluster 5.4 / 6.x during upgrade?

[oliwel]

Hi,

I have a two-node cluster with 5.4 with an external shared storage (FibreChannel SAN) and LVM on top

Is it possible to:

1. migrate all VMs to one node
2. upgrade second node
3. migrate VMs from old 5.4 node to 6.0 "online"

If yes, is it also possible to move back from the 6.0 to the 5.4 node in case there is a problem?

Is it wise/possible to migrate directly to 6.1 or better 5.4 -> 6.0 -> 6.1 ?

best regards

Oliver

 

[wolfgang]

Hi,

you can't mix PVE 5 and PVE 6.
The cluster communication protocol is different.
You can copy the config manually to the new Node and start it there.

Use the Upgrade HowTo for the upgrade.
https://pve.proxmox.com/wiki/Upgrade_from_5.x_to_6.0

You will upgrade 5.4->6.x
x is for the current version

 

[spirit]

I disagree,

you can mix pve5+upgraded corosync3 with pve6 nodes, and do live migration from 5->6.

https://pve.proxmox.com/wiki/Upgrade_from_5.x_to_6.0#In-place_upgrade

(you can still migrate back vm to pve5, but it need stop/start of the vm)

 

[oliwel]

Hi Wolfgang,

1. The HowTo gives the hint on corosyn3 as mentioned by spirit, also - if this makes a difference - I do not use HA failover / fencing so "cluster" for me refers only to replication of the config and live migration of the machines via the UI. Is this done by corosync, too or is this done by PVE owned components ( I guess that is the stuff behind pve-ha-lrm) ?

2. If I do not update in cluster mode, how can I "migrate" a CT/VM from the old node to the new one?

Oliver

 

[spirit]

2. If I do not update in cluster mode, how can I "migrate" a CT/VM from the old node to the new one?

backup/restore

 

[oliwel]

backup/restore

With a shared storage? For 6TB of data - I hope you are kidding....

 

[czechsys]

Shutdown vm, move vm config from old node to new node, start vm.

 

[wolfgang]

I will only recommend things that are tested.
Adding a PVE6 node to a PVE 5 Cluster is not tested.
As we have no note that this is working, I would not recommend this way.
But yes, with Linux and PVE, you can do many things because you are not restricted.

2. If I do not update in cluster mode, how can I "migrate" a CT/VM from the old node to the new one?

You add the FC storage at the new server and copy the config with scp from the cluster to the new node.
Important is you only start the VM only on one Node because you have no protection for concurrent storage access.
If the VM is started on more than one node your Vm images are corrupt.
So check twice if the VM is stopped.
The config is located at /etc/pve/node/<nodename>/[lxc|qemu-server]/<VMID>.conf

 

[oliwel]

I will only recommend things that are tested.
Adding a PVE6 node to a PVE 5 Cluster is not tested.

To be honest: it would be nice to have this upgrade path "tested" at least to some extend and I am very confused about the fact that the cluster/corosync update and an "Inline Update" is mentioned in the HowTo.

Bottom Line - as far as I understand this - the correct way is:

1. Migrate all VMs to one node
2. shutdown lrm and crm on bot nodes
3. Upgrade second node
4. manually stop each VM on the old node
5. manually start each VM on the new node (will involve moving the config directory from node1 subdir to node2 subdir)
6. upgrade old node
7. restart crm and lrm

Oliver

 

[tom]

To be honest: it would be nice to have this upgrade path "tested" at least to some extend and I am very confused about the fact that the cluster/corosync update and an "Inline Update" is mentioned in the HowTo.

We cannot test every corner case. If you do an inplace upgrade to 6.x, you never join a 5.x node to the new 6.x cluster, so we do not test this.

See more on:
https://pve.proxmox.com/wiki/Upgrade_from_5.x_to_6.0

 

[oliwel]

We cannot test every corner case. If you do an inplace upgrade to 6.x, you never join a 5.x node to the new 6.x cluster, so we do not test this.

See more on:
https://pve.proxmox.com/wiki/Upgrade_from_5.x_to_6.0

I don't think an InPlace Upgrade of a 2-node system is a corner case and the HowTo gives - at least to my understanding - hints on how to upgrade an existing cluster. So can you please clarify if an InPlace Upgrade with two nodes as written in my initial post is a tested and supported way?

 

[tom]

I don't think an InPlace Upgrade of a 2-node system is a corner case and the HowTo gives - at least to my understanding - hints on how to upgrade an existing cluster. So can you please clarify if an InPlace Upgrade with two nodes as written in my initial post is a tested and supported way?

You seems confused. The inplace is heavily tested and works. See wiki.

(Not tested: joining new 5.x node to 6.x cluster)

 

[oliwel]

You seems confused. The inplace is heavily tested and works. See wiki.

(Not tested: joining new 5.x node to 6.x cluster)

Yes I am - the initial question was if I can move machines between a 5.4 and a 6.1 node where the 6.1 node was upgraded "InPlace" from a working cluster. Wolfgang said no, spirit set yes and any later reply mixes up different aspects.....

 

[sg90]

Yes I am - the initial question was if I can move machines between a 5.4 and a 6.1 node where the 6.1 node was upgraded "InPlace" from a working cluster. Wolfgang said no, spirit set yes and any later reply mixes up different aspects.....

If you mean machines by VMs? Then yes you can always move a VM from the older node to a new upgraded node during the rolling upgrade of a cluster.

I think earlier people understood you wanted to add a 5.4 node to a 6.x cluster.

 

[wolfgang]

Yes I am - the initial question was if I can move machines between a 5.4 and a 6.1 node where the 6.1 node was upgraded "InPlace" from a working cluster.

Sorry I get you wrong.
Yes, this is tested and working.