pve5to6 output and fixes

[Michael Mansour]

Hi,

I have 4 nodes running Proxmox VE 5.4-10 (only one kernel update behind but will be remedied soon).

When I run the pve5to6 tool I see these two FAILures:

1.

FAIL: ring0_addr 'node1' of node 'node1' is not an IP address, consider replacing it with the currently res
olved IP address.
FAIL: ring0_addr 'node2' of node 'node2' is not an IP address, consider replacing it with the currently res
olved IP address.
FAIL: ring0_addr 'node3' of node 'node3' is not an IP address, consider replacing it with the currently res
olved IP address.
FAIL: ring0_addr 'node4' of node 'node4' is not an IP address, consider replacing it with the currently res
olved IP address.

2.

= CHECKING INSTALLED COROSYNC VERSION =

FAIL: corosync 2.x installed, cluster-wide upgrade to 3.x needed!

Taking point 1, I checked /etc/pve/corosync.conf and it indeed has hostnames for the "ring0_addr" entries instead of IP's. I have gone through the Proxmox documentation and it says it's recommended to have IP's.

So to change this, it's my understanding I need to do the following:

a) edit the /etc/pve/corosync.conf file (the cluster version)

b) change "ring0_addr" for each entry in the nodelist stanza, to change it from hostname to IP

c) increment the "config_version" in the totem stanza.

d) save the file

corosync should then copy this cluster version (/etc/pve/corosync.conf) to the local version (/etc/corosync/corosync.conf) by itself ? or do I need to restart corosync ?

Taking point 2, I see I have to follow this procedure:

https://pve.proxmox.com/wiki/Upgrade_from_5.x_to_6.0#Cluster:_always_upgrade_to_Corosync_3_first

Are there any problems people have found with going through this corosync v3 process or smooth as silk?

Lastly, after pve5to6 passes everything, it should be OK to go through:

https://pve.proxmox.com/wiki/Upgrade_from_5.x_to_6.0

yes? (beyond this scope I will of course make sure backups etc are all present and other things like ZFS snapshots etc won't get in the way).

Thank you.

Michael.

 

[t.lamprecht]

Taking point 1, I checked /etc/pve/corosync.conf and it indeed has hostnames for the "ring0_addr" entries instead of IP's. I have gone through the Proxmox documentation and it says it's recommended to have IP's.

Be aware that a newer version of the pve5to6 script comes with a pve-manager update (version 5.4-11) which does not marks this as plain "FAIL" anymore and does some advanced checks (i.e., tries to resolve them like corosync would).

Are there any problems people have found with going through this corosync v3 process or smooth as silk?

It may make sense to play and upgrade through in a test cluster (could be three virtual machines with PVE inside too), to get a feeling for this. Always good to test things out first.

Lastly, after pve5to6 passes everything, it should be OK to go through:

https://pve.proxmox.com/wiki/Upgrade_from_5.x_to_6.0

yes? (beyond this scope I will of course make sure backups etc are all present and other things like ZFS snapshots etc won't get in the way).

The script has not 100% coverage, and this cannot possibly be achieved. It tries to check basics and the most common pitfalls we could come up with, that does not means that if it tells you "OK", no (unexplained) warn, no fail, all will go 100% guaranteed well. I mean, so much can happen, a disk which is on the brink of failure and the upgrade finally puts it down, something you have changed or special in your setup which not a lot people have (or at least not those people which tested or did the upgrade already) could fail the upgrade as well.

So, TL;DR, it's good and should give you some assurance that the upgrade won't go total catastrophic two seconds in, but it's still good to:
1. Test it in a setup which in it's basics is as similar as your production one (as said, can be virtual too, better than nothing)
2. have backups.

I do not want to scare anybody, and am also quite confident that if one runs "pve5to6", follows the upgrade how-to closely, and test a bit around before, all will go well.

 

[Dubard]

Hi,

I have Proxmox version 5.4.15-15-58.
when I run the pve5to6 script I get the following messages

root@pve1:~# pve5to6
= CHECKING VERSION INFORMATION FOR PVE PACKAGES =


Checking for package updates..
PASS: all packages uptodate


Checking proxmox-ve package version..
PASS: proxmox-ve package has version >= 5.4-2


Checking running kernel version..
PASS: expected running kernel '4.15.18-30-pve'.


Checking for installed stock Debian Kernel..
PASS: Stock Debian kernel package not installed.


= CHECKING CLUSTER HEALTH/SETTINGS =


PASS: systemd unit 'pve-cluster.service' is in state 'active'
PASS: systemd unit 'corosync.service' is in state 'active'
PASS: Cluster Filesystem is quorate.


Analzying quorum settings and state..
INFO: configured votes - nodes: 3
INFO: configured votes - qdevice: 0
INFO: current expected votes: 3
INFO: current total votes: 3


Checking nodelist entries..
WARN: pve1: ring0_addr 'pve1-corosync' resolves to '10.10.17.1'.
Consider replacing it with the currently resolved IP address.
WARN: pve2: ring0_addr 'pve2-corosync' resolves to '10.10.17.3'.
Consider replacing it with the currently resolved IP address.
WARN: pve3: ring0_addr 'pve3-corosync' resolves to '10.10.17.4'.
Consider replacing it with the currently resolved IP address.


Checking totem settings..
PASS: Corosync transport set to implicit default.
PASS: Corosync encryption and authentication enabled.


INFO: run 'pvecm status' to get detailed cluster status..


= CHECKING INSTALLED COROSYNC VERSION =


FAIL: corosync 2.x installed, cluster-wide upgrade to 3.x needed!


= CHECKING HYPER-CONVERGED CEPH STATUS =


INFO: hyper-converged ceph setup detected!
INFO: getting Ceph status/health information..
FAIL: unable to determine Ceph status!
INFO: getting Ceph OSD flags..
FAIL: unable to get Ceph OSD flags!
INFO: getting Ceph daemon versions..
FAIL: unable to determine Ceph daemon versions!
WARN: 'noout' flag not set - recommended to prevent rebalancing during upgrades.
INFO: checking Ceph config..
WARN: No 'mon_host' entry found in ceph config.
  It's recommended to add mon_host with all monitor addresses (without ports) to the global section.
PASS: 'ms_bind_ipv6' not enabled
WARN: [global] config section contains 'keyring' option, which will prevent services from starting with Nautilus.
Move 'keyring' option to [client] section instead.


= CHECKING CONFIGURED STORAGES =


PASS: storage 'zfs-cadstorage-CT' enabled and active.
PASS: storage 'heracles-BACKUP' enabled and active.
PASS: storage 'local' enabled and active.
PASS: storage 'zfs-cadstorage-VM' enabled and active.
WARN: storage 'local-lvm' enabled but not active!


= MISCELLANEOUS CHECKS =


INFO: Checking common daemon services..
PASS: systemd unit 'pveproxy.service' is in state 'active'
PASS: systemd unit 'pvedaemon.service' is in state 'active'
PASS: systemd unit 'pvestatd.service' is in state 'active'
INFO: Checking for running guests..
WARN: 1 running guest(s) detected - consider migrating or stopping them.
INFO: Checking if the local node's hostname 'atlas' is resolvable..
INFO: Checking if resolved IP is configured on local node..
PASS: Resolved node IP '172.14.15.16' configured and active on single interface.
INFO: Check node certificate's RSA key size
PASS: Certificate 'pve-root-ca.pem' passed Debian Busters security level for TLS connections (4096 >= 2048)
PASS: Certificate 'pve-ssl.pem' passed Debian Busters security level for TLS connections (2048 >= 2048)
INFO: Checking KVM nesting support, which breaks live migration for VMs using it..
PASS: KVM nested parameter not set.
INFO: Checking VMs with OVMF enabled and bad efidisk sizes...
PASS: No VMs with OVMF and problematic efidisk found.


= SUMMARY =


TOTAL:    34
PASSED:   22
SKIPPED:  0
WARNINGS: 8
FAILURES: 4


ATTENTION: Please check the output for detailed information!
Try to solve the problems one at a time and then run this checklist tool again.
root@pve1:~#

My questions are as follows:

1) I also have the alert about "ring0_addr" although I have the latest version of the script.
Is this a problem for the migration ?...do I have to modify the file "/etc/pve/corosync.conf" by putting the IP addresses instead of the hostname?

2) I also have the problem on CEPH but I don't use CEPH anymore (it was a test I had done 2 years ago) will it block my installation ?

3) My cluster consists of only 3 nodes. I would like to migrate to only 1 node first in order to test if it is going well. Will this cause problems for my entire cluster ?... if I start by migrating on only 1 node ?
To be clearer, can my Proxmox cluster work with 1 node in version 6 and the 2 others in version 5 (the time I spend 1 to 1 the nodes in version 6) ?


Many thanks

 

[Dubard]

Hi Everybody,

Someone might have some answers to my questions above as I will soon have to prepare the update to version 6.

Thank you

 

[JohnJames]

Did you ever find any more information to your question. If so, please update here. I'm also performing an upgrade from 5to6 and have similar Warn

 

[t.lamprecht]

1) I also have the alert about "ring0_addr" although I have the latest version of the script.
Is this a problem for the migration ?...do I have to modify the file "/etc/pve/corosync.conf" by putting the IP addresses instead of the hostname?

Yes, just replace with correct IP, else you really need to ensure all host<-> IP mappings are correct and complete in /etc/hosts (or your local DNS) of all cluster nodes.

2) I also have the problem on CEPH but I don't use CEPH anymore (it was a test I had done 2 years ago) will it block my installation ?

It shouldn't, you could use pveceph purge to purge most of the stuff, you still may want to include the ceph repo to get the newer packages there.

3) My cluster consists of only 3 nodes. I would like to migrate to only 1 node first in order to test if it is going well. Will this cause problems for my entire cluster ?... if I start by migrating on only 1 node ?
To be clearer, can my Proxmox cluster work with 1 node in version 6 and the 2 others in version 5 (the time I spend 1 to 1 the nodes in version 6) ?

You need to update the whole cluster to corosync 3 first while still being on PVE 5, that is a must.
https://pve.proxmox.com/wiki/Upgrade_from_5.x_to_6.0#Cluster:_always_upgrade_to_Corosync_3_first

After that a partial upgrade does not breaks stuff actively, but things crossing the node boundary (e.g., migration after a VM was freshly started on PVE 6) may not work backwards (from upgraded "new" node to PVE 5 "old" nodes) as we can only guarantee forwards compatibility on such stuff, so sooner or later you need to upgrade the other ones - so forward is the only easy direction to go here as major downgrades are really not tested.

Instead of doing the test on one of the cluster nodes I'd suggest playing through the upgrade on a test system, that can also be a virtual PVE cluster hosted by VMs.