6.x to 7.x explosion

[vapour]

Goal:
Weekend standalone server upgrade from 6.3 to 6.4 to 7.0 so a German based integrator could install and configure a go live system today at 2pm EST.

Background (Short version):
The Proxmox sytem resides on a 60Gb SSD. The 6.3 to 6.4 upgrade went terrible. After following ALL steps, checklists, best practice advice, etc., for reasons unknown at this time, 7.0 was being installed and subsequently failed. Leaving behind a trail of destruction and a system with a 24 minute reboot time and no GUI access. After 43 hours of trying to repair the damage the system was pronounced dead. Cause of death, unknown.
Hardware:
Dell R710 Server, 2x 2.93GHz X5670 12-Cores Total / 144GB RAM / 6x 2TB 3.5 SAS HDD, H700 w/ 512MB / iDRAC6 / 60Gb SSD in a DVD-ROM cage / 2x 870W PSU., Quad NICs.
Damage:
The initial system was lost but not without making copies of /root/*, /etc/*, ../kvm/*, ../cron/*
Actions Taken:
Successful install from 7.02 ISO install to 60Gb SSD.
Outcome and Potential Loss:
The disks that the 31 containers and 8vm's appear untouched as they did not reside on the system disk. I can still see all disks with fdisk -l.

SOS
What do I need to do to bring back up the containers, vm's, re-establish the previous network and system config. by 2pm EST?

Thank you all in advance.
Chris

 

[dcsapak]

hi,

The 6.3 to 6.4 upgrade went terrible. After following ALL steps, checklists, best practice advice, etc., for reasons unknown at this time, 7.0 was being installed and subsequently failed.

the only reason this can happen if the repositories are not correctly configured, i.e. 'bullseye' repos were configured despite 6.x being 'buster'
or did you mean something different?

What do I need to do to bring back up the containers, vm's, re-establish the previous network and system config. by 2pm EST?

you need to reconfigure your system to use the disks how they were used before: lvm/mounted/zfs/etc.
which steps you need to take exactly depends on how it was configured before

then you have to recreate the vm/ct configs if you do not have them anymore.
if you have recent backups, you can extract the config from there,
otherwise you could recreate them by memory, or try to extract them from the old pmxcfs sqlite database (this is stored in /var/lib/pve-cluster/config.db)

 

[vapour]

Thanks for the reply.

the only reason this can happen if the repositories are not correctly configured, i.e. 'bullseye' repos were configured despite 6.x being 'buster'

I was prompted to change Buster Main contrib repo, but not to Bullseye, nor did I do it myself.

you need to reconfigure your system to use the disks how they were used before: lvm/mounted/zfs/etc.
which steps you need to take exactly depends on how it was configured before

I made copies of /root/*, /etc/*, ../kvm/*, ../cron/*. So what I've been doing is replacing content in the new 7.x files with the content of the last working config.

I must have messed up the order because now the server ends up in emergency mode. (See Below)

This error came up after I

1. Replaced fstab file with original
2. Replaced hosts file with original
3. Copied over vm.conf and ct.conf files to /etc/pve/nodes/nodename/lxc and /qemu-server respectively
4. Reboot

This is where I am right now. Let me know if I should send the installed_packages.txt or pvereport.txt.

Thanks

 

[vapour]