Several issue after upgrading from ProxMox v6 to v7

D

dfranco

Active Member
Jul 6, 2015
25
2
43
Hello,

I'm facing several issues after upgraded to ProxMox VE 7.3-6

Second issue
NoVNC console not working not working anymore even booting from an ISO

Second issue

None of my virtual machines can find a valid boot record and/or bootable OS on the attached hard drive.

Some setup details
PVE version
pve-manager/7.3-6/723bb6ec (running kernel: 5.4.203-1-pve)

Virtual machine config
agent: 0
boot: order=ide2;sata0
cores: 2
ide2: none,media=cdrom
memory: 2048
name: jessie-bw
net0: virtio=08:00:27:B9:57:85,bridge=vmbr0
numa: 0
ostype: l26
sata0: local-lvm:vm-110-disk-0,size=45G
sata1: local-lvm:vm-110-disk-1,size=480G
scsihw: virtio-scsi-pci
serial0: socket
smbios1: uuid=d5d256bd-1881-4bbc-8db8-fbe74e62f8eb
sockets: 1
startup: up=5
vga: std,memory=32

Let me know if any further information can be helpful.

As I used to have more than 20 vms, any help would more than welcome.

Thanks
 
Please note that I've restarted my ProxMox VE server several time, and tried with a brand new VM from scratch (got same problem).
 
dfranco said:
running kernel: 5.4.203-1-pve
Click to expand...
dfranco said:
Please note that I've restarted my ProxMox VE server several time
Click to expand...

That is still the PVE 6 kernel. Try to find out, why the 5.15 kernel from PVE 7 does not get booted.
The full output in code-tags of:
  • pveversion -v
  • proxmox-boot-tool status
  • proxmox-boot-tool kernel list
might be helpful.

Edit:
  • How was the upgrade done? With: apt full-upgrade/apt dist-upgrade or only apt upgrade?
  • Any problems/errors during the upgrade?
  • Did you follow: [1]?
  • Did you run: pve6to7 --full? Was all okay?

[1] https://pve.proxmox.com/wiki/Upgrade_from_6.x_to_7.0
 
Last edited:
Neobin said:
That is still the PVE 6 kernel. Try to find out, why the 5.15 kernel from PVE 7 does not get booted.
The full output in code-tags of:
  • pveversion -v
  • proxmox-boot-tool status
  • proxmox-boot-tool kernel list
might be helpful.

Edit:
  • How was the upgrade done? With: apt full-upgrade/apt dist-upgrade or only apt upgrade?
Click to expand...
Yes
Neobin said:
  • Any problems/errors during the upgrade?
Click to expand...
Yes
/bin/sh: 1: /usr/share/proxmox-ve/pve-apt-hook: not found
E: Sub-process /usr/share/proxmox-ve/pve-apt-hook returned an error code (127)
E: Failure running script /usr/share/proxmox-ve/pve-apt-hook
Fixed by using the instruction found here
https://forum.proxmox.com/threads/e...x-ve-pve-apt-hook-not-found.45436/post-516057
Neobin said:
  • Did you follow: [1]?
Click to expand...
Yes, step by step
Neobin said:
  • Did you run: pve6to7 --full? Was all okay?
Click to expand...
Yes I did, everything was green (no error)
Neobin said:
[1] https://pve.proxmox.com/wiki/Upgrade_from_6.x_to_7.0
Click to expand...
pveversion -v
Bash: 
# pveversion -v
proxmox-ve: residual config (running kernel: 5.4.203-1-pve)
pve-manager: 7.3-6 (running version: 7.3-6/723bb6ec)
pve-kernel-helper: 7.3-4
pve-kernel-5.4: 6.4-20
pve-kernel-5.3: 6.1-6
pve-kernel-5.4.203-1-pve: 5.4.203-1
pve-kernel-5.4.157-1-pve: 5.4.157-1
pve-kernel-5.3.18-3-pve: 5.3.18-3
pve-kernel-4.13.13-1-pve: 4.13.13-31
pve-kernel-4.10.17-2-pve: 4.10.17-20
ceph-fuse: 14.2.21-1
corosync: 3.1.7-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: 0.8.36+pve2
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve2
libproxmox-acme-perl: 1.4.3
libproxmox-backup-qemu0: 1.3.1-1
libpve-access-control: 7.3-1
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.3-2
libpve-guest-common-perl: 4.2-3
libpve-http-server-perl: 4.1-5
libpve-storage-perl: 7.3-2
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.2-1
lxcfs: 5.0.3-pve1
novnc-pve: 1.3.0-3
proxmox-backup-client: 2.3.3-1
proxmox-backup-file-restore: 2.3.3-1
proxmox-mail-forward: 0.1.1-1
proxmox-mini-journalreader: 1.3-1
proxmox-offline-mirror-helper: 0.5.1-1
proxmox-widget-toolkit: 3.5.5
pve-cluster: 7.3-2
pve-container: 4.4-2
pve-docs: 7.3-1
pve-edk2-firmware: 3.20220526-1
pve-firewall: 4.2-7
pve-firmware: 3.6-3
pve-ha-manager: 3.5.1
pve-i18n: 2.8-2
pve-qemu-kvm: 7.1.0-4
pve-xtermjs: 4.16.0-1
qemu-server: 7.3-3
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+2
vncterm: 1.7-1
zfsutils-linux: 2.1.9-pve1

proxmox-boot-tool status
Bash: 
# proxmox-boot-tool status

Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
E: /etc/kernel/proxmox-boot-uuids does not exist.

proxmox-boot-tool kernel list
Bash: 
Manually selected kernels
None.

Automatically selected kernels:
5.3.18-3-pve
5.4.157-1-pve
5.4.203-1-pve
 
Last edited:
ok, I found out why the kernel didn't get upgraded as expected

Bash: 
apt-get upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] y
Setting up initramfs-tools (0.140) ...
 update-initramfs: deferring update (trigger activated)
Processing triggers for initramfs-tools (0.140) ...
  update-initramfs: Generating /boot/initrd.img-5.4.203-1-pve
  /etc/initramfs/post-update.d//proxmox-boot-sync: 10: /usr/sbin/proxmox-boot-tool: not found
run-parts: /etc/initramfs/post-update.d//proxmox-boot-sync exited with return code 127
dpkg: error processing package initramfs-tools (--configure):
 installed initramfs-tools package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 initramfs-tools
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@bigboss:~# dpkg -S /usr/sbin/proxmox-boot-tool
dpkg-query: no path found matching pattern /usr/sbin/proxmox-boot-tool

it looks like /usr/sbin/proxmox-boot-tool is missing somehow.

but looking for it
Code: 
dpkg -S proxmox-boot-tool
dpkg-query: no path found matching pattern *proxmox-boot-tool*

Which package does provide this binary/script ?
 
I've found how to fix, at least one vm is running and I can see the console now :)

What I did was run this command

Code: 
apt install proxmox-ve

after a reboot

Code: 
proxmox-boot-tool status
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
E: /etc/kernel/proxmox-boot-uuids does not exist.
root@bigboss:~# proxmox-boot-tool kernel list
Manually selected kernels:
None.

Automatically selected kernels:
5.15.85-1-pve
5.4.203-1-pve

Above output still doesn't make fee confident about the upgrade.

Thought ?
 
dfranco said:
Bash: 
apt-get upgrade
Click to expand...

You should never use only: apt upgrade with Proxmox-products; ever use: apt full-upgrade or apt dist-upgrade!: [1]

dfranco said:
Code: 
proxmox-boot-tool status
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
E: /etc/kernel/proxmox-boot-uuids does not exist.

Above output still doesn't make fee confident about the upgrade.
Click to expand...

Afaiu, this is normal on (older?) installations without / on ZFS (which I assume is the case here, since you have: local-lvm):
Stoiko Ivanov said:
This system does not use proxmox-boot-tool for handling the bootloader config - this is normal when you don't have '/' on ZFS
Click to expand...
See: [2]. Maybe @Stoiko Ivanov can/wants to have a quick look here and can confirm (or not) my understanding. :)

First, please run: apt update again, followed by: apt full-upgrade and post the full output of both in code-tags each.

Second, to get a clear overview, please also provide the full outputs of the following in code-tags each:
  • pveversion -v (yes, again, please!)
  • apt list --installed | grep linux-image
  • grep -r '' /etc/apt/sources.list*

dfranco said:
at least one vm is running and I can see the console now
Click to expand...

Did you only test one or do the others still have problems?
If there are still problems, please do the first step I mentioned above first and then we can see further.

[1] https://forum.proxmox.com/threads/proxmox-ve-7-1-released.99847/post-463941
[2] https://forum.proxmox.com/threads/h...-to-uefi-boot-systemd-boot.120531/post-523610
 
Last edited:
Neobin said:
You should never use only: apt upgrade with Proxmox-products; ever use: apt full-upgrade or apt dist-upgrade!: [1]
Click to expand...
Understood, thanks a lot for the advice
Neobin said:
Afaiu, this is normal on (older?) installations without / on ZFS (which I assume is the case here, since you have: local-lvm):
Click to expand...
Yes, I'm using LVM on that server.

Just curious, do you recommend using ZFS instead of LVM somehow ?

Neobin said:
See: [2]. Maybe @Stoiko Ivanov can/wants to have a quick look here and can confirm (or not) my understanding. :)

First, please run: apt update again, followed by: apt full-upgrade and post the full output of both in code-tags each.
Click to expand...
Code: 
# apt update
Hit:1 http://security.debian.org bullseye-security InRelease
Hit:2 http://ftp.at.debian.org/debian bullseye InRelease
Hit:3 http://ftp.at.debian.org/debian bullseye-updates InRelease
Hit:4 http://download.proxmox.com/debian/pve bullseye InRelease
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
All packages are up to date.

# apt full-upgrade

# apt full-upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
  pve-kernel-5.3 pve-kernel-5.3.18-3-pve pve-kernel-5.4.157-1-pve
Use 'apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Neobin said:
Second, to get a clear overview, please also provide the full outputs of the following in code-tags each:
  • pveversion -v (yes, again, please!)
  • apt list --installed | grep linux-image
  • grep -r '' /etc/apt/sources.list*
Click to expand...
Code: 
# pveversion -v

# pveversion -v
proxmox-ve: 7.3-1 (running kernel: 5.15.85-1-pve)
pve-manager: 7.3-6 (running version: 7.3-6/723bb6ec)
pve-kernel-helper: 7.3-4
pve-kernel-5.15: 7.3-2
pve-kernel-5.4: 6.4-20
pve-kernel-5.3: 6.1-6
pve-kernel-5.15.85-1-pve: 5.15.85-1
pve-kernel-5.4.203-1-pve: 5.4.203-1
pve-kernel-5.4.157-1-pve: 5.4.157-1
pve-kernel-5.3.18-3-pve: 5.3.18-3
pve-kernel-4.13.13-1-pve: 4.13.13-31
pve-kernel-4.10.17-2-pve: 4.10.17-20
ceph-fuse: 14.2.21-1
corosync: 3.1.7-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: 0.8.36+pve2
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve2
libproxmox-acme-perl: 1.4.3
libproxmox-backup-qemu0: 1.3.1-1
libpve-access-control: 7.3-1
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.3-2
libpve-guest-common-perl: 4.2-3
libpve-http-server-perl: 4.1-5
libpve-storage-perl: 7.3-2
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.2-1
lxcfs: 5.0.3-pve1
novnc-pve: 1.3.0-3
proxmox-backup-client: 2.3.3-1
proxmox-backup-file-restore: 2.3.3-1
proxmox-mail-forward: 0.1.1-1
proxmox-mini-journalreader: 1.3-1
proxmox-offline-mirror-helper: 0.5.1-1
proxmox-widget-toolkit: 3.5.5
pve-cluster: 7.3-2
pve-container: 4.4-2
pve-docs: 7.3-1
pve-edk2-firmware: 3.20220526-1
pve-firewall: 4.2-7
pve-firmware: 3.6-3
pve-ha-manager: 3.5.1
pve-i18n: 2.8-2
pve-qemu-kvm: 7.1.0-4
pve-xtermjs: 4.16.0-1
qemu-server: 7.3-3
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+2
vncterm: 1.7-1
zfsutils-linux: 2.1.9-pve1

# apt list --installed | grep linux-image

# apt list --installed | grep linux-image

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

# grep -r '' /etc/apt/sources.list*

grep -r '' /etc/apt/sources.list*
/etc/apt/sources.list:# deb http://ftp.ch.debian.org/debian bullseye main contrib
/etc/apt/sources.list:
/etc/apt/sources.list:# # security updates
/etc/apt/sources.list:# deb http://security.debian.org bullseye-security main contrib
/etc/apt/sources.list:
/etc/apt/sources.list:deb http://ftp.at.debian.org/debian bullseye main contrib
/etc/apt/sources.list:
/etc/apt/sources.list:deb http://ftp.at.debian.org/debian bullseye-updates main contrib
/etc/apt/sources.list:
/etc/apt/sources.list:# security updates
/etc/apt/sources.list:deb http://security.debian.org bullseye-security main contrib
/etc/apt/sources.list.d/pve-enterprise.list.dpkg-old:#deb http://download.proxmox.com/debian/pve bullseye pve-no-subscription
/etc/apt/sources.list.d/pve-enterprise.list:#deb https://enterprise.proxmox.com/debian/pve bullseye pve-enterprise
/etc/apt/sources.list.d/pve-no-subscription.list:deb http://download.proxmox.com/debian/pve bullseye pve-no-subscription
Neobin said:
Did you only test one or do the others still have problems?
Click to expand...
I've checked almost all vms up to now which are running like a charm :)
They're still few I need to verifiy but I'm confident I won't face the same issue with these.
Neobin said:
If there are still problems, please do the first step I mentioned above first and then we can see further.
Click to expand...
It looks like both issues are fixed now.

You couldn't believe how happy am I, thanks a million for your help !

Do not hesitate If you need further information from me, I'd be happy if it could help somebody else having the same kind of issue.
Neobin said:
[1] https://forum.proxmox.com/threads/proxmox-ve-7-1-released.99847/post-463941
[2] https://forum.proxmox.com/threads/h...-to-uefi-boot-systemd-boot.120531/post-523610
Click to expand...
 
Last edited:
  • Like
Reactions: Neobin
This looks all pretty good to me, afaict and I am really glad to hear, that you could solve it all. :)

You might want to consider to switch from: ifupdown to: ifupdown2, since it is the default now for PVE 7: [1].
But better make sure to have local-/IPMI-access or even better install it directly via that (instead via e.g.: SSH), in case something unforseen happens with the network during the installation/switch and if all is okay, I would recommend to do a reboot of the PVE-host to check, if all is still good (network-wise).

I personally would also remove all those (partly really) old kernels. Especially those 4.x ones, because they are not picked up by: apt autoremove (too old).

[1] https://pve.proxmox.com/wiki/Network_Configuration#_apply_network_changes
 
  • Like
Reactions: dfranco
Neobin said:
This looks all pretty good to me, afaict and I am really glad to hear, that you could solve it all. :)

You might want to consider to switch from: ifupdown to: ifupdown2, since it is the default now for PVE 7: [1].
But better make sure to have local-/IPMI-access or even better install it directly via that (instead via e.g.: SSH), in case something unforseen happens with the network during the installation/switch and if all is okay, I would recommend to do a reboot of the PVE-host to check, if all is still good (network-wise).
Click to expand...
ok, it make sense legit, I'll do that
p.s: I have physical access to the server, so no worries in case something goes wrong.
Neobin said:
I personally would also remove all those (partly really) old kernels. Especially those 4.x ones, because they are not picked up by: apt autoremove (too old).
Click to expand...
I already did :)
Neobin said:
[1] https://pve.proxmox.com/wiki/Network_Configuration#_apply_network_changes
Click to expand...
Thanks again for your help !
 
  • Like
Reactions: Neobin
You must log in or register to reply here.
Top Bottom