first no boot then no updates

[bekcl88fx]

Hi everyone,

I have been using Proxmox about 6 months without any problems ( encrypted installation over Debian)
Yesterday I did update via gui, it went ok and since there was kernel update a did reboot my server.

After that it showed "Booting Proxmox VE, Loading Linux 6.8.12-5-pve..." and it is stucked there. If I remove "quiet" from the grub parameter it stucks at random line without any error message.
The server is completely frozen, keyboard not working I have to reboot via power button.
This is the same for kernel 6.8.12-5-pve and also for 6.8.12-4-pve.

If I boot older kernel - 6.8.12-2-pve, server boots up without any problem, but I am not able to do update.

Starting system upgrade: apt-get dist-upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 proxmox-kernel-6.8 : Depends: proxmox-kernel-6.8.12-6-pve-signed but it is not installed or
                               proxmox-kernel-6.8.12-6-pve
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

System not fully up to date (found 51 new packages)

If I try apt --fix-broken install, there is also error:

apt --fix-broken install
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Correcting dependencies... Done
The following packages were automatically installed and are no longer required:
  proxmox-kernel-6.8.12-1-pve-signed proxmox-kernel-6.8.4-3-pve-signed
  proxmox-kernel-6.8.8-1-pve-signed proxmox-kernel-6.8.8-2-pve-signed
Use 'apt autoremove' to remove them.
The following additional packages will be installed:
  proxmox-kernel-6.8.12-6-pve-signed
The following NEW packages will be installed:
  proxmox-kernel-6.8.12-6-pve-signed
0 upgraded, 1 newly installed, 0 to remove and 51 not upgraded.
4 not fully installed or removed.
Need to get 0 B/103 MB of archives.
After this operation, 577 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
(Reading database ... 95473 files and directories currently installed.)
Preparing to unpack .../proxmox-kernel-6.8.12-6-pve-signed_6.8.12-6_amd64.deb ...
Unpacking proxmox-kernel-6.8.12-6-pve-signed (6.8.12-6) ...
dpkg: error processing archive /var/cache/apt/archives/proxmox-kernel-6.8.12-6-pve-signed_6.8.12-6_amd64.deb (--unpack):
 cannot copy extracted data for './boot/System.map-6.8.12-6-pve' to '/boot/System.map-6.8.12-6-pve.dpkg-new': failed to write (No space left on device)
dpkg-deb: error: paste subprocess was killed by signal (Broken pipe)
Examining /etc/kernel/postrm.d.
run-parts: executing /etc/kernel/postrm.d/initramfs-tools 6.8.12-6-pve /boot/vmlinuz-6.8.12-6-pve
update-initramfs: Deleting /boot/initrd.img-6.8.12-6-pve
run-parts: executing /etc/kernel/postrm.d/proxmox-auto-removal 6.8.12-6-pve /boot/vmlinuz-6.8.12-6-pve
run-parts: executing /etc/kernel/postrm.d/zz-proxmox-boot 6.8.12-6-pve /boot/vmlinuz-6.8.12-6-pve
Re-executing '/etc/kernel/postrm.d/zz-proxmox-boot' in new private mount namespace..
No /etc/kernel/proxmox-boot-uuids found, skipping ESP sync.
run-parts: executing /etc/kernel/postrm.d/zz-update-grub 6.8.12-6-pve /boot/vmlinuz-6.8.12-6-pve
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-6.8.12-5-pve
Found linux image: /boot/vmlinuz-6.8.12-4-pve
Found linux image: /boot/vmlinuz-6.8.12-2-pve
Found initrd image: /boot/initrd.img-6.8.12-2-pve
Found linux image: /boot/vmlinuz-6.8.12-1-pve
Found initrd image: /boot/initrd.img-6.8.12-1-pve
Found linux image: /boot/vmlinuz-6.8.8-2-pve
Found initrd image: /boot/initrd.img-6.8.8-2-pve
Found linux image: /boot/vmlinuz-6.8.8-1-pve
Found initrd image: /boot/initrd.img-6.8.8-1-pve
Found linux image: /boot/vmlinuz-6.8.4-3-pve
Found initrd image: /boot/initrd.img-6.8.4-3-pve
done
Errors were encountered while processing:
 /var/cache/apt/archives/proxmox-kernel-6.8.12-6-pve-signed_6.8.12-6_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

if I try to remove older kernel:

apt purge proxmox-kernel-6.8.8-1-pve-signed
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 proxmox-kernel-6.8 : Depends: proxmox-kernel-6.8.12-6-pve-signed but it is not going to be installed or
                               proxmox-kernel-6.8.12-6-pve
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

I would appreciate any help to solve this

 

[bekcl88fx]

so I managed to solve it myself.

the root cause was that there was no free space in /boot. I have to manually delete 3 oldest kernels from /boot.
Then I run apt --fix-broken install and then update via gui and everything is OK now.

 

[cheiss]

Hi,

I have to manually delete 3 oldest kernels from /boot.

It would be best to uninstall old kernels via apt, thereby properly removing there package and all references.
You can list all installed kernel packages using e.g. apt list --installed | grep -E '(pve|proxmox)-kernel', then uninstall old ones as you see fit.

 

[bekcl88fx]

I wanted to remove it properly, but in this case it was not possible. The /boot run out of space during update so the kernel was partially installed but missing some dependencies. When I wanted to remove kernel with apt there was error about dependencies and when I wanted to solve the dependencies there was error about no free space in /boot. So I was running in circle

btw why there is not some sort of old kernel autoremove?

 

[Vittorio]

Hi @cheiss

I'm getting a similar error after I accidentally stopped last update
I've created a new thread but I'm really worried to do a wrong thing.

Any suggestion?

Thanks

 

[cheiss]

So I was running in circle

This was meant as more of a "correct things afterwards" thing - as you had already had solved your problem. Since this actually get's the system in a consistent state again -- as apt cannot know that you removed files manually.

btw why there is not some sort of old kernel autoremove?

Well, there is. But it depends on what packages you got installed exactly.
If you have the proxmox-kernel-X.Y meta-packages installed, old kernels should automatically be marked as auto-removable. The rest is normal Debian administration, e.g. they can be removed automatically using apt autoremove.

For example, from a machine of mine:
A apt dist-upgrade made the following changes:

The following NEW packages will be installed:                                                                                                                                                                       
   proxmox-kernel-6.11.11-1-pve-signed (6.11.11-1)                                                                                                                                                                   
   proxmox-kernel-6.8.12-7-pve-signed (6.8.12-7)                                                                                          
The following packages will be upgraded:        
   proxmox-kernel-6.11 (6.11.0-2 => 6.11.11-1)                                                                                                  
   proxmox-kernel-6.8 (6.8.12-5 => 6.8.12-7)
   [..]

After the fact, apt autoremove proposed the following changes:

The following packages will be REMOVED:
  proxmox-kernel-6.8.12-5-pve-signed
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
Remv proxmox-kernel-6.8.12-5-pve-signed [6.8.12-5]

 

[bekcl88fx]

thx for the explanation, apt autoremove is now working.
I was also searching Proxmox forum and found out that if you have common Proxmox installation your /boot is not on separate partition so this could
happen to you only if you run totally out of space.
In the case of manual installation in Debian your boot is on separate partition (500GB in my case) and you can run out of space easily.

So if you installed Proxmox manually be aware of this and use apt autoremove