Hi juanjfdez!
If the command to
df -h /boot/efi shows how much space is left on the ZFS rpool, then it probably means that
/dev/sda2 nor
/dev/sdb2 is not mounted yet. This is as it should be to prevent any corruption introduced by accidental writes or system crashes to this partition.
Could you temporarily mount both
/dev/sda2 and
/dev/sdb2 and post the output of
df -h and
ls -lha on these? And if possible, could you provide the output of
dpkg -l | grep kernel and a more detailed error messages when running the above mentioned commands (purge, init, ...)?
I guess/suspect that the system has many kernels from different series (e.g. 5.13, 5.15, 6.4, 6.8, ...) installed, which are all kept at the moment and not marked as "auto-removable" [0], which fills up the ESP partition quite a lot.
[0]
https://git.proxmox.com/?p=proxmox-...2c73e0ee3a0ab5218e20;hb=refs/heads/master#l20
Thanks Daniel for the prompt reply despite the time zones difference. this is the info you ask (now before I do anything):
root@nest:~# df -h /dev/sda2
Filesystem Size Used Avail Use% Mounted on
udev 16G 0 16G 0% /dev
root@nest:~# df -h /dev/sdb2
Filesystem Size Used Avail Use% Mounted on
udev 16G 0 16G 0% /dev
root@nest:~# df -h /boot/efi
Filesystem Size Used Avail Use% Mounted on
rpool/ROOT/pve-1 857G 20G 838G 3% /
Yep , Now After I mounted sda2 it confirms it is 512 MB full.
root@nest:~# df -h /dev/sda2
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 511M 511M 0 100% /boot/efi
root@nest:~# ls -lah /dev/sda2
brw-rw---- 1 root disk 8, 2 Nov 17 21:06 /dev/sda2
And Yep the next is consistent with what systemd-boot-tool was reporting: (although I thought I would see more old kernels?)
root@nest:~# ls -lah /boot/efi
total 290M
drwxr-xr-x 5 root root 4.0K Dec 31 1969 .
drwxr-xr-x 5 root root 43 Nov 19 20:22 ..
drwxr-xr-x 6 root root 4.0K Dec 17 2020 EFI
drwxr-xr-x 5 root root 4.0K Nov 23 2023 grub
-rwxr-xr-x 1 root root 58M Nov 23 2023 initrd.img-6.2.16-19-pve
-rwxr-xr-x 1 root root 58M Nov 23 2023 initrd.img-6.5.11-4-pve
-rwxr-xr-x 1 root root 58M Jul 27 13:02 initrd.img-6.5.13-6-pve
-rwxr-xr-x 1 root root 58M Aug 31 12:43 initrd.img-6.8.12-1-pve
drwxr-xr-x 3 root root 4.0K Sep 19 11:32 loader
-rwxr-xr-x 1 root root 13M Oct 24 2023 vmlinuz-6.2.16-19-pve
-rwxr-xr-x 1 root root 13M Nov 20 2023 vmlinuz-6.5.11-4-pve
-rwxr-xr-x 1 root root 13M Jul 26 08:34 vmlinuz-6.5.13-6-pve
-rwxr-xr-x 1 root root 14M Aug 5 12:17 vmlinuz-6.8.12-1-pve
-rwxr-xr-x 1 root root 0 Nov 17 20:35 vmlinuz-6.8.12-2-pve
-rwxr-xr-x 1 root root 8.2M Sep 16 15:32 vmlinuz-6.8.8-4-pve
Now as per your request I will UnMount /devsda2 and then try to mount /dev/sdb2 but probably because I had FORMAT it earlier then it wont mount or show anything yet I guess ... and Yep , as expected it shows nothing or shows almost empty as I could not complete the INIT subcommand the other day ....
root@nest:~# df -h /dev/sdb2
Filesystem Size Used Avail Use% Mounted on
/dev/sdb2 511M 12K 511M 1% /boot/efi
root@nest:~# ls -lah /boot/efi
total 17K
drwxr-xr-x 3 root root 4.0K Dec 31 1969 .
drwxr-xr-x 5 root root 43 Nov 19 20:22 ..
drwxr-xr-x 3 root root 4.0K Nov 19 20:47 EFI
The next is the CURRENT output of dpkg kernel ... just the portion with only kernel stuff ... As you can see I had already tried to purge 6.5.11-1 and 5.4.73-1 therefore they are marked for removal I guess
ii proxmox-default-kernel 1.1.0 all Default Proxmox Kernel Image
ii proxmox-kernel-6.5 6.5.13-6 all Latest Proxmox Kernel Image
rH proxmox-kernel-6.5.11-1-pve 6.5.11-1 amd64 Proxmox Kernel Image
iF proxmox-kernel-6.5.11-2-pve 6.5.11-2 amd64 Proxmox Kernel Image
iF proxmox-kernel-6.5.11-3-pve 6.5.11-3 amd64 Proxmox Kernel Image
ii proxmox-kernel-6.5.13-6-pve-signed 6.5.13-6 amd64 Proxmox Kernel Image (signed)
iU proxmox-kernel-6.8 6.8.12-4 all Latest Proxmox Kernel Image
ii proxmox-kernel-6.8.12-1-pve-signed 6.8.12-1 amd64 Proxmox Kernel Image (signed)
iF proxmox-kernel-6.8.12-2-pve-signed 6.8.12-2 amd64 Proxmox Kernel Image (signed)
iF proxmox-kernel-6.8.12-4-pve-signed 6.8.12-4 amd64 Proxmox Kernel Image (signed)
ii proxmox-kernel-6.8.8-4-pve-signed 6.8.8-4 amd64 Proxmox Kernel Image (signed)
ii proxmox-kernel-helper 8.1.0 all Function for various kernel maintenance tasks.
ii pve-firmware 3.14-1 all Binary firmware code for the pve-kernel
ii pve-kernel-5.13.19-2-pve 5.13.19-4 amd64 The Proxmox PVE Kernel Image
rc pve-kernel-5.15.53-1-pve 5.15.53-1 amd64 Proxmox Kernel Image
ii pve-kernel-5.4 6.4-4 all Latest Proxmox VE Kernel Image
ii pve-kernel-5.4.119-1-pve 5.4.119-1 amd64 The Proxmox PVE Kernel Image
ii pve-kernel-5.4.124-1-pve 5.4.124-1 amd64 The Proxmox PVE Kernel Image
rH pve-kernel-5.4.73-1-pve 5.4.73-1 amd64 The Proxmox PVE Kernel Image
rc pve-kernel-helper 7.3-6 all Function for various kernel maintenance tasks.
Now this is the error I get when I try the INIT subcommand on the NOW UNMOUNTED /dev/sdb2
root@nest:~# proxmox-boot-tool init /dev/sdb2
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
UUID="F8F3-6927" SIZE="536870912" FSTYPE="vfat" PARTTYPE="c12a7328-f81f-11d2-ba4b-00a0c93ec93b" PKNAME="sdb" MOUNTPOINT=""
Mounting '/dev/sdb2' on '/var/tmp/espmounts/F8F3-6927'.
Installing systemd-boot..
E: bootctl is not available - make sure systemd-boot is installed
Now I will post the error message I get when I try to do a simple apt command that involves rebuilding kernel stuff, say apt upgrade -y and Thanks so Much for the help BTW. I will be waiting for more instructions. Cheers JJ
PS: Kernel 5.4.73-1-pve is still marked for removal because 2 days ago I tried apt purge (that package)
root@nest:~# apt upgrade -y
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following packages will be REMOVED:
pve-kernel-5.4.73-1-pve
The following NEW packages will be installed:
libpve-network-api-perl
The following packages will be upgraded:
ifupdown2 libjs-extjs libpve-access-control libpve-cluster-api-perl libpve-cluster-perl libpve-common-perl libpve-guest-common-perl libpve-network-perl libpve-notify-perl libpve-rs-perl libpve-storage-perl
proxmox-backup-client proxmox-backup-file-restore proxmox-firewall proxmox-mail-forward proxmox-ve proxmox-widget-toolkit pve-cluster pve-container pve-docs pve-firewall pve-ha-manager pve-i18n pve-manager
qemu-server
25 upgraded, 1 newly installed, 1 to remove and 0 not upgraded.
8 not fully installed or removed.
Need to get 21.1 MB/138 MB of archives.
After this operation, 284 MB disk space will be freed.
Get:1
http://download.proxmox.com/debian/pve bookworm/pve-no-subscription amd64 libpve-rs-perl amd64 0.9.0 [2,799 kB]
Get:2
http://download.proxmox.com/debian/pve bookworm/pve-no-subscription amd64 libpve-storage-perl all 8.2.9 [146 kB]
Get:3
http://download.proxmox.com/debian/pve bookworm/pve-no-subscription amd64 pve-firewall amd64 5.1.0 [75.2 kB]
Get:4
http://download.proxmox.com/debian/pve bookworm/pve-no-subscription amd64 qemu-server amd64 8.3.0 [289 kB]
Get:5
http://download.proxmox.com/debian/pve bookworm/pve-no-subscription amd64 pve-ha-manager amd64 4.0.6 [65.6 kB]
Get:6
http://download.proxmox.com/debian/pve bookworm/pve-no-subscription amd64 pve-docs all 8.3.1 [15.6 MB]
Get:7
http://download.proxmox.com/debian/pve bookworm/pve-no-subscription amd64 pve-i18n all 3.3.1 [412 kB]
Get:8
http://download.proxmox.com/debian/pve bookworm/pve-no-subscription amd64 pve-manager all 8.3.0 [548 kB]
Get:9
http://download.proxmox.com/debian/pve bookworm/pve-no-subscription amd64 libpve-network-perl all 0.10.0 [36.5 kB]
Get:10
http://download.proxmox.com/debian/pve bookworm/pve-no-subscription amd64 libpve-network-api-perl all 0.10.0 [13.7 kB]
Get:11
http://download.proxmox.com/debian/pve bookworm/pve-no-subscription amd64 proxmox-firewall amd64 0.6.0 [1,042 kB]
Get:12
http://download.proxmox.com/debian/pve bookworm/pve-no-subscription amd64 proxmox-ve all 8.3.0 [5,228 B]
Fetched 21.1 MB in 1s (16.2 MB/s)
apt-listchanges: Reading changelogs...
(Reading database ... 177525 files and directories currently installed.)
Removing pve-kernel-5.4.73-1-pve (5.4.73-1) ...
Examining /etc/kernel/postrm.d.
run-parts: executing /etc/kernel/postrm.d/initramfs-tools 5.4.73-1-pve /boot/vmlinuz-5.4.73-1-pve
update-initramfs: Deleting /boot/initrd.img-5.4.73-1-pve
run-parts: executing /etc/kernel/postrm.d/proxmox-auto-removal 5.4.73-1-pve /boot/vmlinuz-5.4.73-1-pve
run-parts: executing /etc/kernel/postrm.d/zz-proxmox-boot 5.4.73-1-pve /boot/vmlinuz-5.4.73-1-pve
Re-executing '/etc/kernel/postrm.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/1808-B094
Copying kernel and creating boot-entry for 6.5.13-6-pve
Copying kernel and creating boot-entry for 6.8.12-1-pve
Copying kernel and creating boot-entry for 6.8.12-2-pve
mkdir: cannot create directory ‘/var/tmp/espmounts/1808-B094/EFI/proxmox/6.8.12-2-pve’: No space left on device
run-parts: /etc/kernel/postrm.d/zz-proxmox-boot exited with return code 1
Failed to process /etc/kernel/postrm.d at /var/lib/dpkg/info/pve-kernel-5.4.73-1-pve.postrm line 14.
dpkg: error processing package pve-kernel-5.4.73-1-pve (--remove):
installed pve-kernel-5.4.73-1-pve package post-removal script subprocess returned error exit status 1
dpkg: too many errors, stopping
Errors were encountered while processing:
pve-kernel-5.4.73-1-pve
Processing was halted because there were too many errors.
E: Sub-process /usr/bin/dpkg returned an error code (1)
