apt update failed: No space on device

I

ihr

Member
Dec 25, 2022
47
6
13
Hi,

For some reason, the normal upgrade to proxmox-kernel-6.14.11-4 failed here is a screen capture of the process I launched from my iPad.
IMG_2545 en tamaño grande.jpeg
Later, on my PC i tried to clean up space and repeat the process but similar errors occurs

Code: 
root@karajik:/# apt dist-upgrade
The following package was automatically installed and is no longer required:
  proxmox-kernel-6.14.11-2-pve-signed
Use 'apt autoremove' to remove it.

REMOVING:
  proxmox-kernel-6.8.12-13-pve-signed

Summary:
  Upgrading: 0, Installing: 0, Removing: 1, Not Upgrading: 0
  3 not fully installed or removed.
  Freed space: 577 MB

Continue? [Y/n] y
(Reading database ... 89937 files and directories currently installed.)
Removing proxmox-kernel-6.8.12-13-pve-signed (6.8.12-13) ...
Examining /etc/kernel/postrm.d.
run-parts: executing /etc/kernel/postrm.d/initramfs-tools 6.8.12-13-pve /boot/vmlinuz-6.8.12-13-pve
update-initramfs: Deleting /boot/initrd.img-6.8.12-13-pve
run-parts: executing /etc/kernel/postrm.d/proxmox-auto-removal 6.8.12-13-pve /boot/vmlinuz-6.8.12-13-pve
run-parts: executing /etc/kernel/postrm.d/zz-proxmox-boot 6.8.12-13-pve /boot/vmlinuz-6.8.12-13-pve
Re-executing '/etc/kernel/postrm.d/zz-proxmox-boot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/F398-8FFD
    Copying kernel and creating boot-entry for 6.14.11-3-pve
    Copying kernel and creating boot-entry for 6.14.11-4-pve
cp: error writing '/var/tmp/espmounts/F398-8FFD/EFI/proxmox/6.14.11-4-pve/initrd.img-6.14.11-4-pve': No space left on device
run-parts: /etc/kernel/postrm.d/zz-proxmox-boot exited with return code 1
Failed to process /etc/kernel/postrm.d at /var/lib/dpkg/info/proxmox-kernel-6.8.12-13-pve-signed.postrm line 15.
dpkg: error processing package proxmox-kernel-6.8.12-13-pve-signed (--remove):
 installed proxmox-kernel-6.8.12-13-pve-signed package post-removal script subprocess returned error exit status 1
dpkg: too many errors, stopping
Errors were encountered while processing:
 proxmox-kernel-6.8.12-13-pve-signed
Processing was halted because there were too many errors.
Error: Sub-process /usr/bin/dpkg returned an error code (1)
root@karajik:/#

I run plenty of disk space

Code: 
root@karajik:/# df -h
Filesystem                       Size  Used Avail Use% Mounted on
udev                              32G     0   32G   0% /dev
tmpfs                            6.3G  7.6M  6.3G   1% /run
rpool/ROOT/pve-1                 409G  4.4G  405G   2% /
tmpfs                             32G   63M   32G   1% /dev/shm
efivarfs                         256K  239K   13K  96% /sys/firmware/efi/efivars
tmpfs                            5.0M     0  5.0M   0% /run/lock
tmpfs                             32G     0   32G   0% /tmp
tmpfs                            1.0M     0  1.0M   0% /run/credentials/systemd-journald.service
rpool                            405G  128K  405G   1% /rpool
rpool/ROOT                       405G  128K  405G   1% /rpool/ROOT
rpool/data                       405G  128K  405G   1% /rpool/data
/dev/fuse                        128M  148K  128M   1% /etc/pve
tmpfs                            1.0M     0  1.0M   0% /run/credentials/getty@tty1.service
192.168.5.5:/mnt/zpool/machines   12T  364G   11T   4% /mnt/pve/vms
192.168.5.5:/mnt/zpool/backups    12T  645G   11T   6% /mnt/pve/backups
192.168.5.5:/mnt/zpool/isos       11T   44G   11T   1% /mnt/pve/isos
tmpfs                            6.3G  8.0K  6.3G   1% /run/user/0
root@karajik:/#

And at this moment I don't feel safe to restart the computer.

What can I do next?
 
Hi,

What's the return of :
Code: 
df -H /var/tmp/ && df -Hi /var/tmp/

Have you deleted big files ?
What's the return of :
Code: 
lsof | grep deleted

Best regards,
 
Last edited:
janus57 said:
Hi,

What's the return of :
Code: 
df -H /var/tmp/ && df -Hi /var/tmp/

Have you deleted big files ?
What's the return of :
Code: 
df -H /var/tmp/ && df -Hi /var/tmp/

Best regards,
Click to expand...
Here you have it

Code: 
root@karajik:/# df -H /var/tmp/ && df -Hi /var/tmp/
Filesystem        Size  Used Avail Use% Mounted on
rpool/ROOT/pve-1  478G  4.7G  473G   1% /
Filesystem       Inodes IUsed IFree IUse% Mounted on
rpool/ROOT/pve-1   924M  101k  924M    1% /
root@karajik:/# df -H /var/tmp/ && df -Hi /var/tmp/
Filesystem        Size  Used Avail Use% Mounted on
rpool/ROOT/pve-1  478G  4.7G  473G   1% /
Filesystem       Inodes IUsed IFree IUse% Mounted on
rpool/ROOT/pve-1   924M  101k  924M    1% /
 
Let me add more information that now might be more relevant, after reviewing the content of the script at /etc/kernel/postrm.d/zz-proxmox-boot

Last week, both the hard drives (ZFS mirror) was replaced with new units. The process run successfully and the server was able to reboot without any issues in the latest version of the kernel.

At certain moments I'd to update file /etc/kernel/proxmox-boot-uuids

The content of this file at the moment is

Code: 
root@karajik:/# cat /etc/kernel/proxmox-boot-uuids
F398-8FFD
F439-4DB9

And the error message is referencing the first one.
 
I'm starting to see the root of the issue. The problem is with /dev/sda2 and /dev/sdb2 partitions that are vfat used for booting the system.

Code: 
root@karajik:/tmp# df -h
Filesystem                       Size  Used Avail Use% Mounted on
udev                              32G     0   32G   0% /dev
tmpfs                            6.3G  2.4M  6.3G   1% /run
rpool/ROOT/pve-1                 445G  4.4G  441G   1% /
tmpfs                             32G   63M   32G   1% /dev/shm
efivarfs                         256K  239K   13K  96% /sys/firmware/efi/efivars
tmpfs                            5.0M     0  5.0M   0% /run/lock
tmpfs                             32G     0   32G   0% /tmp
tmpfs                            1.0M     0  1.0M   0% /run/credentials/systemd-journald.service
rpool                            441G  128K  441G   1% /rpool
rpool/ROOT                       441G  128K  441G   1% /rpool/ROOT
rpool/data                       441G  128K  441G   1% /rpool/data
/dev/fuse                        128M  148K  128M   1% /etc/pve
tmpfs                            1.0M     0  1.0M   0% /run/credentials/getty@tty1.service
192.168.5.5:/mnt/zpool/machines   12T  364G   11T   4% /mnt/pve/vms
192.168.5.5:/mnt/zpool/backups    12T  645G   11T   6% /mnt/pve/backups
192.168.5.5:/mnt/zpool/isos       11T   44G   11T   1% /mnt/pve/isos
tmpfs                            6.3G  8.0K  6.3G   1% /run/user/0
/dev/sda2                        511M  511M     0 100% /tmp/mntsda2
/dev/sdb2                        511M  488M   24M  96% /tmp/mntsdb2

I've prepared two directories under /tmp/mntsd?2 and they are almost full. I really don't know why. I'm going to explore if I can free-up some space there so I can continue to use apt. Every time I try it is trying to install kernel version 6.14.11-4-pve and it fails.
 
For documentation purposes

0- I moved all VMs and CTs to other hosts just as a precaution.
1- I mounted the partitions as indicated in the previous post.
2. I manually cleaned space by removing old kernels not mentioned in proxmox-boot-tool status
3. Including the old grub directory no longer used
4. I run proxmox-boot-tool refresh just to keep things in sync
5. I repeated all apt commands that didn't finish in the previous installation.

The system restarted with no issues
 
  • Like
Reactions: SteveITS
You must log in or register to reply here.
Top Bottom