[SOLVED] No space left on device

[Ola312]

Hi!

I'm getting the error "Error 500: closing file '/var/tmp/pve-reserved-ports.tmp.1372714' failed - No space left on device" when I try to connect to one of my VMs and also when I try to use Shell when I am in the pve tab.
Do anyone have a suggestion on how to solve this? Is there anyway to solve this in the GUI?

 

[Dunuin]

I guess df -h reports that your root filesystem is full? In that case you will have to use the CLI to find out whats eating up all the space and delete stuff.

 

[Ola312]

How can I run df -h? When I go to Shell I get the following error. Unfortunately I'm on a remote location so I do not have physical access to the server yet.

 

[Dunuin]

How can I run df -h?

Use ssh to access the server or use the physical console via display+keyboard/webKVM.

 

[Ola312]

Thanks!
Yes, it does

 

[Ola312]

Things is going the right way now! Have managed to delete some files now and trying to boot it. This is what it looks like now, but it seems like it's stuck on reboot now :/

This is the error on the pause and stop command:
trying to acquire lock...
TASK ERROR: can't lock file '/var/lock/qemu-server/lock-102.conf' - got timeout

 

[Wazaari]

That's probably a result of the issue you had earlier. You may try the suggestions listed here: https://forum.proxmox.com/threads/t...or-cant-lock-file-var-lock-qemu-server.16235/

 

[Ola312]

Thanks! Do you have any experience with deleting the lock file? Is it safe?

 

[Ola312]

I have deleted the file, but still get an error. Any other suggestions?

 

[Ola312]

Update!
Seems like it is full again, but when I try to cd in to it it says it's Not a directory. What could that be?

 

[Dunuin]

You root filesystem is mounted at "/" not "/dev/mapper/pve-root".
You probably store your guests on the root filesystem? At least there is no default "local-lvm" or "local-zfs" for your virtual disks present.

 

[Wazaari]

You cannot change directly into the disk, you need to check where it's mounted. You can do so by simply running `mount`. That said, Dunuin is right and this particular filesystem will be mounted as root of the file system tree (/).

You will need to find out which files are there. Maybe start with `du -a / 2>/dev/null | sort -n -r | head -n 20` and see what the output will be.

 

[Dunuin]

"ncdu" is a great tool to find big files but you probably won't be able to install it without freeing up some space first.

 

[alexskysilk]

some common reasons for overrunning your root partition:
- runaway logs. check /var/log for large files
- unmounted file systems. this might be a little tricky as the correct filesystems may be mounted, so you will need to mount /dev/mapper/root in an alternate mountpoint to look, eg
mkdir /temproot mount -t bind /dev/mapper/pve-root /temproot du -hs /temproot/mnt/
- if you're using the pve pool for guest storage, untrimmed guest file systems. resolved by running fstrim inside the guests.

since you're out of diskspace, you'll need to clear out SOMETHING before you can do the above- I'm sure you can find something to remove out of the 455GB you've committed to disk. start with

du -hs /
cd into the largest result, repeat
until you find what's eating all your space.

 

[Ola312]

Sorry for the late update! This was solved the same day. Thanks for all your help!

 

[lahmar Aymen]

Hello,
I have a problem with proxmox,please find enclosed the different captures
need solution please

 

[Dunuin]

Your root filesystem is full. Find unneeded files and delete them. Then run a fstrim -av to free up space. And fix the cause that is filling up your disk so it won't happen again.