Proxmox GUI not working even when its logs says it's running

[ahmedsaber9]

HI
my server was working fine, but today it's not
I can only access the server with Rescue only, and I try to see the logs of my Proxmox but can't find it
Syslog.Log

Dec 10 00:00:00 Debian-104-buster-64-minimal rsyslogd:  [origin software="rsyslogd" swVersion="8.1901.0" x-pid="661" x-info="https://www.rsyslog.com"] rsyslogd 
Dec 10 13:37:20 Debian-104-buster-64-minimal spiceproxy[25436]: Warning: unable to close filehandle GEN3 properly: No space left on device at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1821, <DATA> line 755.
Dec 10 20:33:37 Debian-104-buster-64-minimal spiceproxy[12396]: Warning: unable to close filehandle GEN3 properly: No space left on device at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1821, <DATA> line 755.
Dec 10 20:33:37 Debian-104-buster-64-minimal spiceproxy[12396]: error writing access log
Dec 10 20:33:37 Debian-104-buster-64-minimal spiceproxy[12396]: worker exit
Dec 10 23:59:00 Debian-104-buster-64-minimal systemd[1]: Started Proxmox VE replication runner.
Dec 11 00:00:00 Debian-104-buster-64-minimal systemd[1]: Starting Proxmox VE replication runner...
Dec 11 00:00:00 Debian-104-buster-64-minimal systemd[1]: Starting Rotate log files...
Dec 11 00:00:00 Debian-104-buster-64-minimal systemd[1]: Starting Daily man-db regeneration...
Dec 11 00:00:00 Debian-104-buster-64-minimal mandb[5620]: /usr/bin/mandb: can't write to /var/cache/man/5620: No space left on device
Dec 11 00:00:00 Debian-104-buster-64-minimal mandb[5620]: /usr/bin/mandb: can't create index cache /var/cache/man/5620: No space left on device
Dec 11 00:00:00 Debian-104-buster-64-minimal systemd[1]: man-db.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Dec 11 00:00:00 Debian-104-buster-64-minimal systemd[1]: man-db.service: Failed with result 'exit-code'.
Dec 11 00:00:00 Debian-104-buster-64-minimal systemd[1]: Failed to start Daily man-db regeneration.
Dec 11 00:00:00 Debian-104-buster-64-minimal pvefw-logger[20477]: received terminate request (signal)
Dec 11 00:00:00 Debian-104-buster-64-minimal pvefw-logger[20477]: stopping pvefw logger
Dec 11 00:00:00 Debian-104-buster-64-minimal systemd[1]: Stopping Proxmox VE firewall logger...
Dec 11 00:00:00 Debian-104-buster-64-minimal systemd[1]: pvefw-logger.service: Succeeded.
Dec 11 00:00:00 Debian-104-buster-64-minimal systemd[1]: Stopped Proxmox VE firewall logger.
Dec 11 00:00:00 Debian-104-buster-64-minimal systemd[1]: Starting Proxmox VE firewall logger...
Dec 11 00:00:00 Debian-104-buster-64-minimal pvefw-logger[5634]: starting pvefw logger
Dec 11 00:00:00 Debian-104-buster-64-minimal systemd[1]: Started Proxmox VE firewall logger.
Dec 11 00:00:00 Debian-104-buster-64-minimal systemd[1]: pvesr.service: Succeeded.
Dec 11 00:00:00 Debian-104-buster-64-minimal systemd[1]: Started Proxmox VE replication runner.

I can see a problem related to disk space but I remove some logs to get more space the problem is not gone and the GUI is not stating
df -h /mnt
Filesystem Size Used Avail Use% Mounted on
/dev/md2 437G 411G 3.6G 100% /mnt

Your help is highly appreciated

 

[ahmedsaber9]

root@rescue /mnt/var/cache/apt/archives # du -hsx /mnt*
411G /mnt
root@rescue /mnt/var/cache/apt/archives # du -hsx /mnt/var*
401G /mnt/var

 

[bbgeek17]

Why not show entire output of "df" , "mount", etc?
What relationship does /mnt have to / and /var?

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[rason]

It looks like there may be a problem with your disk space, as indicated by the "No space left on device" errors in the logs. It is recommended to check the disk space on your server and free up any unnecessary files to increase the available space. Additionally, you may want to check for any errors related to Proxmox or the spiceproxy service in the logs and try to address those issues as well.

 

[ahmedsaber9]

Why not show entire output of "df" , "mount", etc?
What relationship does /mnt have to / and /var?

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

df /mnt
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/md2 457459232 430445268 3702844 100% /mnt
I mount /dev/md2 to /mnt
I'm using my os in Rescue and I was trying to see if there is any big files i can delete or something

 

[rason]

df /mnt
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/md2 457459232 430445268 3702844 100% /mnt
I mount /dev/md2 to /mnt
I'm using my os in Rescue and I was trying to see if there is any big files i can delete or something

I believe they meant you should provide us with the full df -h output, with no other arguments like "/mnt" involved

 

[bbgeek17]

If when you boot normally you still see no space left error in the log, then you should clean more space:
https://forum.proxmox.com/threads/h...s-drive-to-a-larger-drive.119090/#post-517702

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[ahmedsaber9]

After I delete some data the system boot normally, here is my df -h

df -h

Filesystem Size Used Avail Use% Mounted on
udev 32G 0 32G 0% /dev
tmpfs 6.3G 1.1M 6.3G 1% /run
/dev/md2 437G 411G 3.2G 100% /
tmpfs 32G 43M 32G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 32G 0 32G 0% /sys/fs/cgroup
/dev/md1 485M 126M 334M 28% /boot
/dev/fuse 30M 20K 30M 1% /etc/pve
tmpfs 6.3G 0 6.3G 0% /run/user/0

How could I find the useless files?