error "no space left on device"

[bu2chlc]

Hello all, I have a 3 node cluster (running ceph) version 6.3-2.
When I try to open a console to a VM, I get the error, "no space left on device". I get the error on all 3 nodes. I ran DF from each node but get similar results that do not show any disk being 100% used.
Where should I look? Have I included enough info to give you a hint at my problem?

 

[bofh]

well technically the error is correct which is the best type of correct.

let me answer with a question: where is your VM storage ?

 

[bu2chlc]

well technically the error is correct which is the best type of correct.

let me answer with a question: where is your VM storage ?

stored in a ceph pool, nowhere near being full, less than 10% used actually.

 

[ph0x]

Are you trying to open a VM's console or the host's console? If it's a VM then the assigned disk is probably full.

 

[bu2chlc]

Are you trying to open a VM's console or the host's console? If it's a VM then the assigned disk is probably full.

I cannot start VM's. the server has no free inodes and there are 900k files in the /var/lib/samba/private/msg.sock/ folder. I am using ceph if that helps. I am now trying to just find a thousand or so files so I can get the vms online to back them up somehow.

 

[fabian]

https://bugzilla.proxmox.com/show_bug.cgi?id=2333