Proxmox 6.04. with 2 VM, one is fine, second is giving error running I/O error

[klagio]

Hi, I have an Intel NUC machine, with proxmox installed.

Until this morning everything worked well for many many months now.

I rebooted the machine (unplugged) and replugged.

On the proxmox I have 2 VM, the first gives this "running I/O error)" so it is running, but I can't SSH to it

The second VM works perfectly fine.

WHat can I do?

 

[aaron]

How is your storage configured? Is the disk on which the VMs reside still good?
What is the SMART status of the disk? You can check it in the YourNode->Disks panel.

 

[klagio]

How is your storage configured? Is the disk on which the VMs reside still good?
What is the SMART status of the disk? You can check it in the YourNode->Disks panel.

Hi, well I will try to find out, its an SSD. is this below?

EDIT: on the far end for the first drive its written under WEAROUT -40%,, for the second drive its N/A

 

[klagio]

BTW I can STOP the VM, but I can't SHUTDOWN it, it gives error, so it keeps RUNNING (error I/O), until I stop it. When I start it, after stopped it, it runs for like 30 seconds and then gives this I/O error

 

[aaron]

Which storages are your VMs using? The local-lvm does look quite full!

Can you move a VMs disk to the other storage? A disk can be moved in the Hardware panel of the VM.

 

[klagio]

Which storages are your VMs using? The local-lvm does look quite full!

Can you move a VMs disk to the other storage? A disk can be moved in the Hardware panel of the VM.

How do you see its full?

I have 2 SSD in my NUC one M2 and one SSD (actually is an M2 in a SSD storage chassis). I always had problems understanding how/which are the two in the proxmox setup (I am kind of newb of proxmox and linux in general).
Could you help in how to identify, wwhich is which, and the space available?

On another note, this application is pretty important, now I am stuck,

I can read it, and wish to backup some content. Where do I find? Of VM100

/var/lib/vz/images

are the ISO images that IO use to build VM, but not the data

 

[klagio]

Is there anyway I can access the drive, and maybe remove some data (if its full), or access the data somehow?

I can SSH the proxmox machine

 

[klagio]

Which storages are your VMs using? The local-lvm does look quite full!

Can you move a VMs disk to the other storage? A disk can be moved in the Hardware panel of the VM.

that's my situation, anything I can do? Anyway I ordered a big SSD (500Gb) it will arrive in a week, wish to fix it though now if possible

 

[klagio]

When I go SSH in proxmox, under /dev/pve/ a lot of links but they bring to

error no such file

 

[aaron]

Well, looks like the local-lvm storage ran full. The VMID 100 will most likely not even boot right?

Is that VM a Linux guest? If so you could try to boot if from a Live ISO and issue a TRIM command on the VMs disk to free up unused blocks.

For this to work you will need to first change the Disk Bus Type to Virtio or SCSI (with VirtIO SCSI controller which is already set)

Then adapt the Boot Order in the Options panel of the VM.

First change it to CDROM to boot from the inserted live ISO.

The fstrim command should work to issue the trim on the filesystem of the VM.

After that you should hopefully see the disk usage going down in the local-lvm storage.

 

[klagio]

Well, looks like the local-lvm storage ran full. The VMID 100 will most likely not even boot right?

Is that VM a Linux guest? I

Well it does boot, in the sense that status is RUNNING (although with this I/O error).

Its Home Assistant OS, as far as I know, yes its Linux based.

Ok I will try to follow your instructions, seems not so easy, but will try

 

[klagio]

Well, looks like the local-lvm storage ran full. The VMID 100 will most likely not even boot right?

Is that VM a Linux guest? If so you could try to boot if from a Live ISO and issue a TRIM command on the VMs disk to free up unused blocks.

For this to work you will need to first change the Disk Bus Type to Virtio or SCSI (with VirtIO SCSI controller which is already set)

Then adapt the Boot Order in the Options panel of the VM.

First change it to CDROM to boot from the inserted live ISO.

The fstrim command should work to issue the trim on the filesystem of the VM.

After that you should hopefully see the disk usage going down in the local-lvm storage.

the disk is already VIRTIO SCSI, correct? (see image)

The boot order does not appear boot from USB (if I click the drop down menu is disksata0--cdrom-network-none)

 

[aaron]

No it is not. The SCSI controller is but the disk is still set to SATA.

 

[aaron]

To change the Bus Type of the disk you first have to detach it and then edit it. Once you save the changes it will be attached again.

 

[klagio]

Ok, I got it: to put boot order CD-ROM, I did, but it does not boot, and gives the same RUNNING (I/O ERROR)

 

[klagio]

Ok I think I made the changes (boot order, ONLY CD-ROM, and SCSI also to the disk), changed as attached pictures.

When I click start at becomes green arrow, but after few seconds, again RUNNING I/O ERROR-

If I detach the Harrd Disk, everything works, and it does boot from the USB, but when the Hard Disk is there it stops with the same error, even if boot order is CDROM

 

[aaron]

ah.. I missed that, of course you need to add a CDROM drive and point it to a live CD Iso that you uploaded first.

 

[klagio]

ah.. I missed that, of course you need to add a CDROM drive and point it to a live CD Iso that you uploaded first.

Ok I managed to boot up.

What I need to do is DETACH the SSD, then boot with live CD, and only THEN reattach the SSD (otherwise it goes into running I/O error).

Now the TRIM cokmmand is not there, the fstrim seems yes.

So I did

sudo fstrim -av

it returns nothing, and problem not solved. Will try to dig deeper into trim fstrim

 

[klagio]

So I run

sudo fdisk -l


and have this error on the SSD

The backup GPT table is corrupt, but the primary appears ok, so that will be used

 

[aaron]

Alternatively, you could also try to add an external HDD to your NUC, configure that as a storage and move the VMs disk there as a workaround.