VM can't boot / attempt to read or write outside of disk 'hd0'.

[eisteed]

I'm using proxmox without any problems for some times now, suddenly my vm got some really weird error read/write problems in apache web server.
But the files and folder were there, I could download them from ftp, so I decided to reboot the vm just to see if anything was going wrong on reboot.

But the vm won't boot,

attempt to read or write outside of disk 'hd0'.
Loading Linux 4.19.0-9amd64...
attempt to read or write outside of disk 'hd0'.
Loading initial ramdisk ...
error: you need to load the kernel first.

The vm is running on ZFS mirror 2 x 4TB drives
I have some important datas on this VM and I really don't know what to do to fix this... any help really appreciated.

 

[oguz]

hi,

you can try restoring a snapshot or a backup.

are other VMs working normally?

suddenly my vm got some really weird error read/write problems in apache web server.

randomly? or after an upgrade or anything similar?

it could be a filesystem problem in the virtual disk, which could be fixed by fsck
also could be a bootloader problem, like after an upgrade (grub-related?)
or if other VMs are behaving weird as well, it could be a hardware issue on your host

 

[eisteed]

Hello!
Sadly I have no backup/snapshot recent enough, and I'd need one more 4tb drive to have enough space :/ (yes I should add one..)

The errors came suddenly after installing/updating PHP version if I'm remember correctly, but this is a bit weird to the actual error (can't find a reason why it would break boot). I think the error can be caused by the 3.5 TB virtual disk but I really don't know why/how, seems that other people having similar issues are having large disks to, for my part the grub partition is at the beginning of the disk (if this can affect boot).

Currently I have no problem to read/write on the 3.5 TB disk from a live cd or check partitions from gparted.
Other vms are working fine so I don't think this is an hardware related issue.

How can I use fsck to fix ? From grub ?
Thank you for your answer

 

[oguz]

How can I use fsck to fix ? From grub ?

you can mount the virtual disk on your host and run it there, or just boot the VM from a livecd

 

[eisteed]

Ok so I did try your suggestion booting from a newly installed debian on a separate drive (sdb)
When trying :
fsck /dev/sda

I get :
bad magic number in super-block while trying to open dev/sda
[...]
Found a gpt partition table in /dev/sda

Here's a screenshot of the whole response
Even though I did backup my files I'm a bit afraid of what to do next :S

 

[eisteed]

After more testing,
/dev/sda1 = same problem, bad magic number in super-block (bios)
/dev/sda2 = clean (data/files)
/dev/sda3 = same problem (swap)

I guess proxmox may have nothing to do with the problem, a part from booting method (uefi or not but I'm a bit lost and really not an expert on that part) I really don't understand how that problem can occur..

 

[oguz]

most likely the virtual disk got somehow corrupted (maybe an abrupt shutdown of the VM while it was writing files? just a guess)

you can try to run e2fsck like it suggests in the output, first i would make a backup of the data just to be sure.

 

[eisteed]

I made a backup and ran e2fsck as suggested, but the output is the same for both proposed command
Still "Bad magic number in super-block while trying to open /dev/sda"

And the last reboot I did was from command line so no hard reboot done before this problem ;(
I guess I'll have to reinstall everything and copy my backup.

 

[eisteed]

Just for information on how to do things correctly, is it better to
Create a ZFS (mirror 2x4TB)

and in that :
Create a disk for OS only
Create a second disk for data / files
Create a third disk for backups / snapshots

Thanks for your advices, and thank you for your time / answers.

 

[oguz]

are you reinstalling the whole thing or just the VM?

you could just recreate the VM using the same config, setup the OS and just restore the data from your backup

 

[eisteed]

No just the VM but I want to not have the OS and all the data on the same disk to avoid such problem in the future.
The thing is that I don't have enough physical space to recreate another vm with the same config in proxmox. (eventhought the space is not really used).