After Backup: boot failed: not a bootable disk

[Faris Raouf]

I don't like the sound of this at all. I cannot trust a system where performing a backup or restoring a backup can potentially cause data loss. This kind of thing stops me sleeping soundly.

We absolutely must find out what the root cause is.

On the positive side, I've restored backups from fast HDD ext4 to faster SSD lvm-thin (both local) with no issues (on 5.4). I have no idea how fast the restore was though.

As a precaution, I've made backups of the partition table of each VM, which either in this thread or another ones was what was being damaged as opposed to an even more serious situation, in the hope that should I be hit with this in the future, it might possibly save my skin.

 

[nttec]

nttec, you are making backup on NFS Storage ?

Yes

 

[nttec]

I don't like the sound of this at all. I cannot trust a system where performing a backup or restoring a backup can potentially cause data loss. This kind of thing stops me sleeping soundly.

We absolutely must find out what the root cause is.

On the positive side, I've restored backups from fast HDD ext4 to faster SSD lvm-thin (both local) with no issues (on 5.4). I have no idea how fast the restore was though.

As a precaution, I've made backups of the partition table of each VM, which either in this thread or another ones was what was being damaged as opposed to an even more serious situation, in the hope that should I be hit with this in the future, it might possibly save my skin.

My I ask how to do backup of the partition table? this will be a big help.

 

[Faris Raouf]

My I ask how to do backup of the partition table? this will be a big help.

See https://www.cyberciti.biz/faq/howto-copy-mbr/ (Linux sfdisk example a little down the page)
and also https://www.cyberciti.biz/faq/linux-backup-restore-a-partition-table-with-sfdisk-command/

I have made copies and stored them safely.

Having said that, examining the copies in a text editor, I see the words "disk read error" towards the end of both the 512 and 446 byte copies that I made and that makes me now wonder if they are actually restorable at all!

 

[David Thistlethwaite]

I don't like the sound of this at all. I cannot trust a system where performing a backup or restoring a backup can potentially cause data loss. This kind of thing stops me sleeping soundly.

We absolutely must find out what the root cause is.

On the positive side, I've restored backups from fast HDD ext4 to faster SSD lvm-thin (both local) with no issues (on 5.4). I have no idea how fast the restore was though.

As a precaution, I've made backups of the partition table of each VM, which either in this thread or another ones was what was being damaged as opposed to an even more serious situation, in the hope that should I be hit with this in the future, it might possibly save my skin.

I have not had any issues when performing backups, its only during the restore operations that I encounter issues.

 

[TheMrg]

same here, suddenly. 2vms are unbootable.
fdisk -l /dev/zvol/rpool/vm-100-disk-0
Festplatte /dev/zvol/rpool/vm-100-disk-0: 10 GiB, 10737418240 Bytes, 20971520 Sektoren
Einheiten: Sektoren von 1 * 512 = 512 Bytes
Sektorgröße (logisch/physikalisch): 512 Bytes / 8192 Bytes
E/A-Größe (minimal/optimal): 8192 Bytes / 8192 Bytes

there are no partitions left on the disk.
how to restore the data. restore backup: same
i have to restore 10 days old backup, this was fine. (10 days data lost).
we have 1 VM with no backup, it is not an important machine. but this problem is a big problem. btw: backup file size is 5gbs each (with the corrupt and the working one).

thanks so far

 

[TheMrg]

IIf you need recover this VM, contact me.

how can we restore the VM?

 

[TheMrg]

ok, we get a way to restore the VM (quick howto):
1) boot from gpated rescue cd (to have testdisk tool)
2) testdisk finds /dev/sda
3) quick search partition tables, finds 2 partitions (boot and lvm)
4) boot from centos7 rescure cd, chroot /mnt/sysimage
5) grub2-install /dev/sda to install grub bootloader, exit, init 6 ... reboot and online, be happy

why the partition table and boot loader was lost, we do not know. this is the log:

Aug 16 22:01:01 installer systemd: Started Session 516 of user root.
Aug 16 22:01:01 installer systemd: Removed slice User Slice of root.
Aug 16 22:08:01 installer systemd: Created slice User Slice of root.
Aug 16 22:08:01 installer systemd: Started Session 517 of user root.
Aug 16 22:08:32 installer systemd: Removed slice User Slice of root.
Aug 16 23:01:01 installer systemd: Created slice User Slice of root.
Aug 16 23:01:01 installer systemd: Started Session 518 of user root.
Aug 16 23:01:01 installer systemd: Removed slice User Slice of root.
Aug 16 23:44:09 installer kernel: ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
Aug 16 23:44:09 installer kernel: ata4.00: failed command: WRITE DMA
Aug 16 23:44:09 installer kernel: ata4.00: cmd ca/00:20:a0:65:c1/00:00:00:00:00/e0 tag 29 dma 16384 out#012 res 40/00
Aug 16 23:44:09 installer kernel: ata4.00: status: { DRDY }
Aug 16 23:44:09 installer kernel: ata4: hard resetting link
Aug 16 23:44:24 installer kernel: ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
Aug 16 23:44:24 installer kernel: ata4.00: configured for UDMA/100
Aug 16 23:44:24 installer kernel: ata4.00: device reported invalid CHS sector 0
Aug 16 23:44:24 installer kernel: ata4: EH complete
Aug 16 23:46:01 installer systemd: Created slice User Slice of root.
Aug 16 23:46:01 installer systemd-logind: New session 519 of user root.
Aug 16 23:46:01 installer systemd: Started Session 519 of user root.
Aug 16 23:46:15 installer qemu-ga: info: guest-ping called
Aug 16 23:46:15 installer qemu-ga: info: guest-fsfreeze called
Aug 16 23:46:15 installer qemu-ga: info: executing fsfreeze hook with arg 'freeze'
Aug 16 23:46:16 installer qemu-ga: info: executing fsfreeze hook with arg 'thaw'
Aug 16 23:47:51 installer systemd: Stopped target Multi-User System.
Aug 16 23:47:51 installer systemd: Closed LVM2 poll daemon socket.
Aug 16 23:47:51 installer systemd: Stopping LVM2 PV scan on device 8:2...
Aug 16 23:47:51 installer systemd: Stopping QEMU Guest Agent...
Aug 16 23:47:51 installer systemd: Stopped target Login Prompts.
Aug 16 23:47:51 installer systemd: Stopping Command Scheduler...
Aug 17 02:58:40 installer kernel: Initializing cgroup subsys cpuset
Aug 17 02:58:40 installer kernel: Initializing cgroup subsys cpu
Aug 17 02:58:40 installer kernel: Initializing cgroup subsys cpuacct
Aug 17 02:58:40 installer kernel: Linux version 3.10.0-1127.el7.x86_64 (mockbuild@kbuilder.bsys.centos.org

 

[tnt]

hi,

i use this procedure for repair boot record:
(similar as write: TheMrg)

-download gparted live cd (use latest vesrion https://gparted.org/download.php amd64.iso)
- boot from gparted live cd
- i use default setting. (press 3x enter)
- after starting gparted, i close this program
-run terminal icon
-switch to root: sudo su
-run: testdisk
- select: no log + enter
-select: hdd /dev/sda + proceed
-select: intel
-select: analyse
-select: quick search
-select: linux (partition) + enter
-then i see partion on hdd. select: wrtite
-confirm: yes
-press enter
- close test disk: ctrl+c

now with: fdisk -l /dev/sda
i see partition table

- run: mount /dev/sdaXX /mnt (if you use lvm see end)
- run: grub-install --no-floppy --root-directory=/mnt /dev/sda
this must be without error

close terminal
click on red exit icon.
select : shutdown

select virtual pc boot from hdd ....

this works form me, for repair : no bootable device

if you use lvm:
vgscan --mknodes
vgchange -ay
lvdisplay (see LV Path)
mount {LV_PATH} /mnt/



now:
i have backup MBR from all my virtual linux PC
one copy i have in virtual
second on my storage

 

[christian7mx]

If you need support, I do this for you. Recover your VM data, you can pay me via Paypal. Send me a PM, bye!

Hi,i have the same problem, can you help me? What is the cost?

 

[pcready.cl]

Hi,i have the same problem, can you help me? What is the cost?

Hi, What is the version of PVE you have? And what operating system does the VM have installed?

 

[christian7mx]

Hi, What is the version of PVE you have? And what operating system does the VM have installed?

6.3-6 and Centos 7


Im could restore partitions but to initialization post grub is stopped in

"Startd update UTMP about system Boot/shutdown"



I working in a VM duplicated, the original VM is offline

 

[tnt]

i set on all VM (i use old proxmox 5.1-35)
hdd to: ide0

and i dont get: no bootable device

 

[RolandK]

this may be an issue with virtual sata disk.

https://bugzilla.proxmox.com/show_bug.cgi?id=2874#c51


can you confirm that this issue did only happen with virtual sata (or maybe ide) disks ?

 

[tnt]

this may be an issue with virtual sata disk.

https://bugzilla.proxmox.com/show_bug.cgi?id=2874#c51


can you confirm that this issue did only happen with virtual sata (or maybe ide) disks ?

i think with ide this issue dont happen

 

[David Thistlethwaite]

how can we restore the VM?

I have had success restoring such vm's at a very low restore data rate, that seemed to help

 

[David Thistlethwaite]

this may be an issue with virtual sata disk.

https://bugzilla.proxmox.com/show_bug.cgi?id=2874#c51


can you confirm that this issue did only happen with virtual sata (or maybe ide) disks ?

I only use sata disks so I cannot comment on ide drives