Proxmox VE 7.4 crashing during ML workload and encountered GRUB Bootloader Error with Missing LVMID Disk

[benny2023]

Hi,

I am encountering a problem with Proxmox VE 7.4 crashing when running an ML workload. After multiple reboots, I am now encountered an error with GRUB bootloader with a missing disk identified by LVMID.

Welcome to GRUB!
error: disk `lvmid/KBiyoi-scvW-hp7G-hONS-yK3m-RfbW-t j2FQr/6WXtBv-jbDe-ux1R-uApc-21ae-bx8a-JkuDVa` not found.
grub rescue>

I followed the steps outlined in the Proxmox Wiki (https://pve.proxmox.com/wiki/Recover_From_Grub_Failure) to recover from GRUB failure, but unfortunately, I am unable to proceed. Specifically, the vgscan command returns nothing, and I am unsure what could be causing this issue.

I have also tried pvscan and fdisk -l, and have attached their output here. Additionally, pvck -t /dev/sdb3 returns "TEST MODE: Metadata will NOT be updated and volumes will not be deactivated. Found label on /dev/sdb3, sector 1, type=LVM2 001. Found text metadata area: offset=4096, size =1044400".

I appreciate any guidance or suggestions on how to proceed with the recovery process .

Thank you in advance for your help

 

[mira]

Did you boot into a Live CD and add a small LV as mentioned in the wiki?

• Boot from a live USB/CD/DVD with LVM support, e.g. grml
• Run vgscan
• Create a 4MB logical volume named grubtemp in the pve volume group: lvcreate -L 4M pve -n grubtemp

Basically anything that updates/writes the metadata helps fix the issue.

 

[benny2023]

Did you boot into a Live CD and add a small LV as mentioned in the wiki?

Basically anything that updates/writes the metadata helps fix the issue.

Hi Mira,

Yes i boot from the grml but vgscan returns nothing.

Thanks

 

[mira]

What's the output of pvs, vgs and lvs?