GRUB "no server is specified" / can't boot after extending pve/root LV onto second disk — bug ruled out

[trev458]

Setup:

• Proxmox VE, boot disk originally a 128GB drive (/dev/sda, GPT: sda1 BIOS boot, sda2 ESP, sda3 LVM member)
• pve VG originally lived entirely on sda3
• Recently extended pve VG by adding /dev/nvme0n1p1 (a separate ~128GB NVMe) as a second PV, then lvextend -l +100%FREE /dev/pve/root + resize2fs — completed successfully, system ran fine for weeks afterward
• Also present: a second, unrelated VG pveold (renamed from pve-old during troubleshooting) on a separate ~930GB NVMe, containing 17 thin-provisioned LVs (unrelated VM/CT disks)
• Legacy BIOS boot (i386-pc), GRUB 2.12-9+pmx2

Symptom:
After a routine kernel update + reboot, system drops straight to grub> rescue prompt. No boot menu shown.


What I've tried/ruled out:

1. insmod lvm; insmod ext2; ls (lvm/pve-root)/boot/grub → error: no server is specified.
2. Even bare ls (lvm) (before specifying any VG/LV) fails with the same error — happens before GRUB even resolves a specific volume.
3. Booted Proxmox VE 9.2 ISO Rescue Boot / debug shell, confirmed via lvm vgs/lvm lvs that pve/root and pve/swap are intact and activate fine under the kernel's own LVM (-wi-a-----), i.e. this is GRUB-specific, not actual LVM corruption.
4. Chrooted in, ran update-initramfs -u -k all and update-grub cleanly (no errors), then grub-install /dev/sda — "Installation finished. No error reported." Still boots to grub> after.
5. Checked for the known LVM metadata ring-buffer wraparound bug (bugs.debian.org #987008) using vgscan -vvv 2>&1 | grep "Reading metadata" — all PVs show (+0), so no wraparound. Also tried the documented workaround (lvchange --addtag to force a metadata rewrite) — no change.
6. Renamed pve-old → pveold in case the embedded hyphen in the VG name was tripping GRUB's LVM parser — no change.
7. proxmox-boot-tool status confirms this system does not use the ESP-based boot scheme (/etc/kernel/proxmox-boot-uuids does not exist) — legacy grub-pc only, ESP sync is correctly skipped.

Current working theory: Since pve/root now spans two physical disks (sda3 + nvme0n1p1) and even a bare ls (lvm) fails before resolving any specific LV, I suspect GRUB's LVM module has trouble with a logical volume whose extents span multiple PVs — separate from the documented ring-buffer bug. Haven't found this specific scenario documented anywhere yet.


Question: Has anyone hit this specific multi-PV-spanning-LV GRUB failure? Is the only real fix to shrink pve/root back onto a single PV and move /boot (or the whole root) off LVM entirely, or is there a known workaround?


System boots fine from the rescue ISO chroot, so no data at risk currently — just can't boot unattended.

 

[tschmidt]

Are you sure you're booting from sda? Booting an outdated grub stage1 from another drive has troubled me in the past.

 

[Kingneutron]

I would reinstall with LVM root on (1) disk, and create a separate LVM space for the 2nd disk. Restore LXCs/VMs from backups

It's generally not recommended to extend LVM over multiple PVs unless you have RAID underneath. Personally I would not extend a boot/root LVM over devices - as you have seen, it can interfere with booting. K.I.S.S. principle.

https://github.com/kneutron/ansitest/blob/master/proxmox/bkpcrit-proxmox.sh

https://github.com/kneutron/ansitest/blob/master/proxmox/proxmox-create-additional-lvm-thin.sh

 

[trev458]

Resolved: GRUB rescue prompt after kernel update — root cause was multi-PV LVM boot


System: Proxmox VE 9.2, UEFI boot, pve VG originally spanning two physical volumes (/dev/sda3 + a second NVMe partition, added earlier via vgextend/lvextend to grow root).


Symptom: After a routine kernel update + reboot, system dropped straight to grub> rescue prompt, no menu. update-initramfs, update-grub, and grub-install all ran clean but didn't fix it.


Root cause: GRUB cannot reliably boot from an LVM logical volume whose extents span multiple physical volumes — this is a known, still-open Debian bug (Bug#1035317). It doesn't matter that the LV mounts and reads fine under normal Linux LVM; GRUB's own LVM-parsing code chokes on it.


Fix: Booted from the Proxmox installer ISO in Rescue/Debug mode, chrooted in, then:

1. Shrunk the filesystem (resize2fs) and LV (lvreduce) so all extents fit on the single original PV
2. pvmove to consolidate extents off the second PV (may report "no data to move" if they already landed on the first PV after shrinking)
3. vgreduce pve /dev/<second-pv> to drop the second PV from the VG
4. Re-ran update-initramfs -u -k all and update-grub, rebooted

VG is now back to single-PV and boots normally. No general fix exists for the multi-PV case other than keeping /boot's LV (or the whole root LV, in this setup) on one PV.

 

[fabian]

you could also do what proxmox-boot-tool does and put the grub stage2, kernel and initrd on the ESP - grub only needs to read from the ESP then, and the kernel takes over before LVM is involved at all.