Where is boot/grub in zfs root for grub rescue?

[mathx]

Read lots of threads about grub rescue, but short of mounting a live iso to boot from (can't right now, the remote kvm is half busted), we're stuck at grub rescue>

however, looking at (hd0) throught (hd7) (the max # of drives presented to bios by the JBOD controller), I can see (hd0,gpt2)/ROOT/pve-1@/boot and (hd[1-7],gpt1)/ROOT/pve-1@/boot but there's no grub/ subdir in any of them just vmlinuz, initrd, system.map etc as you'd expect.

So where's the grub subdir hiding? Im not sure why (hd0,gpt1) isnt working (perhaps thats a corrupted partition that caused the crash) - is that where it is, the only copy on this zfs 16-drive raid10? Am I stuck having to visit the machine in the datacentre or get a usb/iso working?

 

[fabian]

Read lots of threads about grub rescue, but short of mounting a live iso to boot from (can't right now, the remote kvm is half busted), we're stuck at grub rescue>

however, looking at (hd0) throught (hd7) (the max # of drives presented to bios by the JBOD controller), I can see (hd0,gpt2)/ROOT/pve-1@/boot and (hd[1-7],gpt1)/ROOT/pve-1@/boot but there's no grub/ subdir in any of them just vmlinuz, initrd, system.map etc as you'd expect.

So where's the grub subdir hiding? Im not sure why (hd0,gpt1) isnt working (perhaps thats a corrupted partition that caused the crash) - is that where it is, the only copy on this zfs 16-drive raid10? Am I stuck having to visit the machine in the datacentre or get a usb/iso working?

the grub subdir should be exactly where you are looking (/boot/grub, as part of the rpool/ROOT/pve-1 dataset). the only thing that only the first redundant vdev group gets is the stage1(.5) of grub on a BIOS boot partition.

 

[mathx]

Yeah, boot/grub is just not there for whatever reason. Bizarre.

This was a pve 3.4 system with zfs raid 10 (with 12 drives I think now, I've expanded it, originally was 4 at install time).

I booted a Ubuntu 16.04 desktop usb to live and installed the zfs packages (apt-get install zfsutils-linux), imported the pool with -R /mnt , mount --bind /dev /mnt/dev; chroot /mnt; mount -t proc proc /proc; mount -t sysfs sys /sys; then grub-install /dev/sda through sdn (in case i choose a diff boot drive from JBOD card bios, i think one disk has a problem booting which caused this whole fiasco though I cant find any disk errors with smartctl in ubuntu live). But Im still dropping to grub rescue> when I reboot the system off hdd.

In grub rescue now I cant even read any of the partitions, "error: compression algorithm inherit not supported". This would be understandable if I installed grub from another system (like the ubuntu live) but I did not - I chrooted into the rpool, and used grub-install from the original system's root disk and binaries. (I had never upgraded it past 3.4 original install or anything.) insmod zfs in rescue works, but zfsinfo gives "command not found".

Little confused as to what's going on. Next step is to install pve (and upgrade while Im at it I guess, to 5.1) onto 2 new disks raid 1, and then import the old filesystem to get at the data. (hoping this "inherit not supported" doenst stop me there too!)

I hopefully can then get the data, but I guess Ill never be able to merge/combine the pools. Have to copy out of the old pool into new (though there's 20TB, I need to get 20TB going in raid 10 in 4 disk slots...).