Hello there,
I've dumped my reliable XPEnology setup for the sake of being more flexible (and legally compliant) with Proxmox and I'm slowly starting to regret it. I hope I can get it fixed with your help though... Here's what I've got and what I have tried:
* Proxmox 5.3-2 installed via USB, then updated (as below)
* 4x3TB harddrives in the 4 bays of the Microserver Gen8 G1610T
* SATA is configured as AHCI (not RAID)
* RAIDZ-1 over all 4 harddrives,
* rootfs on the zpool as created by Proxmox installer
I've done several reinstallations because I initially thought I was the problem. During the process, I encountered that after apt-get-upgrading the system, the next boot will fail. I'm sent to the GRUB rescue mode shell with the error message "no such device: 1b2406d9cc9ad984". I've been on a long journey through a lot of threads on the web, also on this forum, but all the instructions and good ideas didn't lead me to success. My actions boil down to the following:
I've also explored my GRUB configuration at that point which told me which modules to insert:
Inserting the modules worked, except for the ls part. There I get the message "unknown filesystem". Setting debug=zfs, I can see that the actual checksum only consists of zeroes while the expected checksum is some generic hex. (EDIT: After another try, there's an actual checksum that mismatches:
) Googling that, I found some information about some bug, but I can't recall now and I'm pretty sure it was for another (older) GRUB version. Anyway, I'm also struggling to find out where that hex ID that GRUB wants to use initially comes from. Reading /boot/grub/grub.cfg, I can see that the ID "1b2406..." is in the end of the "search" commands. As I grep through /etc/grub.d and /etc/default/grub, from where the file is supposed to be generated, I find nothing.
Edit: Here's a part of my grub.cfg where the UID resides:
In the meanwhile I've also tried to copy my /boot directory from the zfs to a USB drive and install GRUB on there but still no success booting the system. The zpool itself is healthy and has also completed a scrub without any errors. One strange thing is that I cannot access my VM files even though the zfs seems to be mounted, but that's most certainly my own mistake since I can see the datasets using "zfs list".
It appears to me that using zfs as root volume is a fairly new thing or not as supported as other options. Is that true? And, most importantly, what can I do to make my PVE installation boot again? I've had to survive five days of downtime now and I'm pretty sick of it.
Another EDIT:
I just encountered that the proxmox installer inserts USB drives inbetween the four disks (sda=bay1,sdb=bay2,sdc=usbdrive,sdd=usbdrive,sde=bay3,sdf=bay4). I read somewhere else that the Proxmox installer installs the zraid to the /dev/sdX devices instead of using UUIDs. Could this be related since I installed from USB?
I've dumped my reliable XPEnology setup for the sake of being more flexible (and legally compliant) with Proxmox and I'm slowly starting to regret it. I hope I can get it fixed with your help though... Here's what I've got and what I have tried:
* Proxmox 5.3-2 installed via USB, then updated (as below)
* 4x3TB harddrives in the 4 bays of the Microserver Gen8 G1610T
* SATA is configured as AHCI (not RAID)
* RAIDZ-1 over all 4 harddrives,
* rootfs on the zpool as created by Proxmox installer
I've done several reinstallations because I initially thought I was the problem. During the process, I encountered that after apt-get-upgrading the system, the next boot will fail. I'm sent to the GRUB rescue mode shell with the error message "no such device: 1b2406d9cc9ad984". I've been on a long journey through a lot of threads on the web, also on this forum, but all the instructions and good ideas didn't lead me to success. My actions boil down to the following:
Code:
#Download and boot sysresccd-5.2.0_zfs_0.7.9.iso
mkdir /zfsmnt && zpool import rpool -R /zfsmnt
mount -t /proc /zfsmnt/proc
mount --rbind /dev /zfsmnt/dev
mount --rbind /sys /zfsmnt/sys
chroot /zfsmnt /bin/bash # yay, i've now chrooted into my pve install
apt-get update && apt-get upgrade && apt-get dist-upgrade # There was a kernel update and I hoped to solve any inconsistencies within boot-related configuration
grub-install /dev/sda && grub-install /dev/sdb && grub-install /dev/sdc && grub-install /dev/sdd # all without errors
update-initramfs -u
grub-probe / # returns zfsI've also explored my GRUB configuration at that point which told me which modules to insert:
Code:
insmod part_gpt
insmod zfsEdit: Here's a part of my grub.cfg where the UID resides:
In the meanwhile I've also tried to copy my /boot directory from the zfs to a USB drive and install GRUB on there but still no success booting the system. The zpool itself is healthy and has also completed a scrub without any errors. One strange thing is that I cannot access my VM files even though the zfs seems to be mounted, but that's most certainly my own mistake since I can see the datasets using "zfs list".
It appears to me that using zfs as root volume is a fairly new thing or not as supported as other options. Is that true? And, most importantly, what can I do to make my PVE installation boot again? I've had to survive five days of downtime now and I'm pretty sick of it.
Another EDIT:
I just encountered that the proxmox installer inserts USB drives inbetween the four disks (sda=bay1,sdb=bay2,sdc=usbdrive,sdd=usbdrive,sde=bay3,sdf=bay4). I read somewhere else that the Proxmox installer installs the zraid to the /dev/sdX devices instead of using UUIDs. Could this be related since I installed from USB?
Last edited:
