Hello!
About a month ago I've decided to give Proxmox-VE a try. The installation and the creation of a few test VMs went without hitches. Everything was working smooth (reboots also) until yesterday when I performed dist-upgrade to version 5.1.42. After the upgrade I had to reboot because of the new kernel version and after the reboot the Grub failed to continue after stage 1. It gives some errors and then drops to rescue.
Machine setup info
ProLiant DL360e Gen8 (not UEFI) with two 4TB disks. Proxmox-VE with ZFS in mirror (RAID1) mode.
Here is the Grub stage 1 output:
What I've tried so far
- Used the Proxmox-VE CD to boot into rescue environment. Then imported and chrooted the zfs pool, and after that re-installed grub to /dev/sda and dev/sdb and regenerated the grub config. After reboot the boot process ended again with the same grub errors.
Useful detailed info
returns zfs
corresponds to sda2/sdb2 (zfs pool) when converted to decimal
The output of the blkid is redacted (zfs volumes removed) for brevity.
I guess I can use thumb drive (as a last resort) to install the boot partition on and boot from it.
Do you have anything in mind what is going on as I am kinda getting out of ideas? Any help would be highly appreaciated! Thank you in advance!
About a month ago I've decided to give Proxmox-VE a try. The installation and the creation of a few test VMs went without hitches. Everything was working smooth (reboots also) until yesterday when I performed dist-upgrade to version 5.1.42. After the upgrade I had to reboot because of the new kernel version and after the reboot the Grub failed to continue after stage 1. It gives some errors and then drops to rescue.
Machine setup info
ProLiant DL360e Gen8 (not UEFI) with two 4TB disks. Proxmox-VE with ZFS in mirror (RAID1) mode.
Here is the Grub stage 1 output:
Code:
Attempting Boot From Hard Drive (C:)
error: no such device 295864b8ee73d9fe.
error: unknown filesystem.
Entering rescue mode ...What I've tried so far
- Used the Proxmox-VE CD to boot into rescue environment. Then imported and chrooted the zfs pool, and after that re-installed grub to /dev/sda and dev/sdb and regenerated the grub config. After reboot the boot process ended again with the same grub errors.
Useful detailed info
Code:
grub-probe /
Code:
295864b8ee73d9fe
Code:
grub-rescue> ls
(hd0) (hd0,gpt9) (hd0,gpt2) (hd0,gpt1) (hd1) (hd1,gpt9) (hd1,gpt2) (hd1,gpt1)
grub-rescue> ls (hd0,gpt2)
(hd0,gpt2): Filesystem is unknown.
grub-rescue> set
cmdpath=(hd0)
prefix=(hd0)/ROOT/pve-1@/boot/grub
root=hd0
grub-rescue> insmod normal
error: unknown filesystem.
grub-rescue> insmod zfs
grub-rescue> set debug=zfs
grub-rescue> ls (hd0,gpt2)
fs/zfs/zfs.c:1192: label ok 0
fs/zfs/zfs.c:1007: check 2 passed
fs/zfs/zfs.c:1018: check 3 passed
fs/zfs/zfs.c:1025: check 4 passed
fs/zfs/zfs.c:1035: check 6 passed
fs/zfs/zfs.c:1043: check 7 passed
fs/zfs/zfs.c:1054: check 8 passed
fs/zfs/zfs.c:1064: check 9 passed
fs/zfs/zfs.c:1086: check 11 passed
fs/zfs/zfs.c:1112: check 10 passed
fs/zfs/zfs.c:1128: str=com.delphix:hole_birth
fs/zfs/zfs.c:1128: str=com.delphix:embedded_data
fs/zfs/zfs.c:1137: check 12 passed (feature flags)
fs/zfs/zfs.c:1878: zio_read: E 0: size 2048/2048
fs/zfs/zfs.c:1899: endian = -1
fs/zfs/zfs.c:595: dva=8, 11c0717f8
fs/zfs/zfs.c:442: checksum feltcher4 verification failed
fs/zfs/zfs.c:447: actual checksum 0000000000000000 0000000000000000 0000000000000000 0000000000000000
fs/zfs/zfs.c:452: expected checksum 00000004498e0fdc 000007fa34d3423a ee2726 04a3580d3cb4fb7d
fs/zfs/zfs.c:1922: incorrect checksum
(hd0,gpt2): Filesystem is unknown.
grub-rescue>
Code:
root@proxmox:/# blkid
/dev/sda2: LABEL="rpool" UUID="2979241898942913022" UUID_SUB="11474682908795965178" TYPE="zfs_member" PARTLABEL="zfs" PARTUUID="1a99e21a-3305-4f4c-93e2-1c4ce9490810"
/dev/sdb2: LABEL="rpool" UUID="2979241898942913022" UUID_SUB="10144792332914432543" TYPE="zfs_member" PARTLABEL="zfs" PARTUUID="5a3ee6dc-8185-4cc5-aaaf-c7a89a399613"
/dev/sda1: PARTUUID="redacted"
/dev/sda9: PARTUUID="redacted"
/dev/sdb1: PARTUUID="redacted"
/dev/sdb9: PARTUUID="redacted"
root@proxmox:/#The output of the blkid is redacted (zfs volumes removed) for brevity.
Code:
root@proxmox:/# zpool get all
NAME PROPERTY VALUE SOURCE
rpool size 3.6T -
rpool capacity 1% -
rpool altroot /mnt default
rpool health ONLINE -
rpool guid 2979241898942913022 -
rpool version - default
rpool bootfs rpool/ROOT/pve-1 local
rpool delegation on default
rpool autoreplace off default
rpool cachefile none local
rpool failmode wait default
rpool listsnapshots off default
rpool autoexpand off default
rpool dedupditto 0 default
rpool dedupratio 1.00x -
rpool free 3.57T -
rpool allocated 57.0G -
rpool readonly off -
rpool ashift 12 local
rpool comment - default
rpool expandsize - -
rpool freeing 0 -
rpool fragmentation 0% -
rpool leaked 0 -
rpool multihost off default
rpool feature@async_destroy enabled local
rpool feature@empty_bpobj active local
rpool feature@lz4_compress active local
rpool feature@multi_vdev_crash_dump enabled local
rpool feature@spacemap_histogram active local
rpool feature@enabled_txg active local
rpool feature@hole_birth active local
rpool feature@extensible_dataset active local
rpool feature@embedded_data active local
rpool feature@bookmarks enabled local
rpool feature@filesystem_limits enabled local
rpool feature@large_blocks enabled local
rpool feature@large_dnode enabled local
rpool feature@sha512 enabled local
rpool feature@skein enabled local
rpool featureeedonr enabled local
rpool feature@userobj_accounting active local
root@proxmox:/#I guess I can use thumb drive (as a last resort) to install the boot partition on and boot from it.
Do you have anything in mind what is going on as I am kinda getting out of ideas? Any help would be highly appreaciated! Thank you in advance!
