Grub Rescue ZFS Errors on Proxmox 5.3

R

redmop

Well-Known Member
Feb 24, 2015
121
2
58
Fresh Proxmox 5.3 install and updated.

After I've moved my data over and rebooted a couple more times, it started giving me this error right before the grub rescue> prompt "error: compression algorithm inherit not supported"

If I "ls (hd0,gpt3)/ROOT@/, it gives me the following error: "error: compression algorithm 119 not supported"

4 disk raidz with compression=on and changed to compesssion=lz4 after this happened.

If I boot pmagic, zfs mounts just fine and I can chroot and read everything. I've reinstalled zfs, zfs-initramfs, grub*,, updated initramfs and grub, and ran grub-install. (https://forum.proxmox.com/threads/need-to-regenerate-grub-at-each-reboot.44810/#post-215439)

Any ideas on how I can get this going again?
 
Dell Poweredge T130 with no raid card. Proxmox 5.3 fresh install via the installler fully updated raid z with compression=on. Ran on 4.x for years.
 
Just tried reproducing the setup here - (RAIDZ1 rpool with compression on, rpool/ROOT/pve-1 having compression inherited) and it boots...

which grub version do you have installed? (2.02-pve6 here)
 
2.02-pve6
It booted a couple times before this happened. It fully installed and patched.

I've ran grub-install several times. I've force reinstalled grub*, I've regenerated initramfs (I don't think that will help anyway)
 
* you ran `grub-install` on all disks that are part of the RAIDZ?
* could you please post the output of:
** `zpool status`
** `zfs list`
** `zfs get all rpool/ROOT/pve-1` (and all its ancestors for the compression)
** `zpool history`


As for a fix/workaround - you could try to `zfs send/recv` your root dataset (first to some external drive and then back to a new dataset, which has compressions=lz4 explicitly , and afterwards rename it to the original pve-1 name)
 
Yes, grub-install on all the disks. It also did that when I reinstalled grub.

root@PartedMagic:~# zpool status
pool: rpool
state: ONLINE
scan: scrub repaired 0B in 0h0m with 0 errors on Wed Jan 2 21:18:11 2019
config:

NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
wwn-0x6668794066075013120x-part3 ONLINE 0 0 0
wwn-0x13675550682808209408x-part3 ONLINE 0 0 0
wwn-0x13202954171141214208x-part3 ONLINE 0 0 0
wwn-0x13669317444016164864x-part3 ONLINE 0 0 0

root@PartedMagic:~# zfs list
NAME USED AVAIL REFER MOUNTPOINT
rpool 6.32T 3.89T 151K /rpool
rpool/ROOT 2.05G 3.89T 140K /rpool/ROOT
rpool/ROOT/pve-1 2.05G 3.89T 2.05G /
rpool/borgtest 140K 3.89T 140K /rpool/borgtest
rpool/data 140K 3.89T 140K /rpool/data
rpool/migrate-tmp 6.28T 3.89T 151K /rpool/migrate-tmp
rpool/migrate-tmp/proxmox3 1.39T 3.89T 128K /rpool/migrate-tmp/proxmox3
rpool/migrate-tmp/proxmox3/vms 1.39T 3.89T 140K /rpool/migrate-tmp/proxmox3/vms
rpool/migrate-tmp/proxmox3/vms/old 128K 3.89T 128K /rpool/migrate-tmp/proxmox3/vms/old
rpool/migrate-tmp/proxmox3/vms/pve-config 47.0M 3.89T 587K /rpool/migrate-tmp/proxmox3/vms/pve-config
rpool/migrate-tmp/proxmox3/vms/sync-15m 6.45G 3.89T 128K /rpool/migrate-tmp/proxmox3/vms/sync-15m
rpool/migrate-tmp/proxmox3/vms/sync-15m/vm-202-disk-1 6.45G 3.89T 6.45G -
rpool/migrate-tmp/proxmox3/vms/sync-1d 128K 3.89T 128K /rpool/migrate-tmp/proxmox3/vms/sync-1d
rpool/migrate-tmp/proxmox3/vms/sync-1h 128K 3.89T 128K /rpool/migrate-tmp/proxmox3/vms/sync-1h
rpool/migrate-tmp/proxmox3/vms/sync-custom 36.9G 3.89T 128K /rpool/migrate-tmp/proxmox3/vms/sync-custom
rpool/migrate-tmp/proxmox3/vms/sync-custom/vm-100-disk-1 36.9G 3.89T 36.9G -
rpool/migrate-tmp/proxmox3/vms/vm-102-disk-1 16.5G 3.89T 4.68G -
rpool/migrate-tmp/proxmox3/vms/vm-103-disk-1 30.9G 3.89T 7.97G -
rpool/migrate-tmp/proxmox3/vms/vm-103-disk-2 1.24T 3.89T 1.11T -
rpool/migrate-tmp/proxmox3/vms/vm-105-disk-1 7.09G 3.89T 2.18G -
rpool/migrate-tmp/proxmox3/vms/vm-106-disk-1 21.1G 3.89T 11.3G -
rpool/migrate-tmp/proxmox3/vms/vm-112-disk-1 8.39G 3.89T 4.39G -
rpool/migrate-tmp/proxmox3/vms/vm-130-disk-1 20.5G 3.89T 13.9G -
rpool/migrate-tmp/proxmox3/vms/vm-202-disk-1 11.5G 3.89T 11.5G -
rpool/migrate-tmp/proxmox4 4.89T 3.89T 128K /rpool/migrate-tmp/proxmox4
rpool/migrate-tmp/proxmox4/vms 4.89T 3.89T 140K /rpool/migrate-tmp/proxmox4/vms
rpool/migrate-tmp/proxmox4/vms/old 110G 3.89T 128K /rpool/migrate-tmp/proxmox4/vms/old
rpool/migrate-tmp/proxmox4/vms/old/vm-129-disk-1.bak 20.9G 3.89T 7.31G -
rpool/migrate-tmp/proxmox4/vms/old/vm-129-disk-2.bak 89.2G 3.89T 638G -
rpool/migrate-tmp/proxmox4/vms/pve-config 53.5M 3.89T 587K /rpool/migrate-tmp/proxmox4/vms/pve-config
rpool/migrate-tmp/proxmox4/vms/sync-15m 128K 3.89T 128K /rpool/migrate-tmp/proxmox4/vms/sync-15m
rpool/migrate-tmp/proxmox4/vms/sync-1d 128K 3.89T 128K /rpool/migrate-tmp/proxmox4/vms/sync-1d
rpool/migrate-tmp/proxmox4/vms/sync-1h 128K 3.89T 128K /rpool/migrate-tmp/proxmox4/vms/sync-1h
rpool/migrate-tmp/proxmox4/vms/sync-custom 128K 3.89T 128K /rpool/migrate-tmp/proxmox4/vms/sync-custom
rpool/migrate-tmp/proxmox4/vms/vm-117-disk-1 19.1G 3.89T 2.72G -
rpool/migrate-tmp/proxmox4/vms/vm-118-disk-1 2.24G 3.89T 2.24G -
rpool/migrate-tmp/proxmox4/vms/vm-126-disk-1 3.71G 3.89T 2.25G -
rpool/migrate-tmp/proxmox4/vms/vm-126-disk-2 81.4K 3.89T 81.4K -
rpool/migrate-tmp/proxmox4/vms/vm-129-disk-1 760G 3.89T 3.97G -
rpool/migrate-tmp/proxmox4/vms/vm-129-disk-2 3.03T 3.89T 788G -
rpool/migrate-tmp/proxmox4/vms/vm-202-disk-1 6.45G 3.89T 6.45G -
rpool/migrate-tmp/proxmox4/vms/vm-209-disk-1 8.21G 3.89T 8.21G -
rpool/migrate-tmp/proxmox4/vms/vm-211-disk-1 65.5G 3.89T 65.5G -
rpool/migrate-tmp/proxmox4/vms/vm-219-disk-1 30.4G 3.89T 3.83G -
rpool/migrate-tmp/proxmox4/vms/vm-219-disk-2 894G 3.89T 654G -
rpool/migrate-tmp/proxmox4/vms/vm-300-disk-1 3.70G 3.89T 2.25G -

root@PartedMagic:~# zfs get all rpool/ROOT/pve-1
NAME PROPERTY VALUE SOURCE
rpool/ROOT/pve-1 type filesystem -
rpool/ROOT/pve-1 creation Wed Jan 2 20:16 2019 -
rpool/ROOT/pve-1 used 2.05G -
rpool/ROOT/pve-1 available 3.89T -
rpool/ROOT/pve-1 referenced 2.05G -
rpool/ROOT/pve-1 compressratio 2.03x -
rpool/ROOT/pve-1 mounted no -
rpool/ROOT/pve-1 quota none default
rpool/ROOT/pve-1 reservation none default
rpool/ROOT/pve-1 recordsize 128K default
rpool/ROOT/pve-1 mountpoint / local
rpool/ROOT/pve-1 sharenfs off default
rpool/ROOT/pve-1 checksum on default
rpool/ROOT/pve-1 compression lz4 inherited from rpool
rpool/ROOT/pve-1 atime off inherited from rpool
rpool/ROOT/pve-1 devices on default
rpool/ROOT/pve-1 exec on default
rpool/ROOT/pve-1 setuid on default
rpool/ROOT/pve-1 readonly off default
rpool/ROOT/pve-1 zoned off default
rpool/ROOT/pve-1 snapdir hidden default
rpool/ROOT/pve-1 aclinherit restricted default
rpool/ROOT/pve-1 createtxg 10 -
rpool/ROOT/pve-1 canmount on default
rpool/ROOT/pve-1 xattr on default
rpool/ROOT/pve-1 copies 1 default
rpool/ROOT/pve-1 version 5 -
rpool/ROOT/pve-1 utf8only off -
rpool/ROOT/pve-1 normalization none -
rpool/ROOT/pve-1 casesensitivity sensitive -
rpool/ROOT/pve-1 vscan off default
rpool/ROOT/pve-1 nbmand off default
rpool/ROOT/pve-1 sharesmb off default
rpool/ROOT/pve-1 refquota none default
rpool/ROOT/pve-1 refreservation none default
rpool/ROOT/pve-1 guid 7448222599031128143 -
rpool/ROOT/pve-1 primarycache all default
rpool/ROOT/pve-1 secondarycache all default
rpool/ROOT/pve-1 usedbysnapshots 0B -
rpool/ROOT/pve-1 usedbydataset 2.05G -
rpool/ROOT/pve-1 usedbychildren 0B -
rpool/ROOT/pve-1 usedbyrefreservation 0B -
rpool/ROOT/pve-1 logbias latency default
rpool/ROOT/pve-1 dedup off default
rpool/ROOT/pve-1 mlslabel none default
rpool/ROOT/pve-1 sync standard inherited from rpool
rpool/ROOT/pve-1 dnodesize legacy default
rpool/ROOT/pve-1 refcompressratio 2.03x -
rpool/ROOT/pve-1 written 2.05G -
rpool/ROOT/pve-1 logicalused 3.86G -
rpool/ROOT/pve-1 logicalreferenced 3.86G -
rpool/ROOT/pve-1 volmode default default
rpool/ROOT/pve-1 filesystem_limit none default
rpool/ROOT/pve-1 snapshot_limit none default
rpool/ROOT/pve-1 filesystem_count none default
rpool/ROOT/pve-1 snapshot_count none default
rpool/ROOT/pve-1 snapdev hidden default
rpool/ROOT/pve-1 acltype off default
rpool/ROOT/pve-1 context none default
rpool/ROOT/pve-1 fscontext none default
rpool/ROOT/pve-1 defcontext none default
rpool/ROOT/pve-1 rootcontext none default
rpool/ROOT/pve-1 relatime off default
rpool/ROOT/pve-1 redundant_metadata all default
rpool/ROOT/pve-1 overlay off default

root@PartedMagic:~# zpool history
History for 'rpool':
2019-01-02.20:16:20 zpool create -f -o cachefile=none -o ashift=12 rpool raidz1 /dev/sda3 /dev/sdb3 /dev/sdc3 /dev/sdd3
2019-01-02.20:16:20 zfs create rpool/ROOT
2019-01-02.20:16:21 zfs create rpool/data
2019-01-02.20:16:21 zfs create rpool/ROOT/pve-1
2019-01-02.20:16:21 zfs set atime=off rpool
2019-01-02.20:16:21 zfs set compression=on rpool
2019-01-02.20:16:26 zfs set sync=disabled rpool
2019-01-02.20:29:04 zfs set sync=standard rpool
2019-01-02.20:29:04 zfs set mountpoint=/ rpool/ROOT/pve-1
2019-01-02.20:29:05 zpool set bootfs=rpool/ROOT/pve-1 rpool
2019-01-02.20:29:05 zpool export rpool
2019-01-02.20:47:25 zpool import -N rpool
2019-01-02.20:53:21 zpool import -N rpool
2019-01-04.11:47:09 zfs rename rpool/migrate-tmp/proxmox4/vms/sync-15m/vm-117-disk-1 rpool/migrate-tmp/proxmox4/vms/vm-117-disk-1
2019-01-04.11:47:20 zfs rename rpool/migrate-tmp/proxmox4/vms/sync-15m/vm-118-disk-1 rpool/migrate-tmp/proxmox4/vms/vm-118-disk-1
2019-01-04.11:47:32 zfs rename rpool/migrate-tmp/proxmox4/vms/sync-15m/vm-202-disk-1 rpool/migrate-tmp/proxmox4/vms/vm-202-disk-1
2019-01-04.11:47:40 zfs rename rpool/migrate-tmp/proxmox4/vms/sync-15m/vm-209-disk-1 rpool/migrate-tmp/proxmox4/vms/vm-209-disk-1
2019-01-04.11:48:33 zfs rename rpool/migrate-tmp/proxmox4/vms/sync-1d/vm-126-disk-1 rpool/migrate-tmp/proxmox4/vms/vm-126-disk-1
2019-01-04.11:48:38 zfs rename rpool/migrate-tmp/proxmox4/vms/sync-1d/vm-126-disk-2 rpool/migrate-tmp/proxmox4/vms/vm-126-disk-2
2019-01-04.11:48:53 zfs rename rpool/migrate-tmp/proxmox4/vms/sync-1d/vm-219-disk-1 rpool/migrate-tmp/proxmox4/vms/vm-219-disk-1
2019-01-04.11:48:59 zfs rename rpool/migrate-tmp/proxmox4/vms/sync-1d/vm-219-disk-2 rpool/migrate-tmp/proxmox4/vms/vm-219-disk-2
2019-01-04.11:49:11 zfs rename rpool/migrate-tmp/proxmox4/vms/sync-1d/vm-300-disk-1 rpool/migrate-tmp/proxmox4/vms/vm-300-disk-1
2019-01-04.11:49:33 zfs rename rpool/migrate-tmp/proxmox4/vms/sync-1h/vm-129-disk-1 rpool/migrate-tmp/proxmox4/vms/vm-129-disk-1
2019-01-04.11:49:57 zfs rename rpool/migrate-tmp/proxmox4/vms/sync-1h/vm-129-disk-2 rpool/migrate-tmp/proxmox4/vms/vm-129-disk-2
2019-01-04.11:50:16 zfs rename rpool/migrate-tmp/proxmox4/vms/sync-custom/vm-211-disk-1 rpool/migrate-tmp/proxmox4/vms/vm-211-disk-1
2019-01-04.19:41:31 zpool import -d /dev/disk/by-id/ -f -aN
2019-01-04.20:09:35 zpool export rpool
2019-01-04.20:09:59 zpool export rpool
2019-01-04.20:25:44 zpool export rpool
2019-01-04.20:25:59 zpool import rpool -R /mnt
2019-01-04.20:26:05 zpool export rpool
2019-01-04.20:52:22 zpool import -d /dev/disk/by-id/ -f -aN
2019-01-04.20:52:40 zpool export rpool
2019-01-04.20:53:01 zpool import rpool -R /mnt
2019-01-04.21:11:23 zfs set compression=lz4 rpool
2019-01-04.21:11:37 zpool export rpool
2019-01-04.21:52:24 zpool import -d /dev/disk/by-id/ -f -aN
2019-01-04.22:01:34 zpool export rpool
2019-01-04.22:01:55 zpool import rpool -R /mnt
2019-01-04.22:10:22 zpool export rpool
2019-01-04.22:11:16 zpool import rpool -R /mnt
2019-01-05.00:12:41 zpool import -d /dev/disk/by-id/ -f -aN
2019-01-05.17:23:46 zpool export rpool
2019-01-05.17:24:01 zpool import rpool -R /mnt
2019-01-05.17:32:09 zpool export rpool
2019-01-08.18:20:53 zpool import -d /dev/disk/by-id/ -f -aN
 
I removed the creates/rollbacks/etc from the send receives from "zfs history". They are just putting the VM data back in place.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back