Ok guys I have an update, it is good news.
First of all some output to show the configuration been tested.
Code:
# proxmox-boot-tool status
Re-executing '/usr/sbin/proxmox-boot-tool' in new private mount namespace..
System currently booted with legacy bios
WARN: /dev/disk/by-uuid/7C8B-41CC does not exist - clean '/etc/kernel/proxmox-boot-uuids'! - skipping
WARN: /dev/disk/by-uuid/7C8B-9B78 does not exist - clean '/etc/kernel/proxmox-boot-uuids'! - skipping
This shows its been booted by grub (I also see grub on local display), and the 2 missing uuid's are the EFI partitions I deleted to mimic the server I have concerns with.
Code:
# ls /sys/firmware/efi
ls: cannot access '/sys/firmware/efi': No such file or directory
Further confirmation not an efi boot.
I initially just did a basic 6.2 to 6.4 update to have it boot from openzfs 2.0 on a system that was installed on zfs on linux 0.8x, and the system booted albeit with disabled features.
Code:
# dpkg -l | grep zfs
ii libzfs4linux 2.0.5-pve1~bpo10+1 amd64 OpenZFS filesystem library for Linux
ii zfs-initramfs 2.0.5-pve1~bpo10+1 all OpenZFS root filesystem capabilities for Linux - initramfs
ii zfs-zed 2.0.5-pve1~bpo10+1 amd64 OpenZFS Event Daemon
ii zfsutils-linux 2.0.5-pve1~bpo10+1 amd64 command-line tools to manage OpenZFS filesystems
Code:
# zpool status
pool: rpool
state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(5) for details.
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
ata-SAMSUNG_SSD_830_Series_S0VYNEAC704291-part3 ONLINE 0 0 0
ata-SAMSUNG_SSD_830_Series_S0WJNEAC705087-part3 ONLINE 0 0 0
Code:
Some supported features are not enabled on the following pools. Once a
feature is enabled the pool may become incompatible with software
that does not support the feature. See zpool-features(5) for details.
POOL FEATURE
---------------
rpool
redaction_bookmarks
redacted_datasets
bookmark_written
log_spacemap
livelist
device_rebuild
zstd_compress
I then ran zpool upgrade to update it, with all new features set to enabled.
Code:
# zpool upgrade rpool
This system supports ZFS pool feature flags.
Enabled the following features on 'rpool':
redaction_bookmarks
redacted_datasets
bookmark_written
log_spacemap
livelist
device_rebuild
zstd_compress
confirmed with zpool get all rpool and zpool status
Code:
# zpool get all
NAME PROPERTY VALUE SOURCE
rpool size 19G -
rpool capacity 8% -
rpool altroot - default
rpool health ONLINE -
rpool guid 13857213527387179236 -
rpool version - default
rpool bootfs rpool/ROOT/pve-1 local
rpool delegation on default
rpool autoreplace off default
rpool cachefile - default
rpool failmode wait default
rpool listsnapshots off default
rpool autoexpand off default
rpool dedupratio 1.00x -
rpool free 17.5G -
rpool allocated 1.54G -
rpool readonly off -
rpool ashift 12 local
rpool comment - default
rpool expandsize - -
rpool freeing 0 -
rpool fragmentation 2% -
rpool leaked 0 -
rpool multihost off default
rpool checkpoint - -
rpool load_guid 5200106711538973796 -
rpool autotrim off default
rpool feature@async_destroy enabled local
rpool feature@empty_bpobj active local
rpool feature@lz4_compress active local
rpool feature@multi_vdev_crash_dump enabled local
rpool feature@spacemap_histogram active local
rpool feature@enabled_txg active local
rpool feature@hole_birth active local
rpool feature@extensible_dataset active local
rpool feature@embedded_data active local
rpool feature@bookmarks enabled local
rpool feature@filesystem_limits enabled local
rpool feature@large_blocks enabled local
rpool feature@large_dnode enabled local
rpool feature@sha512 enabled local
rpool feature@skein enabled local
rpool feature@edonr enabled local
rpool feature@userobj_accounting active local
rpool feature@encryption enabled local
rpool feature@project_quota active local
rpool feature@device_removal enabled local
rpool feature@obsolete_counts enabled local
rpool feature@zpool_checkpoint enabled local
rpool feature@spacemap_v2 active local
rpool feature@allocation_classes enabled local
rpool feature@resilver_defer enabled local
rpool feature@bookmark_v2 enabled local
rpool feature@redaction_bookmarks enabled local
rpool feature@redacted_datasets enabled local
rpool feature@bookmark_written enabled local
rpool feature@log_spacemap active local
rpool feature@livelist enabled local
rpool feature@device_rebuild enabled local
rpool feature@zstd_compress enabled local
root@AMD:~# zpool status
pool: rpool
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
ata-SAMSUNG_SSD_830_Series_S0VYNEAC704291-part3 ONLINE 0 0 0
ata-SAMSUNG_SSD_830_Series_S0WJNEAC705087-part3 ONLINE 0 0 0
errors: No known data errors
.
rebooted again and absolutely no problems.
So the issue is when the flags are active rather than enabled, we already know zstd_compress is one of those flags, not sure about the other new features.
So it seems the issue is nowhere near as bad as has been indicated. As long as you dont blindly start using the new features without converting to the new proxmox boot tool first, everything should be fine, especially making sure to not start using zstd on the root zfs.
Creating a new pool will of course use zstd as its default, but that would be ok as the pool isnt required to be loaded by grub, creating a new dataset or volume would inherit the property from the rpool root. This is the unknown as to whether they inherit "on" with lz4 been the default or "on" with zstd been the default, to play safe I would specifically set the rpool to lz4 to ensure any child datasets and volumes will use lz4.
It is sad to see that by default proxmox still doesnt put grub on both vdev mirror's meaning if disk 1 fails I dont think proxmox would boot, I might file a bug report on that. However if proxmox by default on 6.4 and 7+ now uses proxmox-boot moving forwards then this lack of grub redundancy issue might be obsoleted.
