[SOLVED] modprobe: ERROR: could not insert 'zfs': Invalid argument error after dist-upgrade

[Sennn]

Hi all,

I have run a dist upgrade today on the no-sub repository and have been unable to get ZFS running afterwards.

I am new to this forum and will be happy to provide any information that can help. Thank you in advance.

Kind regards,

-Sen

 

[tom]

Please post the output of:

> pveversion -v

 

[Sennn]

Hi Tom,

Thanks for the reply. Please see the requested information below:

root@pve:~# pveversion -v
proxmox-ve: 6.0-2 (running kernel: 5.0.21-5-pve)
pve-manager: 6.0-11 (running version: 6.0-11/2140ef37)
pve-kernel-helper: 6.0-12
pve-kernel-5.0: 6.0-11
pve-kernel-5.0.21-5-pve: 5.0.21-10
pve-kernel-5.0.21-3-pve: 5.0.21-7
pve-kernel-5.0.15-1-pve: 5.0.15-1
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.0.2-pve4
criu: 3.11-3
glusterfs-client: 5.5-3
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.13-pve1
libpve-access-control: 6.0-3
libpve-apiclient-perl: 3.0-2
libpve-common-perl: 6.0-7
libpve-guest-common-perl: 3.0-2
libpve-http-server-perl: 3.0-3
libpve-storage-perl: 6.0-9
libqb0: 1.0.5-1
lvm2: 2.03.02-pve3
lxc-pve: 3.2.1-1
lxcfs: 3.0.3-pve60
novnc-pve: 1.1.0-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.0-8
pve-cluster: 6.0-7
pve-container: 3.0-10
pve-docs: 6.0-8
pve-edk2-firmware: 2.20190614-1
pve-firewall: 4.0-7
pve-firmware: 3.0-4
pve-ha-manager: 3.0-3
pve-i18n: 2.0-3
pve-qemu-kvm: 4.0.1-5
pve-xtermjs: 3.13.2-1
qemu-server: 6.0-13
smartmontools: 7.0-pve2
spiceterm: 3.1-1
vncterm: 1.6-1
zfsutils-linux: 0.8.2-pve2

Cheers

 

[dietmar]

I have run a dist upgrade today on the no-sub repository and have been unable to get ZFS running afterwards.

What problem do you have exactly?

 

[Sennn]

In short, ZFS is not working.

In a little bit longer,

After upgrading and restarting I noticed that my storage was showing an error .

I tried running ZFS list to see if anything was working and was greeted by the message,
"The ZFS modules are not loaded.
Try running '/sbin/modprobe zfs' as root to load them."

Following the advice of my machine overlord, I ran "sudo /sbin/modprobe zfs" and the output was the text in the subject of this thread.

I have been unable to past these two statements. I have tried a number of different uninstall and reinstall paths and have been unable to progress any further.

Thanks again,

Sen

 

[dietmar]

So what happened to your zfs.ko module? Does modinfo show something?

# modinfo zfs

 

[Sennn]

I have not touched anything to do with my ZFS since setting it up and using Samba to share it around 2 months ago. No configuration changes have been made between it working and not.

Please also see the below modinfo in three parts:

I appreciate the assistance.

Kind regards,

Sen

 

[Sennn]

root@pve:~# modinfo zfs
filename: /lib/modules/5.0.21-5-pve/zfs/zfs.ko
version: 0.8.2-pve2
license: CDDL
author: OpenZFS on Linux
description: ZFS
alias: devname:zfs
alias: char-major-10-249
srcversion: 7974A38E326E18F22E88682
depends: spl,znvpair,icp,zlua,zunicode,zcommon,zavl
retpoline: Y
name: zfs
vermagic: 5.0.21-5-pve SMP mod_unload modversions
parm: zvol_inhibit_devo not create zvol device nodes (uint)
parm: zvol_major:Major number for zvol device (uint)
parm: zvol_threads:Max number of threads to handle I/O requests (uint)
parm: zvol_request_sync:Synchronously handle bio requests (uint)
parm: zvol_max_discard_blocks:Max number of blocks to discard (ulong)
parm: zvol_prefetch_bytesrefetch N bytes at zvol start+end (uint)
parm: zvol_volmodeefault volmode property value (uint)
parm: zfs_key_max_salt_uses:Max number of times a salt value can be used for generating encryption keys before it is rotated (ulong)
parm: zio_slow_io_ms:Max I/O completion time (milliseconds) before marking it as slow (int)
parm: zio_requeue_io_start_cut_in_linerioritize requeued I/O (int)
parm: zfs_sync_pass_deferred_freeefer frees starting in this pass (int)
parm: zfs_sync_pass_dont_compresson't compress starting in this pass (int)
parm: zfs_sync_pass_rewrite:Rewrite new bps starting in this pass (int)
parm: zio_dva_throttle_enabled:Throttle block allocations in the ZIO pipeline (int)
parm: zio_deadman_log_all:Log all slow ZIOs, not just those with vdevs (int)
parm: zfs_commit_timeout_pct:ZIL block open timeout percentage (int)
parm: zil_replay_disableisable intent logging replay (int)
parm: zil_nocacheflushisable ZIL cache flushes (int)
parm: zil_slog_bulk:Limit in bytes slog sync writes per commit (ulong)
parm: zfs_object_mutex_size:Size of znode hold array (uint)
parm: zfs_unlink_suspend_progress:Set to prevent async unlinks (debug - leaks space into the unlinked set) (int)
parm: zfs_delete_blockselete files larger than N blocks async (ulong)
parm: zfs_read_chunk_size:Bytes to read per chunk (ulong)
parm: zfs_immediate_write_sz:Largest data block to write to zil (long)
parm: zfs_dbgmsg_enable:Enable ZFS debug message log (int)
parm: zfs_dbgmsg_maxsize:Maximum ZFS debug log size (int)
parm: zfs_admin_snapshot:Enable mkdir/rmdir/mv in .zfs/snapshot (int)
parm: zfs_expire_snapshot:Seconds to expire .zfs/snapshot (int)
parm: zfs_lua_max_instrlimit:Max instruction limit that can be specified for a channel program (ulong)
parm: zfs_lua_max_memlimit:Max memory limit that can be specified for a channel program (ulong)
parm: zap_iterate_prefetch:When iterating ZAP object, prefetch it (int)
parm: zfs_trim_extent_bytes_max:Max size of TRIM commands, larger will be split (uint)
parm: zfs_trim_extent_bytes_min:Min size of TRIM commands, smaller will be skipped (uint)
parm: zfs_trim_metaslab_skip:Skip metaslabs which have never been initialized (uint)
parm: zfs_trim_txg_batch:Min number of txgs to aggregate frees before issuing TRIM (uint)
parm: zfs_trim_queue_limit:Max queued TRIMs outstanding per leaf vdev (uint)
parm: zfs_removal_ignore_errors:Ignore hard IO errors when removing device (int)
parm: zfs_remove_max_segment:Largest contiguous segment to allocate when removing device (int)
parm: vdev_removal_max_span:Largest span of free chunks a remap segment can span (int)
parm: zfs_removal_suspend_progressause device removal after this many bytes are copied (debug use only - causes removal to hang) (int)
parm: zfs_vdev_raidz_impl:Select raidz implementation.
parm: zfs_vdev_aggregation_limit:Max vdev I/O aggregation size (int)
parm: zfs_vdev_aggregation_limit_non_rotating:Max vdev I/O aggregation size for non-rotating media (int)
parm: zfs_vdev_aggregate_trim:Allow TRIM I/O to be aggregated (int)
parm: zfs_vdev_read_gap_limit:Aggregate read I/O over gap (int)
parm: zfs_vdev_write_gap_limit:Aggregate write I/O over gap (int)
parm: zfs_vdev_max_active:Maximum number of active I/Os per vdev (int)
parm: zfs_vdev_async_write_active_max_dirty_percent:Async write concurrency max threshold (int)
parm: zfs_vdev_async_write_active_min_dirty_percent:Async write concurrency min threshold (int)
parm: zfs_vdev_async_read_max_active:Max active async read I/Os per vdev (int)
parm: zfs_vdev_async_read_min_active:Min active async read I/Os per vdev (int)
parm: zfs_vdev_async_write_max_active:Max active async write I/Os per vdev (int)
parm: zfs_vdev_async_write_min_active:Min active async write I/Os per vdev (int)
parm: zfs_vdev_initializing_max_active:Max active initializing I/Os per vdev (int)
parm: zfs_vdev_initializing_min_active:Min active initializing I/Os per vdev (int)
parm: zfs_vdev_removal_max_active:Max active removal I/Os per vdev (int)
parm: zfs_vdev_removal_min_active:Min active removal I/Os per vdev (int)
parm: zfs_vdev_scrub_max_active:Max active scrub I/Os per vdev (int)
parm: zfs_vdev_scrub_min_active:Min active scrub I/Os per vdev (int)
parm: zfs_vdev_sync_read_max_active:Max active sync read I/Os per vdev (int)
parm: zfs_vdev_sync_read_min_active:Min active sync read I/Os per vdev (int)
parm: zfs_vdev_sync_write_max_active:Max active sync write I/Os per vdev (int)
parm: zfs_vdev_sync_write_min_active:Min active sync write I/Os per vdev (int)
parm: zfs_vdev_trim_max_active:Max active trim/discard I/Os per vdev (int)
parm: zfs_vdev_trim_min_active:Min active trim/discard I/Os per vdev (int)
parm: zfs_vdev_queue_depth_pct:Queue depth percentage for each top-level vdev (int)
parm: zfs_vdev_mirror_rotating_inc:Rotating media load increment for non-seeking I/O's (int)
parm: zfs_vdev_mirror_rotating_seek_inc:Rotating media load increment for seeking I/O's (int)
parm: zfs_vdev_mirror_rotating_seek_offset:Offset in bytes from the last I/O which triggers a reduced rotating media seek increment (int)

 

[Sennn]

parm: zfs_vdev_mirror_non_rotating_inc:Non-rotating media load increment for non-seeking I/O's (int)
parm: zfs_vdev_mirror_non_rotating_seek_inc:Non-rotating media load increment for seeking I/O's (int)
parm: zfs_initialize_value:Value written during zpool initialize (ulong)
parm: zfs_condense_indirect_vdevs_enable:Whether to attempt condensing indirect vdev mappings (int)
parm: zfs_condense_min_mapping_bytes:Minimum size of vdev mapping to condense (ulong)
parm: zfs_condense_max_obsolete_bytes:Minimum size obsolete spacemap to attempt condensing (ulong)
parm: zfs_condense_indirect_commit_entry_delay_mselay while condensing vdev mapping (int)
parm: zfs_reconstruct_indirect_combinations_max:Maximum number of combinations when reconstructing split segments (int)
parm: zfs_vdev_scheduler:I/O scheduler
parm: zfs_vdev_cache_max:Inflate reads small than max (int)
parm: zfs_vdev_cache_size:Total size of the per-disk cache (int)
parm: zfs_vdev_cache_bshift:Shift size to inflate reads too (int)
parm: zfs_vdev_default_ms_count:Target number of metaslabs per top-level vdev (int)
parm: zfs_vdev_min_ms_count:Minimum number of metaslabs per top-level vdev (int)
parm: zfs_vdev_ms_count_limitractical upper limit of total metaslabs per top-level vdev (int)
parm: zfs_slow_io_events_per_second:Rate limit slow IO (delay) events to this many per second (uint)
parm: zfs_checksum_events_per_second:Rate limit checksum events to this many checksum errors per second (do not set below zedthreshold). (uint)
parm: zfs_scan_ignore_errors:Ignore errors during resilver/scrub (int)
parm: vdev_validate_skip:Bypass vdev_validate() (int)
parm: zfs_nocacheflushisable cache flushes (int)
parm: zfs_txg_timeout:Max seconds worth of delta per txg (int)
parm: zfs_read_history:Historical statistics for the last N reads (int)
parm: zfs_read_history_hits:Include cache hits in read history (int)
parm: zfs_txg_history:Historical statistics for the last N txgs (int)
parm: zfs_multihost_history:Historical statistics for last N multihost writes (int)
parm: zfs_flags:Set additional debugging flags (uint)
parm: zfs_recover:Set to attempt to recover from fatal errors (int)
parm: zfs_free_leak_on_eio:Set to ignore IO errors during free and permanently leak the space (int)
parm: zfs_deadman_synctime_msool sync expiration time in milliseconds
parm: zfs_deadman_ziotime_ms:IO expiration time in milliseconds
parm: zfs_deadman_checktime_msead I/O check interval in milliseconds (ulong)
parm: zfs_deadman_enabled:Enable deadman timer (int)
parm: zfs_deadman_failmode:Failmode for deadman timer
parm: spa_asize_inflation:SPA size estimate multiplication factor (int)
parm: spa_slop_shift:Reserved free space in pool
parm: zfs_ddt_data_is_speciallace DDT data into the special class (int)
parm: zfs_user_indirect_is_speciallace user data indirect blocks into the special class (int)
parm: zfs_special_class_metadata_reserve_pct:Small file blocks in special vdevs depends on this much free space available (int)
parm: spa_config_path:SPA config file (/etc/zfs/zpool.cache) (charp)
parm: zfs_autoimport_disableisable pool import at module load (int)
parm: zfs_spa_discard_memory_limit:Maximum memory for prefetching checkpoint space map per top-level vdev while discarding checkpoint (ulong)
parm: spa_load_verify_maxinflight:Max concurrent traversal I/Os while verifying pool during import -X (int)
parm: spa_load_verify_metadata:Set to traverse metadata on pool import (int)
parm: spa_load_verify_data:Set to traverse data on pool import (int)
parm: spa_load_print_vdev_treerint vdev tree to zfs_dbgmsg during pool import (int)
parm: zio_taskq_batch_pctercentage of CPUs to run an IO worker thread (uint)
parm: zfs_max_missing_tvds:Allow importing pool with up to this number of missing top-level vdevs (in read-only mode) (ulong)
parm: zfs_multilist_num_sublists:Number of sublists used in each multilist (int)
parm: zfs_multihost_fail_intervals:Max allowed period without a successful mmp write (uint)
parm: zfs_multihost_interval:Milliseconds between mmp writes to each leaf
parm: zfs_multihost_import_intervals:Number of zfs_multihost_interval periods to wait for activity (uint)
parm: metaslab_aliquot:allocation granularity (a.k.a. stripe size) (ulong)
parm: metaslab_debug_load:load all metaslabs when pool is first opened (int)
parm: metaslab_debug_unloadrevent metaslabs from being unloaded (int)
parm: metaslab_preload_enabledreload potential metaslabs during reassessment (int)
parm: zfs_mg_noalloc_thresholdercentage of free space for metaslab group to allow allocation (int)
parm: zfs_mg_fragmentation_threshold:fragmentation for metaslab group to allow allocation (int)
parm: zfs_metaslab_fragmentation_threshold:fragmentation for metaslab to allow allocation (int)
parm: metaslab_fragmentation_factor_enabled:use the fragmentation metric to prefer less fragmented metaslabs (int)
parm: metaslab_lba_weighting_enabledrefer metaslabs with lower LBAs (int)
parm: metaslab_bias_enabled:enable metaslab group biasing (int)
parm: zfs_metaslab_segment_weight_enabled:enable segment-based metaslab selection (int)
parm: zfs_metaslab_switch_threshold:segment-based metaslab selection maximum buckets before switching (int)
parm: metaslab_force_ganging:blocks larger than this size are forced to be gang blocks (ulong)
parm: zfs_zevent_len_max:Max event queue length (int)
parm: zfs_zevent_cols:Max event column width (int)
parm: zfs_zevent_console:Log events to the console (int)
parm: zfs_scan_vdev_limit:Max bytes in flight per leaf vdev for scrubs and resilvers (ulong)
parm: zfs_scrub_min_time_ms:Min millisecs to scrub per txg (int)
parm: zfs_obsolete_min_time_ms:Min millisecs to obsolete per txg (int)
parm: zfs_free_min_time_ms:Min millisecs to free per txg (int)
parm: zfs_resilver_min_time_ms:Min millisecs to resilver per txg (int)
parm: zfs_scan_suspend_progress:Set to prevent scans from progressing (int)
parm: zfs_no_scrub_io:Set to disable scrub I/O (int)
parm: zfs_no_scrub_prefetch:Set to disable scrub prefetching (int)
parm: zfs_async_block_max_blocks:Max number of blocks freed in one txg (ulong)
parm: zfs_free_bpobj_enabled:Enable processing of the free_bpobj (int)
parm: zfs_scan_mem_lim_fact:Fraction of RAM for scan hard limit (int)
parm: zfs_scan_issue_strategy:IO issuing strategy during scrubbing. 0 = default, 1 = LBA, 2 = size (int)
parm: zfs_scan_legacy:Scrub using legacy non-sequential method (int)
parm: zfs_scan_checkpoint_intval:Scan progress on-disk checkpointing interval (int)
parm: zfs_scan_max_ext_gap:Max gap in bytes between sequential scrub / resilver I/Os (ulong)
parm: zfs_scan_mem_lim_soft_fact:Fraction of hard limit used as soft limit (int)
parm: zfs_scan_strict_mem_lim:Tunable to attempt to reduce lock contention (int)
parm: zfs_scan_fill_weight:Tunable to adjust bias towards more filled segments during scans (int)
parm: zfs_resilver_disable_deferrocess all resilvers immediately (int)
parm: zfs_dirty_data_max_percentercent of ram can be dirty (int)
parm: zfs_dirty_data_max_max_percent:zfs_dirty_data_max upper bound as % of RAM (int)
parm: zfs_delay_min_dirty_percent:transaction delay threshold (int)
parm: zfs_dirty_data_max:determines the dirty space limit (ulong)
parm: zfs_dirty_data_max_max:zfs_dirty_data_max upper bound in bytes (ulong)
parm: zfs_dirty_data_sync_percent:dirty data txg sync threshold as a percentage of zfs_dirty_data_max (int)
parm: zfs_delay_scale:how quickly delay approaches infinity (ulong)
parm: zfs_sync_taskq_batch_pct:max percent of CPUs that are used to sync dirty data (int)
parm: zfs_zil_clean_taskq_nthr_pct:max percent of CPUs that are used per dp_sync_taskq (int)
parm: zfs_zil_clean_taskq_minalloc:number of taskq entries that are pre-populated (int)
parm: zfs_zil_clean_taskq_maxalloc:max number of taskq entries that are cached (int)
parm: zfs_disable_ivset_guid_check:Set to allow raw receives without IVset guids (int)
parm: zfs_max_recordsize:Max allowed record size (int)
parm: zfs_prefetch_disableisable all ZFS prefetching (int)
parm: zfetch_max_streams:Max number of streams per zfetch (uint)
parm: zfetch_min_sec_reap:Min time before stream reclaim (uint)
parm: zfetch_max_distance:Max bytes to prefetch per stream (default 8MB) (uint)
parm: zfetch_array_rd_sz:Number of bytes in a array_read (ulong)
parm: zfs_pd_bytes_max:Max number of bytes to prefetch (int)
parm: ignore_hole_birth:Alias for send_holes_without_birth_time (int)
parm: send_holes_without_birth_time:Ignore hole_birth txg for zfs send (int)
parm: zfs_override_estimate_recordsize:Record size calculation override for zfs send estimates (ulong)
parm: zfs_send_corrupt_data:Allow sending corrupt data (int)
parm: zfs_send_queue_length:Maximum send queue length (int)
parm: zfs_send_unmodified_spill_blocks:Send unmodified spill blocks (int)

 

[Sennn]

parm: zfs_recv_queue_length:Maximum receive queue length (int)
parm: dmu_object_alloc_chunk_shift:CPU-specific allocator grabs 2^N objects at once (int)
parm: zfs_nopwrite_enabled:Enable NOP writes (int)
parm: zfs_per_txg_dirty_frees_percentercentage of dirtied blocks from frees in one TXG (ulong)
parm: zfs_dmu_offset_next_sync:Enable forcing txg sync to find holes (int)
parm: dmu_prefetch_max:Limit one prefetch call to this size (int)
parm: zfs_dedup_prefetch:Enable prefetching dedup-ed blks (int)
parm: zfs_dbuf_state_index:Calculate arc header index (int)
parm: dbuf_cache_max_bytes:Maximum size in bytes of the dbuf cache. (ulong)
parm: dbuf_cache_hiwater_pctercentage over dbuf_cache_max_bytes when dbufs must be evicted directly. (uint)
parm: dbuf_cache_lowater_pctercentage below dbuf_cache_max_bytes when the evict thread stops evicting dbufs. (uint)
parm: dbuf_metadata_cache_max_bytes:Maximum size in bytes of the dbuf metadata cache. (ulong)
parm: dbuf_metadata_cache_shift:int
parm: dbuf_cache_shift:Set the size of the dbuf cache to a log2 fraction of arc size. (int)
parm: zfs_arc_min:Min arc size (ulong)
parm: zfs_arc_max:Max arc size (ulong)
parm: zfs_arc_meta_limit:Meta limit for arc size (ulong)
parm: zfs_arc_meta_limit_percentercent of arc size for arc meta limit (ulong)
parm: zfs_arc_meta_min:Min arc metadata (ulong)
parm: zfs_arc_meta_prune:Meta objects to scan for prune (int)
parm: zfs_arc_meta_adjust_restarts:Limit number of restarts in arc_adjust_meta (int)
parm: zfs_arc_meta_strategy:Meta reclaim strategy (int)
parm: zfs_arc_grow_retry:Seconds before growing arc size (int)
parm: zfs_arc_p_dampener_disable:disable arc_p adapt dampener (int)
parm: zfs_arc_shrink_shift:log2(fraction of arc to reclaim) (int)
parm: zfs_arc_pc_percentercent of pagecache to reclaim arc to (uint)
parm: zfs_arc_p_min_shift:arc_c shift to calc min/max arc_p (int)
parm: zfs_arc_average_blocksize:Target average block size (int)
parm: zfs_compressed_arc_enabledisable compressed arc buffers (int)
parm: zfs_arc_min_prefetch_ms:Min life of prefetch block in ms (int)
parm: zfs_arc_min_prescient_prefetch_ms:Min life of prescient prefetched block in ms (int)
parm: l2arc_write_max:Max write bytes per interval (ulong)
parm: l2arc_write_boost:Extra write bytes during device warmup (ulong)
parm: l2arc_headroom:Number of max device writes to precache (ulong)
parm: l2arc_headroom_boost:Compressed l2arc_headroom multiplier (ulong)
parm: l2arc_feed_secs:Seconds between L2ARC writing (ulong)
parm: l2arc_feed_min_ms:Min feed interval in milliseconds (ulong)
parm: l2arc_noprefetch:Skip caching prefetched buffers (int)
parm: l2arc_feed_again:Turbo L2ARC warmup (int)
parm: l2arc_norw:No reads during writes (int)
parm: zfs_arc_lotsfree_percent:System free memory I/O throttle in bytes (int)
parm: zfs_arc_sys_free:System free memory target size in bytes (ulong)
parm: zfs_arc_dnode_limit:Minimum bytes of dnodes in arc (ulong)
parm: zfs_arc_dnode_limit_percentercent of ARC meta buffers for dnodes (ulong)
parm: zfs_arc_dnode_reduce_percentercentage of excess dnodes to try to unpin (ulong)
parm: zfs_abd_scatter_enabled:Toggle whether ABD allocations must be linear. (int)
parm: zfs_abd_scatter_min_size:Minimum size of scatter allocations. (int)
parm: zfs_abd_scatter_max_order:Maximum order allocation used for a scatter ABD. (uint)

 

[dietmar]

OK so far. So modprobe should also work:

# modprobe zfs

If not, maybe you manually set some parameters for zfs in /etc/modules or /etc/modprobe.d/?