[solved] Prevent ProxMox from mounting non existing pool

E

Elleni

Well-Known Member
Jul 6, 2020
234
20
58
52
I had tried to change from a luks_encrypted raid1 data partition to a zfs data pool, but I reverted back, and now my hosted server gets kernel panic because aparently it tries to mount a pool that does not exist anymore. Where can I delete the obsolete entries to re-establish successfull boot of the server? It is setup on ext4 so I could even try to disable zfs temporary to be able to boot the server successfully and regain ssh access. Thanks in advance for any help.

In /var/log/messges I see:

Oct 24 18:36:12 hrs kernel: [2052656.820053] WARNING: Pool 'hrs_daten' has encountered an uncorrectable I/O failure and has been suspended.
Oct 24 18:36:12 hrs kernel: [2052656.820053]
Oct 24 18:39:00 hrs kernel: [2052824.839607] zed D 0 29631 1 0x00000000
Oct 24 18:39:00 hrs kernel: [2052824.839609] Call Trace:
Oct 24 18:39:00 hrs kernel: [2052824.839615] __schedule+0x2e6/0x6f0
Oct 24 18:39:00 hrs kernel: [2052824.839616] ? _cond_resched+0x19/0x30
Oct 24 18:39:00 hrs kernel: [2052824.839619] schedule+0x33/0xa0
Oct 24 18:39:00 hrs kernel: [2052824.839620] schedule_preempt_disabled+0xe/0x10
Oct 24 18:39:00 hrs kernel: [2052824.839621] __mutex_lock.isra.10+0x2c9/0x4c0
Oct 24 18:39:00 hrs kernel: [2052824.839623] __mutex_lock_slowpath+0x13/0x20
Oct 24 18:39:00 hrs kernel: [2052824.839624] mutex_lock+0x2c/0x30
Oct 24 18:39:00 hrs kernel: [2052824.839659] spa_all_configs+0x3b/0x120 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.839691] zfs_ioc_pool_configs+0x1b/0x70 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.839723] zfsdev_ioctl+0x6db/0x8f0 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.839725] ? do_futex+0xc7/0xc50
Oct 24 18:39:00 hrs kernel: [2052824.839727] do_vfs_ioctl+0xa9/0x640
Oct 24 18:39:00 hrs kernel: [2052824.839728] ksys_ioctl+0x67/0x90
Oct 24 18:39:00 hrs kernel: [2052824.839729] __x64_sys_ioctl+0x1a/0x20
Oct 24 18:39:00 hrs kernel: [2052824.839731] do_syscall_64+0x57/0x190
Oct 24 18:39:00 hrs kernel: [2052824.839732] entry_SYSCALL_64_after_hwframe+0x44/0xa9
Oct 24 18:39:00 hrs kernel: [2052824.839734] RIP: 0033:0x7f8d63264427
Oct 24 18:39:00 hrs kernel: [2052824.839735] Code: 00 00 00 50 25 41 2b 36 30 f2 46 00 00 00 00 48 49 47 48 45 53 54 4d 4f 44 53 45 51 00 00 00 63 68 65 00 c8 f4 c3 49 87 7f 00 <00> 00 9d 31 56 87 7f 00 00 0f 00 00 00 00 00 00 00 e1 68 fe ac 9d
Oct 24 18:39:00 hrs kernel: [2052824.839736] RSP: 002b:00007f8d62403618 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
Oct 24 18:39:00 hrs kernel: [2052824.839737] RAX: ffffffffffffffda RBX: 0000559cf2e71aa0 RCX: 00007f8d63264427
Oct 24 18:39:00 hrs kernel: [2052824.839738] RDX: 00007f8d62403650 RSI: 0000000000005a04 RDI: 000000000000000b
Oct 24 18:39:00 hrs kernel: [2052824.839738] RBP: 00007f8d62406c30 R08: 00007f8d54040bb0 R09: 00007f8d540000d0
Oct 24 18:39:00 hrs kernel: [2052824.839739] R10: 00007f8d540008d0 R11: 0000000000000246 R12: 0000000000000000
Oct 24 18:39:00 hrs kernel: [2052824.839739] R13: 00007f8d62403650 R14: 0000559cf0f278b0 R15: 00007f8d62406d00
Oct 24 18:39:00 hrs kernel: [2052824.839934] txg_sync D 0 3375 2 0x80004000
Oct 24 18:39:00 hrs kernel: [2052824.839935] Call Trace:
Oct 24 18:39:00 hrs kernel: [2052824.839936] __schedule+0x2e6/0x6f0
Oct 24 18:39:00 hrs kernel: [2052824.839938] schedule+0x33/0xa0
Oct 24 18:39:00 hrs kernel: [2052824.839938] schedule_timeout+0x152/0x330
Oct 24 18:39:00 hrs kernel: [2052824.839944] ? taskq_member+0x18/0x30 [spl]
Oct 24 18:39:00 hrs kernel: [2052824.839945] ? __next_timer_interrupt+0xd0/0xd0
Oct 24 18:39:00 hrs kernel: [2052824.839947] io_schedule_timeout+0x1e/0x50
Oct 24 18:39:00 hrs kernel: [2052824.839949] __cv_timedwait_common+0x12f/0x170 [spl]
Oct 24 18:39:00 hrs kernel: [2052824.839951] ? wait_woken+0x80/0x80
Oct 24 18:39:00 hrs kernel: [2052824.839954] __cv_timedwait_io+0x19/0x20 [spl]
Oct 24 18:39:00 hrs kernel: [2052824.839984] zio_wait+0x130/0x270 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.840006] dbuf_read+0x4e1/0xb80 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.840028] dmu_buf_will_dirty_impl+0x10f/0x130 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.840050] dmu_buf_will_dirty+0x16/0x20 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.840072] dmu_write_impl+0x42/0xd0 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.840095] dmu_write.part.10+0x9e/0x100 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.840118] dmu_write+0x14/0x20 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.840149] spa_history_write+0x18b/0x1d0 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.840180] spa_history_log_sync+0x1a8/0x7a0 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.840183] ? spl_kmem_cache_free+0x10f/0x1f0 [spl]
Oct 24 18:39:00 hrs kernel: [2052824.840212] dsl_sync_task_sync+0xa9/0xf0 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.840240] dsl_pool_sync+0x3df/0x500 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.840271] spa_sync+0x5a7/0xfa0 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.840272] ? _cond_resched+0x19/0x30
Oct 24 18:39:00 hrs kernel: [2052824.840303] ? spa_txg_history_init_io+0x104/0x110 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.840335] txg_sync_thread+0x2d9/0x4c0 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.840375] ? txg_thread_exit.isra.12+0x60/0x60 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.840378] thread_generic_wrapper+0x74/0x90 [spl]
Oct 24 18:39:00 hrs kernel: [2052824.840380] kthread+0x120/0x140
Oct 24 18:39:00 hrs kernel: [2052824.840384] ? __thread_exit+0x20/0x20 [spl]
Oct 24 18:39:00 hrs kernel: [2052824.840385] ? kthread_park+0x90/0x90
Oct 24 18:39:00 hrs kernel: [2052824.840386] ret_from_fork+0x22/0x40
Oct 24 18:39:00 hrs kernel: [2052824.840587] zpool D 0 11859 1 0x00004004
Oct 24 18:39:00 hrs kernel: [2052824.840588] Call Trace:
Oct 24 18:39:00 hrs kernel: [2052824.840589] __schedule+0x2e6/0x6f0
Oct 24 18:39:00 hrs kernel: [2052824.840591] schedule+0x33/0xa0
Oct 24 18:39:00 hrs kernel: [2052824.840592] io_schedule+0x16/0x40
Oct 24 18:39:00 hrs kernel: [2052824.840595] cv_wait_common+0xb5/0x130 [spl]
Oct 24 18:39:00 hrs kernel: [2052824.840596] ? wait_woken+0x80/0x80
Oct 24 18:39:00 hrs kernel: [2052824.840598] __cv_wait_io+0x18/0x20 [spl]
Oct 24 18:39:00 hrs kernel: [2052824.840631] txg_wait_synced_impl+0xc9/0x110 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.840664] txg_wait_synced+0x10/0x40 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.840697] spa_export_common+0x4c1/0x580 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.840729] spa_destroy+0x1a/0x20 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.840762] zfs_ioc_pool_destroy+0x1a/0x20 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.840794] zfsdev_ioctl+0x6db/0x8f0 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.840796] ? lru_cache_add_active_or_unevictable+0x39/0xb0
Oct 24 18:39:00 hrs kernel: [2052824.840798] do_vfs_ioctl+0xa9/0x640
Oct 24 18:39:00 hrs kernel: [2052824.840800] ? handle_mm_fault+0xc9/0x1f0
Oct 24 18:39:00 hrs kernel: [2052824.840801] ksys_ioctl+0x67/0x90
Oct 24 18:39:00 hrs kernel: [2052824.840802] __x64_sys_ioctl+0x1a/0x20
Oct 24 18:39:00 hrs kernel: [2052824.840803] do_syscall_64+0x57/0x190
Oct 24 18:39:00 hrs kernel: [2052824.840804] entry_SYSCALL_64_after_hwframe+0x44/0xa9
Oct 24 18:39:00 hrs kernel: [2052824.840805] RIP: 0033:0x7f0119d04427
Oct 24 18:39:00 hrs kernel: [2052824.840808] Code: Bad RIP value.
Oct 24 18:39:00 hrs kernel: [2052824.840808] RSP: 002b:00007ffefa917a18 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
Oct 24 18:39:00 hrs kernel: [2052824.840809] RAX: ffffffffffffffda RBX: 000055756dbfbb90 RCX: 00007f0119d04427
Oct 24 18:39:00 hrs kernel: [2052824.840809] RDX: 00007ffefa917a20 RSI: 0000000000005a01 RDI: 0000000000000003
Oct 24 18:39:00 hrs kernel: [2052824.840810] RBP: 00007ffefa91b410 R08: 0000000000000040 R09: 00007f0119dcfca0
Oct 24 18:39:00 hrs kernel: [2052824.840810] R10: 0000000000000040 R11: 0000000000000246 R12: 00007ffefa917a20
Oct 24 18:39:00 hrs kernel: [2052824.840811] R13: 000055756dbf6430 R14: 000055756dbf83e0 R15: 000055756cc34a00
Oct 24 18:39:00 hrs kernel: [2052824.840996] zpool D 0 11909 1046 0x00000004
Oct 24 18:39:00 hrs kernel: [2052824.840997] Call Trace:
Oct 24 18:39:00 hrs kernel: [2052824.840998] __schedule+0x2e6/0x6f0
Oct 24 18:39:00 hrs kernel: [2052824.841000] schedule+0x33/0xa0
Oct 24 18:39:00 hrs kernel: [2052824.841001] schedule_preempt_disabled+0xe/0x10
Oct 24 18:39:00 hrs kernel: [2052824.841002] __mutex_lock.isra.10+0x2c9/0x4c0
Oct 24 18:39:00 hrs kernel: [2052824.841003] __mutex_lock_slowpath+0x13/0x20
Oct 24 18:39:00 hrs kernel: [2052824.841004] mutex_lock+0x2c/0x30
Oct 24 18:39:00 hrs kernel: [2052824.841036] spa_open_common+0x62/0x4d0 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.841069] spa_get_stats+0x57/0x570 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.841070] ? kmalloc_large_node+0x86/0x90
Oct 24 18:39:00 hrs kernel: [2052824.841071] ? __kmalloc_node+0x262/0x330
Oct 24 18:39:00 hrs kernel: [2052824.841104] zfs_ioc_pool_stats+0x39/0x90 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.841137] zfsdev_ioctl+0x6db/0x8f0 [zfs]
Oct 24 18:39:00 hrs kernel: [2052824.841138] ? lru_cache_add_active_or_unevictable+0x39/0xb0
Oct 24 18:39:00 hrs kernel: [2052824.841139] do_vfs_ioctl+0xa9/0x640
Oct 24 18:39:00 hrs kernel: [2052824.841141] ? handle_mm_fault+0xc9/0x1f0
Oct 24 18:39:00 hrs kernel: [2052824.841141] ksys_ioctl+0x67/0x90
Oct 24 18:39:00 hrs kernel: [2052824.841142] __x64_sys_ioctl+0x1a/0x20
Oct 24 18:39:00 hrs kernel: [2052824.841144] do_syscall_64+0x57/0x190
Oct 24 18:39:00 hrs kernel: [2052824.841145] entry_SYSCALL_64_after_hwframe+0x44/0xa9
Oct 24 18:39:00 hrs kernel: [2052824.841145] RIP: 0033:0x7fe424cd7427
Oct 24 18:39:00 hrs kernel: [2052824.841147] Code: Bad RIP value.
Oct 24 18:39:00 hrs kernel: [2052824.841147] RSP: 002b:00007ffd2984a4f8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
Oct 24 18:39:00 hrs kernel: [2052824.841148] RAX: ffffffffffffffda RBX: 00007ffd2984a520 RCX: 00007fe424cd7427
Oct 24 18:39:00 hrs kernel: [2052824.841148] RDX: 00007ffd2984a520 RSI: 0000000000005a05 RDI: 0000000000000003
Oct 24 18:39:00 hrs kernel: [2052824.841149] RBP: 00007ffd2984db00 R08: 000055680fbd5680 R09: 00007fe424da2de0
Oct 24 18:39:00 hrs kernel: [2052824.841149] R10: 000055680fbd3010 R11: 0000000000000246 R12: 000055680fbd5530
Oct 24 18:39:00 hrs kernel: [2052824.841149] R13: 000055680fbd32e0 R14: 0000000000000000 R15: 00007ffd2984db14
 
Last edited:
I only have access to a rescue system (grml) which does not have zfs tools installed. Also it is a 32 bit grml which prevents me from chrooting into the installed proxmox. Because else I would have tried to recreate a pool of the same name to see if this lets boot the server os without panic. I moved the zpool.cache file away from /etc/zfs/ what else can be done to let system(d) forget about this not existing zpool thus enable successful boot of ProxMox installed on ext4? Is there a way to completely disable any zfs related service by editing some systemd config file or at least disable automatic import of zpools? Can I disable storage by a modification of a file from rescue system, as I cannot use pvesm set zfs --disable 0 while proxmox is not running
 
Last edited:
As the only pool was a data but no root pool I finally found out that it can not be related to zfs. The problem was a failed disk, which corrupted the root partition on my md0 device. I finally took the opportunity to reinstall after disk has been changed. This time I did not install debian and added ProxMox later, but installed from the ProxMox ISO directly, created a small (32GB) mirrored root Pool on zfs and created a second, encrypted pool for data.
 
You must log in or register to reply here.
Top Bottom