Unable to boot into proxmox after restart

gsm113 · Dec 3, 2022

First time Proxmox user here. I set up my machine in September with the intent to host a Windows VM and a couple Linux VMs. It's been up serving as a surveillance NVR (Blue Iris) since then with no issues until today.

I wanted to create a new LXC container for pihole, and while setting it up the dropdown menu for setting the storage location was empty, which I thought was odd. So I did a reboot and the machine fails to start proxmox, getting the following error:

Failed to start File System Check on /dev/disk/by-label/storageprox.
Dependency failed for /mnt/data.
Dependency failed for Local File Systems.

Any help would be greatly appreciated.

Here is part of the journalctl that seemed relevant:

Dec 03 11:08:04 proxmox systemd-journald[329]: Journal started
Dec 03 11:08:04 proxmox systemd-journald[329]: Runtime Journal (/run/log/journal/8273b157844d4c4a8a268cd8134ad487) is 8.0M, max 158.7M, 150.7M free.
Dec 03 11:08:04 proxmox systemd-modules-load[330]: Inserted module 'iscsi_tcp'
Dec 03 11:08:04 proxmox lvm[323]: 2 logical volume(s) in volume group "pve" monitored
Dec 03 11:08:04 proxmox systemd[1]: Starting Flush Journal to Persistent Storage...
Dec 03 11:08:04 proxmox systemd[1]: Started Journal Service.
Dec 03 11:08:04 proxmox systemd-journald[329]: Time spent on flushing to /var/log/journal/8273b157844d4c4a8a268cd8134ad487 is 61.183ms for 879 entries.
Dec 03 11:08:04 proxmox systemd-journald[329]: System Journal (/var/log/journal/8273b157844d4c4a8a268cd8134ad487) is 72.0M, max 4.0G, 3.9G free.
Dec 03 11:08:04 proxmox kernel: iscsi: registered transport (iser)
Dec 03 11:08:04 proxmox kernel: spl: loading out-of-tree module taints kernel.
Dec 03 11:08:04 proxmox kernel: znvpair: module license 'CDDL' taints kernel.
Dec 03 11:08:04 proxmox kernel: Disabling lock debugging due to kernel taint
Dec 03 11:08:04 proxmox systemd-modules-load[330]: Inserted module 'ib_iser'
Dec 03 11:08:04 proxmox systemd[1]: Finished Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling.
Dec 03 11:08:04 proxmox systemd[1]: Reached target Local File Systems (Pre).
Dec 03 11:08:04 proxmox systemd-modules-load[330]: Inserted module 'vhost_net'
Dec 03 11:08:04 proxmox systemd[1]: Finished Coldplug All udev Devices.
Dec 03 11:08:04 proxmox systemd[1]: Starting Helper to synchronize boot up for ifupdown...
Dec 03 11:08:04 proxmox systemd[1]: Starting Wait for udev To Complete Device Initialization...
Dec 03 11:08:04 proxmox systemd[1]: Started Rule-based Manager for Device Events and Files.
Dec 03 11:08:04 proxmox systemd[1]: Finished Flush Journal to Persistent Storage.
Dec 03 11:08:04 proxmox kernel: ee1004 0-0051: 512 byte EE1004-compliant SPD EEPROM, read-only
Dec 03 11:08:04 proxmox kernel: mei_me 0000:00:16.0: enabling device (0000 -> 0002)
Dec 03 11:08:04 proxmox kernel: ee1004 0-0053: 512 byte EE1004-compliant SPD EEPROM, read-only
Dec 03 11:08:04 proxmox systemd-udevd[369]: Using default interface naming scheme 'v247'.
Dec 03 11:08:04 proxmox systemd[1]: Created slice system-lvm2\x2dpvscan.slice.
Dec 03 11:08:04 proxmox systemd-udevd[369]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Dec 03 11:08:04 proxmox systemd[1]: Starting LVM event activation on device 259:3...
Dec 03 11:08:04 proxmox lvm[399]: pvscan[399] PV /dev/nvme0n1p3 online, VG pve is complete.
Dec 03 11:08:04 proxmox lvm[399]: pvscan[399] VG pve skip autoactivation.
Dec 03 11:08:04 proxmox systemd[1]: Found device TEAM TM8FP6256G 2.
Dec 03 11:08:04 proxmox systemd[1]: Starting File System Check on /dev/disk/by-uuid/E91B-774C...
Dec 03 11:08:04 proxmox systemd[1]: Started File System Check Daemon to report status.
Dec 03 11:08:04 proxmox kernel: input: PC Speaker as /devices/platform/pcspkr/input/input5
Dec 03 11:08:04 proxmox systemd-fsck[410]: fsck.fat 4.2 (2021-01-31)
Dec 03 11:08:04 proxmox systemd-fsck[410]: /dev/nvme0n1p2: 5 files, 82/130812 clusters
Dec 03 11:08:04 proxmox systemd[1]: Finished File System Check on /dev/disk/by-uuid/E91B-774C.
Dec 03 11:08:04 proxmox kernel: asus_wmi: ASUS WMI generic driver loaded
Dec 03 11:08:04 proxmox kernel: asus_wmi: Initialization: 0x0
Dec 03 11:08:04 proxmox kernel: asus_wmi: BIOS WMI version: 0.9
Dec 03 11:08:04 proxmox kernel: asus_wmi: SFUN value: 0x0
Dec 03 11:08:04 proxmox kernel: eeepc-wmi eeepc-wmi: Detected ASUSWMI, use DCTS
Dec 03 11:08:04 proxmox kernel: input: Eee PC WMI hotkeys as /devices/platform/eeepc-wmi/input/input6
Dec 03 11:08:04 proxmox kernel: RAPL PMU: API unit is 2^-32 Joules, 4 fixed counters, 655360 ms ovfl timer
Dec 03 11:08:04 proxmox kernel: RAPL PMU: hw unit of domain pp0-core 2^-14 Joules
Dec 03 11:08:04 proxmox kernel: RAPL PMU: hw unit of domain package 2^-14 Joules
Dec 03 11:08:04 proxmox kernel: RAPL PMU: hw unit of domain dram 2^-14 Joules
Dec 03 11:08:04 proxmox kernel: RAPL PMU: hw unit of domain pp1-gpu 2^-14 Joules
Dec 03 11:08:04 proxmox kernel: cryptd: max_cpu_qlen set to 1000
Dec 03 11:08:04 proxmox systemd[1]: Mounting /boot/efi...
Dec 03 11:08:04 proxmox systemd[1]: Finished LVM event activation on device 259:3.
Dec 03 11:08:04 proxmox systemd[1]: Mounted /boot/efi.
Dec 03 11:08:04 proxmox kernel: pstore: Using crash dump compression: deflate
Dec 03 11:08:04 proxmox kernel: pstore: Registered efi as persistent store backend
Dec 03 11:08:04 proxmox kernel: AVX2 version of gcm_enc/dec engaged.
Dec 03 11:08:04 proxmox kernel: AES CTR mode by8 optimization enabled
Dec 03 11:08:04 proxmox kernel: cfg80211: Loading compiled-in X.509 certificates for regulatory database
Dec 03 11:08:04 proxmox kernel: cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'
Dec 03 11:08:04 proxmox kernel: platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
Dec 03 11:08:04 proxmox kernel: cfg80211: failed to load regulatory.db
Dec 03 11:08:04 proxmox systemd[1]: Found device WDC_WD42PURZ-85B4YY0 storageprox.
Dec 03 11:08:04 proxmox systemd-udevd[360]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Dec 03 11:08:04 proxmox systemd[1]: Starting File System Check on /dev/disk/by-label/storageprox...
Dec 03 11:08:04 proxmox systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.
Dec 03 11:08:04 proxmox udevadm[357]: systemd-udev-settle.service is deprecated. Please fix zfs-import-scan.service, zfs-import-cache.service not to pull it in.
Dec 03 11:08:04 proxmox kernel: ZFS: Loaded module v2.1.4-pve1, ZFS pool version 5000, ZFS filesystem version 5
Dec 03 11:08:04 proxmox kernel: rtl8821ae 0000:03:00.0: enabling device (0000 -> 0003)
Dec 03 11:08:04 proxmox systemd-modules-load[330]: Inserted module 'zfs'
Dec 03 11:08:04 proxmox systemd[1]: Finished Load Kernel Modules.
Dec 03 11:08:04 proxmox kernel: checking generic (e0000000 300000) vs hw (f6000000 1000000)
Dec 03 11:08:04 proxmox kernel: checking generic (e0000000 300000) vs hw (e0000000 10000000)
Dec 03 11:08:04 proxmox kernel: fb0: switching to i915 from simple
Dec 03 11:08:04 proxmox systemd[1]: Found device /dev/pve/swap.
Dec 03 11:08:04 proxmox systemd[1]: Activating swap /dev/pve/swap...
Dec 03 11:08:04 proxmox systemd[1]: Starting Apply Kernel Variables...
Dec 03 11:08:04 proxmox systemd[1]: Finished Apply Kernel Variables.
Dec 03 11:08:04 proxmox kernel: Console: switching to colour dummy device 80x25
Dec 03 11:08:04 proxmox kernel: i915 0000:00:02.0: vgaarb: deactivate vga console
Dec 03 11:08:04 proxmox kernel: i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem

wns=io+mem
Dec 03 11:08:04 proxmox kernel: Adding 8388604k swap on /dev/mapper/pve-swap. Priority:-2 extents:1 across:8388604k SSFS
Dec 03 11:08:04 proxmox systemd[1]: Activated swap /dev/pve/swap.
Dec 03 11:08:04 proxmox systemd[1]: Reached target Swap.
Dec 03 11:08:04 proxmox kernel: mei_hdcp 0000:00:16.0-b638ab7e-94e2-4ea2-a552-d1c54b627f04: bound 0000:00:02.0 (ops i915_hdcp_component_ops [i915])
Dec 03 11:08:04 proxmox kernel: i915 0000:00:02.0: [drm] Finished loading DMC firmware i915/kbl_dmc_ver1_04.bin (v1.4)
Dec 03 11:08:04 proxmox kernel: rtl8821ae: Using firmware rtlwifi/rtl8812aefw.bin
Dec 03 11:08:04 proxmox kernel: rtl8821ae: Using firmware rtlwifi/rtl8812aefw_wowlan.bin
Dec 03 11:08:04 proxmox systemd[1]: Starting Load/Save RF Kill Switch Status...
Dec 03 11:08:04 proxmox kernel: ieee80211 phy0: Selected rate control algorithm 'rtl_rc'
Dec 03 11:08:04 proxmox kernel: rtl8821ae 0000:03:00.0: Direct firmware load for rtlwifi/rtl8812aefw_wowlan.bin failed with error -2
Dec 03 11:08:04 proxmox kernel: rtl8821ae 0000:03:00.0: Direct firmware load for rtlwifi/rtl8812aefw.bin failed with error -2
Dec 03 11:08:04 proxmox kernel: rtlwifi: rtlwifi: wireless switch is on
Dec 03 11:08:04 proxmox kernel: rtlwifi: Loading alternative firmware rtlwifi/rtl8821aefw.bin
Dec 03 11:08:04 proxmox kernel: rtlwifi: Loading alternative firmware rtlwifi/rtl8821aefw.bin
Dec 03 11:08:04 proxmox systemd-udevd[378]: Using default interface naming scheme 'v247'.
Dec 03 11:08:04 proxmox systemd-udevd[378]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Dec 03 11:08:04 proxmox kernel: rtl8821ae 0000:03:00.0 wlp3s0: renamed from wlan0
Dec 03 11:08:04 proxmox systemd[1]: Started Load/Save RF Kill Switch Status.
Dec 03 11:08:04 proxmox systemd-fsck[466]: storageprox: Note: if several inode or block bitmap blocks or part
Dec 03 11:08:04 proxmox systemd-fsck[466]: of the inode table require relocation, you may wish to try
Dec 03 11:08:04 proxmox systemd-fsck[466]: running e2fsck with the '-b 32768' option first. The problem
Dec 03 11:08:04 proxmox systemd-fsck[466]: may lie only with the primary block group descriptors, and
Dec 03 11:08:04 proxmox systemd-fsck[466]: the backup block group descriptors may be OK.
Dec 03 11:08:04 proxmox systemd-fsck[466]: storageprox: Block bitmap for group 0 is not in group. (block 7449066062970102630)
Dec 03 11:08:04 proxmox systemd-fsck[466]: storageprox: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
Dec 03 11:08:04 proxmox systemd-fsck[466]: (i.e., without -a or -p options)
Dec 03 11:08:04 proxmox kernel: [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.0 on minor 0
Dec 03 11:08:04 proxmox systemd-udevd[360]: Using default interface naming scheme 'v247'.
Dec 03 11:08:04 proxmox systemd-udevd[360]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Dec 03 11:08:04 proxmox kernel: ACPI: video: Video Device [GFX0] (multi-head: yes rom: no post: no)
Dec 03 11:08:04 proxmox kernel: input: Video Bus as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A08:00/LNXVIDEO:00/input/input7
Dec 03 11:08:04 proxmox kernel: snd_hda_intel 0000:00:1f.3: bound 0000:00:02.0 (ops i915_audio_component_bind_ops [i915])
Dec 03 11:08:04 proxmox kernel: fbcon: i915drmfb (fb0) is primary device
Dec 03 11:08:05 proxmox kernel: Console: switching to colour frame buffer device 240x67
Dec 03 11:08:05 proxmox systemd-fsck[448]: fsck failed with exit status 4.
Dec 03 11:08:05 proxmox systemd-fsck[448]: Running request emergency.target/start/replace
Dec 03 11:08:05 proxmox systemd[1]: systemd-fsck@dev-disk-by\x2dlabel-storageprox.service: Main process exited, code=exited, status=1/FAILURE
Dec 03 11:08:05 proxmox systemd[1]: systemd-fsck@dev-disk-by\x2dlabel-storageprox.service: Failed with result 'exit-code'.
Dec 03 11:08:05 proxmox systemd[1]: Failed to start File System Check on /dev/disk/by-label/storageprox.
Dec 03 11:08:05 proxmox kernel: i915 0000:00:02.0: [drm] fb0: i915drmfb frame buffer device
Dec 03 11:08:05 proxmox systemd[1]: Dependency failed for /mnt/data.
Dec 03 11:08:05 proxmox systemd[1]: Dependency failed for Local File Systems.
Dec 03 11:08:05 proxmox systemd[1]: local-fs.target: Job local-fs.target/start failed with result 'dependency'.
Dec 03 11:08:05 proxmox systemd[1]: local-fs.target: Triggering OnFailure= dependencies.
Dec 03 11:08:05 proxmox systemd[1]: mnt-data.mount: Job mnt-data.mount/start failed with result 'dependency'.

shanreich · Dec 5, 2022

Looks like an issue with your filesystem. This can be caused by the underlying disk malfunctioning. Can you post the output of the following commands?

Code:

lsblk --ascii
ls -l /dev/disk/by-*/

gsm113 · Dec 10, 2022

Output from these 2 commands.

shanreich · Dec 12, 2022

It would be smart to run fsck manually on that disk, in order to check the filesystem manually for any inconsistencies. This should be done while the respective filesystem is not mounted.

You can then just run the filesystem check via:

Code:

fsck /dev/sda1

Additionally it would be interesting why the filesystem had inconsistencies in the first place. This can be due to power outage, but could also be a sign that your disk is failing. You should at least check the SMART values of the respective disk via

Code:

smartctl -a /dev/sda

Please also make sure you have/create a backup of this disk asap, as long as you don't know what is causing the problems.

Search

Search

Unable to boot into proxmox after restart

gsm113

New Member

shanreich

Proxmox Staff Member

gsm113

New Member

shanreich

Proxmox Staff Member

We value your privacy