Proxmox 9 boot issues

S

sagiziv3

Member
Sep 14, 2024
11
0
6
I have an HPE microserver gen 8 that I just upgrade to version 9 and I get those terrible failures on boot.
I initially tried to do a clean install but it showed an error related to GRUB files, so I installed version 8 and then upgraded to 9 (`pve8to9 --full` showed no errors).
I followed the official guide and everything seemed to go well, but when I rebooted the machine I got those errors.

I can boot to the previous version of the kernel and to the recovery mode of the new kernel, but I am not sure what I need to do from here to fix this.
I can continue with the old kernel, but I still prefer to be on the latest version if possible.



1791623949192.png
 
Could you please share the output using the forum’s code formatting?

This may be related to filesystem errors, an underlying disk or I/O issue, or a storage-driver regression in the newer kernel.
 
Last edited:
  • Like
Reactions: leesteken
sagiziv3 said:
@kawano-y I got it from the ILO interface so I don't really have it as text, but I'll convert it
Click to expand...
Markdown (GitHub flavored): 
[   16.434665] EXT4-fs (dm-1): Remounting filesystem read-only
[FAILED] Failed to start tailscaled.service - Tailscale node agent.
[FAILED] Failed to start tailscaled.service - Tailscale node agent.
[FAILED] Failed to start lxc-net.service - LXC network bridge setup.
[FAILED] Failed to start open-iscsi.service - Login to default iSCSI targets.
[FAILED] Failed to start pve-cluster.service - The Proxmox VE cluster filesystem.
[FAILED] Failed to start rbdmap.service - Map RBD devices.
[FAILED] Failed to start ssh.service - OpenBSD Secure Shell server.
[FAILED] Failed to start tailscaled.service - Tailscale node agent.
[FAILED] Failed to start blk-availability.service - Availability of block devices.
[FAILED] Failed to start lxc.service - LXC Container Initialization and Autoboot Code.
[FAILED] Failed to start pve-firewall-commit.service - Commit Proxmox VE Firewall changes.
[FAILED] Failed to start pve-sdn-commit.service - Commit Proxmox VE SDN changes.
[FAILED] Failed to start pvedaemon.service - PVE API Daemon.
[FAILED] Failed to start pvestatd.service - PVE Status Daemon.
[FAILED] Failed to start systemd-user-sessions.service - Permit User Sessions.
[FAILED] Failed to start pveproxy.service - PVE API Proxy Server.
[FAILED] Failed to start pve-firewall.service - Proxmox VE firewall.
[FAILED] Failed to start spiceproxy.service - PVE SPICE Proxy Server.
[FAILED] Failed to start pve-guests.service - PVE guests.
[FAILED] Failed to start pvescheduler.service - Proxmox VE scheduler.
[FAILED] Failed to start pve-cluster.service - The Proxmox VE cluster filesystem.
[FAILED] Failed to start ssh.service - OpenBSD Secure Shell server.
[FAILED] Failed to start pve-cluster.service - The Proxmox VE cluster filesystem.
[FAILED] Failed to start ssh.service - OpenBSD Secure Shell server.
[FAILED] Failed to start pvedaemon.service - PVE API Daemon.
[FAILED] Failed to start tailscaled.service - Tailscale node agent.
[FAILED] Failed to start pveproxy.service - PVE API Proxy Server.
[FAILED] Failed to start pve-cluster.service - The Proxmox VE cluster filesystem.
[FAILED] Failed to start ssh.service - OpenBSD Secure Shell server.
[FAILED] Failed to start pvedaemon.service - PVE API Daemon.
[FAILED] Failed to start tailscaled.service - Tailscale node agent.
[FAILED] Failed to start pveproxy.service - PVE API Proxy Server.
[FAILED] Failed to start tailscaled.service - Tailscale node agent.
[FAILED] Failed to start pve-cluster.service - The Proxmox VE cluster filesystem.
[FAILED] Failed to start ssh.service - OpenBSD Secure Shell server.
[FAILED] Failed to start pvedaemon.service - PVE API Daemon.
[FAILED] Failed to start pveproxy.service - PVE API Proxy Server.
[FAILED] Failed to start pve-cluster.service - The Proxmox VE cluster filesystem.
[FAILED] Failed to start ssh.service - OpenBSD Secure Shell server.
[FAILED] Failed to start pvedaemon.service - PVE API Daemon.
[FAILED] Failed to start pveproxy.service - PVE API Proxy Server.
[FAILED] Failed to start pve-cluster.service - The Proxmox VE cluster filesystem.
[FAILED] Failed to start cron.service - Regular background program processing daemon.
[FAILED] Failed to start pvedaemon.service - PVE API Daemon.
[FAILED] Failed to start ssh.service - OpenBSD Secure Shell server.
[FAILED] Failed to start pveproxy.service - PVE API Proxy Server.
 
sagiziv3 said:
[ 16.434665] EXT4-fs (dm-1): Remounting filesystem read-only
Click to expand...
Well..., now the seconds before this one would be interesting. Everything here is too late.

Is it possible to catch them the same way? There must be a reason for "remount read-only" - possibly/hopefully logged in the seconds before.

ILO has a local device management and an error log, right? Did you check that?

What is the state (on the console) at the end? Do you get a login prompt? (Just press "Enter" to (possibly) trigger it to get redrawn).
 
Last edited:
  • Like
Reactions: leesteken
UdoB said:
Well..., now the seconds before this one would be interesting. Everything here is too late.

Is it possible to catch them the same way? There must be a reason for "remount read-only" - possibly/hopefully logged in the seconds before.

ILO has a local device management and an error log, right? Did you check that?

What is the state (on the console) at the end? Do you get a login prompt? (Just press "Enter" to (possibly) trigger it to get redrawn).
Click to expand...
I don't see it in the ILO logs, will try to reboot again to capture those logs before.

Anyway, after those errors I get a blank console and no option to login and it doesn't respond to any input
 
UdoB said:
Well..., now the seconds before this one would be interesting. Everything here is too late.

Is it possible to catch them the same way? There must be a reason for "remount read-only" - possibly/hopefully logged in the seconds before.

ILO has a local device management and an error log, right? Did you check that?

What is the state (on the console) at the end? Do you get a login prompt? (Just press "Enter" to (possibly) trigger it to get redrawn).
Click to expand...
This is all the output I get after the selecting the new kernel, and again, after that is a blank screen (the zfs-error is fine, I have TrueNAS VM):


Markdown (GitHub flavored): 
 Found volume group "pve" using metadata type lvm2
 13 logical volume(s) in volume group "pve" now active
/dev/mapper/pve-root: clean, 90574/4308992 files, 4868709/17207296 blocks
[FAILED] Failed to start zfs-import-scan.service - Import ZFS pools by device scanning.
[   15.921391] EXT4-fs error (device dm-1): ext4_wait_block_bitmap:582: comm ext4lazyinit: Cannot read block bitmap - block_group = 125, block_bitmap = 3670029
[   15.921436] EXT4-fs error (device dm-1): ext4_journal_check_start:86: comm systemd-journal: Detected aborted journal
[   15.921483] EXT4-fs error (device dm-1): in ext4_reserve_inode_write:6305: IO failure
[   15.921578] EXT4-fs (dm-1): Remounting filesystem read-only
[FAILED] Failed to start pvebanner.service - Proxmox VE Login Banner.
[FAILED] Failed to start e2scrub_reap.service - Remove Stale Online ext4 Metadata Check Snapshots.
[FAILED] Failed to start tailscaled.service - Tailscale node agent.
[FAILED] Failed to start tailscaled.service - Tailscale node agent.
[FAILED] Failed to start lxc-net.service - LXC network bridge setup.
[FAILED] Failed to start open-iscsi.service - Login to default iSCSI targets.
[FAILED] Failed to start pve-cluster.service - The Proxmox VE cluster filesystem.
[FAILED] Failed to start rbdmap.service - Map RBD devices.
[FAILED] Failed to start ssh.service - OpenBSD Secure Shell server.
[FAILED] Failed to start blk-availability.service - Availability of block devices.
[FAILED] Failed to start lxc.service - LXC Container Initialization and Autoboot Code.
[FAILED] Failed to start pve-firewall-commit.service - Commit Proxmox VE Firewall changes.
[FAILED] Failed to start pve-sdn-commit.service - Commit Proxmox VE SDN changes.
[FAILED] Failed to start pvedaemon.service - PVE API Daemon.
[FAILED] Failed to start pvestatd.service - PVE Status Daemon.
[FAILED] Failed to start systemd-user-sessions.service - Permit User Sessions.
[FAILED] Failed to start pve-firewall.service - Proxmox VE firewall.
[FAILED] Failed to start pveproxy.service - PVE API Proxy Server.
[FAILED] Failed to start tailscaled.service - Tailscale node agent.
[FAILED] Failed to start spiceproxy.service - PVE SPICE Proxy Server.
[FAILED] Failed to start pve-guests.service - PVE guests.
[FAILED] Failed to start pve-cluster.service - The Proxmox VE cluster filesystem.
[FAILED] Failed to start ssh.service - OpenBSD Secure Shell server.
[FAILED] Failed to start pvescheduler.service - Proxmox VE scheduler.
 
Please boot with the older kernel, log in, and run:

Code: 
journalctl --list-boots

Please share the output. We can then identify the failed boot and determine the command needed to retrieve its kernel log.
 
Last edited:
  • Like
Reactions: UdoB
You must log in or register to reply here.
Top Bottom
Back