Node lost after update Proxmox

[digidax]

Hi there,

I did update a node to the latest available version of PVE today. After rebooting, I can ping the node but get no access via SSH or GUI, also not from an other node of the cluster. When I plug on a video monitor, the displaying of the boot up sequenze will end with a blank screen and the monitor switches off. Tested with VGA and HDMI and a 2nd monitor. I have taken a video about the boot sequence and this is the last visible output:



Anyone a idea?

Thanks Frank

 

[oguz]

hi,

* does it work if you boot from the older kernel (the one that you had before the update)
* what hardware do you have?

 

[digidax]

Its a ASUS Prime A320M-A Board with CPU socket 1331
CPU: AMD A8-9600 Radeon

Sorry for asking, how can I select the kernel I want to boot?
After starting up, I can only select "Start Proxmox PVE" or reboot to Firmware setting (BIOS).

Have booted from an USB drive with the Proxmox Iso installer, when I select "Rescue Boot", I get the error:
no such device rpool
ERROR: unabel to find boot disc automatically

Thanks, Frank

 

[entilza]

I don't think rescue works with ZFS.

Just before proxmox begins booting, You should see a screen that lets you pick the kernel. It times out after a few seconds so press the arrow key to stop it and pick.

 

[digidax]

Yes, with other Linux distros I work, I have this possibility to select a kernel or have line for manually adding parameter for the kernel I want to boot.
But not here. Have tried it 3 times, hitting the down or up arrow key, only this menue comes up:

Where is coming from the "macOS" tab? This node was setting up 5 months ago with two new HDDs. I have not used any macOS systems here,
Very strange.

I think, reinstalling of the node will be the only way I have to it running or are there any other ways?
The two HDDs are bonded during the first installation as a ZFS mirror.

 

[digidax]

After reading our task logs we've done on the machine, we found this:

The node has a built in LTO streamer, in a LXC container Bareos (Bacula fork) makes the job for the backup.
There is used "backup to disk" before "disk to tape" to avoid start-stop issues with the tape.
The tape works with 300 MB/s and is connected to the SAS HBA. One HDD was used for spooling, but with 190 MB/s too slow.
A 2nd HDD was combined as a ZFS stripe, this produces 240 MB/s read speed. So the ZFS stripe is not much faster.
Then mdadm was installed and the two disks for spooling added to a RAID-0 stripe, now we get 310 MB/s. This works nice since 3 months until I did yesterday a PVE update. Not the update seems the problem to be, it was a reboot and using of a changed kernel.

I have now installed PVE 6.3 from scratch: no problems
Then installed mdadm:

# apt-get install mdadm
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
  dracut-core
The following NEW packages will be installed:
  mdadm
0 upgraded, 1 newly installed, 0 to remove and 68 not upgraded.
Need to get 449 kB of archives.
After this operation, 1,240 kB of additional disk space will be used.
Get:1 http://ftp.de.debian.org/debian buster/main amd64 mdadm amd64 4.1-1 [449 kB]
Fetched 449 kB in 1s (898 kB/s)
Preconfiguring packages ...
Selecting previously unselected package mdadm.
(Reading database ... 44220 files and directories currently installed.)
Preparing to unpack .../archives/mdadm_4.1-1_amd64.deb ...
Unpacking mdadm (4.1-1) ...
Setting up mdadm (4.1-1) ...
Generating mdadm.conf... done.
update-initramfs: deferring update (trigger activated)
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.4.73-1-pve
Found initrd image: /boot/initrd.img-5.4.73-1-pve
Found memtest86+ image: /ROOT/pve-1@/boot/memtest86+.bin
Found memtest86+ multiboot image: /ROOT/pve-1@/boot/memtest86+_multiboot.bin
Adding boot menu entry for EFI firmware configuration
done
update-rc.d: warning: start and stop actions are no longer supported; falling back to defaults
Processing triggers for man-db (2.8.5-2) ...
Processing triggers for systemd (241-7~deb10u4) ...
Processing triggers for initramfs-tools (0.133+deb10u1) ...
update-initramfs: Generating /boot/initrd.img-5.4.73-1-pve
Running hook script 'zz-pve-efiboot'..
Re-executing '/etc/kernel/postinst.d/zz-pve-efiboot' in new private mount namespace..
Copying and configuring kernels on /dev/disk/by-uuid/B07A-09FC
        Copying kernel and creating boot-entry for 5.4.73-1-pve

After rebooting, PVE 6.3 is broken. As you can see above, mdadm reconfigured the kernel.
Now, on a fresh install I can see, that no new kernel was installed, so I can't switch back.
UEFI boot is disable in BIOS.

After a new installation, SSH and WebGUI is working, but aber some lines during the startup, the dsplay connected on the node goes in standby, no local access possible on the node.

So how can I get working mdadm on a PVE node and how can I get local acces?