After update to proxmox-kernel-6.5.13-5-pve-signed:amd64 (6.5.13-5, automatic) GUI inaccesible, SSH works okay.
- Thread starter k.jings
- Start date
What is the output of:
journalctl -u pvedaemon -b
ip a
hostname
cat /etc/hosts
cat /etc/network/interfaces
more /etc/hostname
journalctl -n 500
what happens when you stop the service and try : pvedaemon --verbose
Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
journalctl -u pvedaemon -b
ip a
hostname
cat /etc/hosts
cat /etc/network/interfaces
more /etc/hostname
journalctl -n 500
what happens when you stop the service and try : pvedaemon --verbose
Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
Hi everyone,
Thanks for your potential troubleshooting steps, bbgeek17. I realize I should have provided more context in my initial post.
Environment:
Following this recent Proxmox upgrade, my drbd version was downgraded from 9.2.8-2 (built with pve-headers in the DKMS tree) to 8.4.11. This downgrade caused issues with the LINSTOR satellite service, which requires >= DRBD 9.
Error Message:
systemctl status linstor-satellite.service (linstor.satellite.txt). Excerpt:
Apr 10 17:07:21 node3 Satellite[1185]: 17:07:21.732 [MainWorkerPool-2] ERROR LINSTOR/Satellite - SYSTEM - Received a resource that requires DRBD9_KERNEL but that external tool is not supported on this satellite [Report number 6616C73F-73CF5-000000]
Apr 10 17:07:21 node3 Satellite[1185]: 17:07:21.735 [MainWorkerPool-2] ERROR LINSTOR/Satellite - SYSTEM - FullSync error: missing required ext tools 3
cat /proc/drbd
version: 8.4.11 (api:1/proto:86-101)
srcversion: BAA5FC0DB44B6F02BBEAD0B
The downgrade could potentially have been caused by any of the upgraded packages.
Questions:
Thanks in advance for any assistance!
journalctl -u pvedaemon -b
hostname
cat /etc/network/interfaces (network.interfaces.txt)
more /etc/hostname
what happens when you stop the service and try : pvedaemon --verbose
systecmctl stop pvedaemone
pvedaemon --verbose
Thanks for your potential troubleshooting steps, bbgeek17. I realize I should have provided more context in my initial post.
Environment:
- 3-node Proxmox Cluster
- LINSTOR with proxmox-plugin (Linstor-HA mode)
Following this recent Proxmox upgrade, my drbd version was downgraded from 9.2.8-2 (built with pve-headers in the DKMS tree) to 8.4.11. This downgrade caused issues with the LINSTOR satellite service, which requires >= DRBD 9.
Error Message:
systemctl status linstor-satellite.service (linstor.satellite.txt). Excerpt:
Apr 10 17:07:21 node3 Satellite[1185]: 17:07:21.732 [MainWorkerPool-2] ERROR LINSTOR/Satellite - SYSTEM - Received a resource that requires DRBD9_KERNEL but that external tool is not supported on this satellite [Report number 6616C73F-73CF5-000000]
Apr 10 17:07:21 node3 Satellite[1185]: 17:07:21.735 [MainWorkerPool-2] ERROR LINSTOR/Satellite - SYSTEM - FullSync error: missing required ext tools 3
cat /proc/drbd
version: 8.4.11 (api:1/proto:86-101)
srcversion: BAA5FC0DB44B6F02BBEAD0B
The downgrade could potentially have been caused by any of the upgraded packages.
Questions:
- Given my LINSTOR setup, is this a matter for the Proxmox team or the LINSTOR developers?
- How can I best address the downgraded DRBD version and restore functionality to my LINSTOR HA setup, especially the satellite service?
Thanks in advance for any assistance!
journalctl -u pvedaemon -b
ip a (ip.a.txt)
hostname
cat /etc/hosts (etc.hosts.txt)
cat /etc/network/interfaces (network.interfaces.txt)
more /etc/hostname
journalctl -n 500 (journalctl.n.500.txt)
what happens when you stop the service and try : pvedaemon --verbose
systecmctl stop pvedaemone
pvedaemon --verbose
Your "GUI inaccesible, SSH works okay" just became 10x more complicated than originally presented.
It does seem like communication with Linbit might be interfering in proper cluster operations. However, there is not enough information to be sure - the log data you presented is not helpful. You could try to disable/remove Linbit storage from PVE config temporary to see if that helps.
Beyond that, seeing as you dont have subscription, based on forum information, your options are:
- troubleshoot on your own (ie look at journal log, the one you attached contains nothing but an error connecting to Linbit controller. You need to go further in log. Reboot systems and analyze the log from second 0.
- stay in forum and hope that one of the members has enough familiarity with your specific combination of technologies.
- buy PVE subscription and work with PVE support, and/or Linbit support ( I am not familiar with their support offering).
- recover from backups prior to upgrade.
- reinstall and recover VM data from backups.
Good luck
Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
It does seem like communication with Linbit might be interfering in proper cluster operations. However, there is not enough information to be sure - the log data you presented is not helpful. You could try to disable/remove Linbit storage from PVE config temporary to see if that helps.
Beyond that, seeing as you dont have subscription, based on forum information, your options are:
- troubleshoot on your own (ie look at journal log, the one you attached contains nothing but an error connecting to Linbit controller. You need to go further in log. Reboot systems and analyze the log from second 0.
- stay in forum and hope that one of the members has enough familiarity with your specific combination of technologies.
- buy PVE subscription and work with PVE support, and/or Linbit support ( I am not familiar with their support offering).
- recover from backups prior to upgrade.
- reinstall and recover VM data from backups.
Good luck
Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
You have to install kernel headers manually, dkms won't build anything without headers:
dkms: autoinstall for kernel 6.5.13-5-pve was skipped since the kernel headers for this kernel do not seem to be installed.apt install proxmox-headers-6.5dpkg-reconfigure proxmox-kernel-6.5.13-5-pve-signed