VPS crashing

P

paradox55

Well-Known Member
May 31, 2019
103
10
58
35
I have a couple 32GB RAM orange pi 5 pluses I'm attempting to use with the arm64 proxmox.

I am using the following UEFI firmware: https://github.com/edk2-porting/edk2-rk3588

VPS run, but they crash frequently. I thought at first it was a OS issue (ubuntu 26.04) as debian 13 seemed solid. Yet, I just setup another OP5+ node and the VPS is crashing frequently on that as well... during installation of the VPS.

I checked dmesg on the OP5+ and there are no errors listed.

How would I go about debugging this? I have ACPI selected in the UEFI firmware and not device tree.
 
After messing around with this more. QEMU (Max) CPU flags don't cause the crash.

Yet, Host flags do.

I also noticed that I can't manually select either cortex-a55 or corex-a76.

I think this is a bug?
 
Your dmesg is clean because the host isn't the thing crashing. The guests are. Host dmesg will never show a guest kernel panic. Three things I'd do:
1. Capture the guest console.
Code: 
# On the host: attach a serial socket to the VM
qm set <vmid> -serial0 socket

# Connect to it (leave this open when the guest crashes)
qm terminal <vmid>

# In the guest, add to the kernel cmdline in /etc/default/grub:
#   GRUB_CMDLINE_LINUX_DEFAULT="... console=ttyAMA0,115200 console=tty0"
# then: update-grub && reboot

# Host-side logs for that VM
cat /var/log/pve/qemu-server/<vmid>.log
journalctl -u qemu-server@<vmid> -b
That tells you whether the guest is panicking or QEMU itself is exiting. Different bugs.
2. Pin the VM to the big cores. The RK3588 is 4x A76 plus 4x A55, and KVM shows the guest one sanitized CPU. When a vCPU migrates between clusters mid-execution, guests fall over. That's why an install crashes, it's the heaviest scheduling churn of the whole lifecycle. Confirm the layout with lscpu -e. The ~2.4GHz cores are the A76s, usually cpu4-7. Then:
Code: 
qm set <vmid> --affinity 4-7 --cores 2
If the crashes stop, you found it. Also explains why Debian 13 felt solid and Ubuntu 26.04 doesn't. Newer guest kernel, more aggressive CPU feature use.
3. Check what ACPI is actually telling KVM. Picking ACPI in the edk2 menu is necessary but not sufficient. That firmware's ACPI path is developed and tested against Windows.
Code: 
dmesg | grep -iE 'gic|arch_timer|hyp|vhe'
ls /sys/firmware/acpi/tables/
A bad MADT or GTDT gives you a host that boots fine and guests that die under load.

The other thing to consider is RK3588 is ARMv8.2-A. The Proxmox arm64 floor is ARMv9-A, so you're below the baseline, not in the best-effort tier.
 
  • Like
Reactions: leesteken
No, nevermind. I just got lucky. I've found every 1 in 10 restarts to be stable on the KVM.

I'm not sure at this point. I guess I'll just live with LXC (Which is still better than bare metal/docker!)
 
In case anyone is still following this little saga, it turns out the virtio network driver was causing the crashing.

E1000 seems solid.
 
You must log in or register to reply here.
Top Bottom
Back