Proxmox - Kernel Panic

[henriii]

Hi,

I'm a beginner in proxmox, and now, after around 24h I got the below error message on my screen, and nothing is working anymore.

Here are my main hardwares:

• AMD Ryzen 9 7950X
• ASUS TUF GAMING B650-PLUS
• 2x Samsung 990 Pro 4 To
• CORSAIR VENGEANCE RGB DDR5 RAM 4x 32Go

I installed proxmox 7.4-17 on it.

Since I cannot copy/paster the logs, I took a picture of the screen. Nothing works anymore. Yesterday, when it happened for the first time, it worked again after restarting.


Thanks a lot for your support, I really don't know what to do...

 

[esi_y]

You would need to provide more details in how you installed it, also why not install most recent v8.1 since this is a fresh install apparently?

Could you post your history and journalctl -b -1 into [ CODE ] [ /CODE ] tags or attach as a file here?

 

[esi_y]

Since I cannot copy/paster the logs, I took a picture of the screen. Nothing works anymore. Yesterday, when it happened for the first time, it worked again after restarting.

If it does not boot anymore, you would need to live-boot e.g. Debian [1] and take it from there [1]. Or maybe record the boot screen messages with a camera post the video link it through here?

[1] https://www.debian.org/CD/live/

 

[henriii]

The system is still booting. When I restart it, everything goes normal (for how long?).

history (the name of my server is luigi)

root@pve:~# history
    1  shutdown now
    2  reboot
    3  nano /etc/default/grub
    4  update grub
    5  update-grub
    6  update-initramfs
    7  update-initramfs -u
    8  reboot
    9  ls
   10  sudo apt install ethtool -y
   11  apt install ethtool -y
   12  ip addr
   13  sudo ethtool eno1
   14  ethtool enno1
   15  ethtool eno1
   16  sudo ethtool -s eno1 wol g
   17  ethtool -s eno1 wol g
   18  ethtool eno1
   19  sudo nano /etc/network/interfaces
   20  nano /etc/network/interfaces
   21  ip addr
   22  ethtool eno1
   23  shutdown now
   24  ethtool eno1
   25  nano /etc/network/interfaces
   26  ethtool -s eno1 wol g
   27  ethtool eno1
   28  shutdown now
   29  ethtool eno1
   30  nano /etc/network/interfaces
   31  ethtool eno1
   32  ethtool -s eno1 wol g
   33  ethtool eno1
   34  shutdown now
   35  ethtool eno1
   36  ethtool eno1
   37  shutdown now
   38  shutdown now
   39  shutdown now
   40  lsblk
   41  mkdir /mnt/external
   42  ls
   43  ls
   44  ls -la
   45  mount /dev/sdc /mnt/external
   46  ls /mnt/external
   47  umount /mnt/external
   48  lsblk
   49  lsblk -f
   50  fsck.ext4 -f /dev/sdc
   51  mount /dev/sdc /mnt/external
   52  umount /mnt/external
   53  lsblk -f
   54  lsblk -f
   55  fsck.ext4 -f /dev/sdd
   56  lsblk -f
   57  mount /dev/sdd /mnt/external
   58  umount /mnt/external
   59  lsblk -f
   60  lsblk -f
   61  fsck.ext4 -f /dev/sdc
   62  mount /dev/sdc /mnt/external
   63  cd ..
   64  ls
   65  ls
   66  cd home/
   67  ls
   68  ls -la
   69  clear
   70  lsblk
   71  lsblk
   72  lsblk
   73  lsblk
   74  umount sdd
   75  fdisk /dev/sdd
   76  lsblk
   77  mkfs.ext4 /dev/sdd1
   78  lsblk
   79  ethtool eno1
   80  clear
   81  ls
   82  lsblk
   83  lsblk
   84  fdisk -l
   85  ls
   86  lsblk
   87  cd /mnt/
   88  ls
   89  cd external/
   90  ls
   91  cd ..
   92  mkdir backup-temp
   93  ls
   94  mount /dev/sdd1 /mnt/backup-temp/
   95  lsblk
   96  UUID=3f4c90be-ae0c-4680-a4f8-0d88cdb9e3b0 /mnt/backup-temp/ ext4 defaults 0 2
   97  UUID=3f4c90be-ae0c-4680-a4f8-0d88cdb9e3b0 /mnt/sdd1 ext4 defaults 0 2
   98  UUID=3f4c90be-ae0c-4680-a4f8-0d88cdb9e3b0 /mnt/external ext4 defaults 0 2
   99  nano /etc/fstab
  100  mount -a
  101  nano /etc/fstab
  102  mount -a
  103  ls
  104  lsblk
  105  df -h
  106  shutdown now
  107  shutdown now
  108  shutdown now
  109  clear
  110  ls -la
  111  apt update
  112  apt install nut-client
  113  cd /etc/nut/
  114  ls
  115  nano upsmon.conf
  116  lsblk
  117  umount /dev/sdc
  118  clear
  119  pvdisplay
  120  vgdisplay
  121  lvdisplay
  122  lsof
  123  fuser
  124  fuser sdc
  125  reboot
  126  ls
  127  cd /etc/nut/
  128  ls
  129  clear
  130  nano nut.conf
  131  nano upsmon.conf
  132  ls
  133  mkdir example
  134  cp upsmon.conf example/upsmon.confg
  135  cp upssched.conf example/upssched.conf
  136  ls
  137  cd example/
  138  ls
  139  cd ..
  140  clear
  141  nano upsmon.conf
  142  nano upsmon.conf
  143  nano nut.conf
  144  nano upssched.conf
  145  nano upssched.conf
  146  nano /bin/upssched-cmd
  147  nano /bin/upssched-cmd
  148  nano upssched.conf
  149  ls
  150  cd example/
  151  ls
  152  nano upssched.conf
  153  cd ..
  154  ls
  155  nano upssched.conf
  156  nano upssched-cmd
  157  nano upssched-cmd
  158  nano upssched-cmd
  159  nano /usr/local/bin/send_pushover.sh
  160  nano upssched-cmd
  161  nano /usr/local/bin/send_pushover.sh
  162  ls
  163  nano upssched.conf
  164  nano /usr/sbin/upssched
  165  nano upssched-cmd
  166  /usr/local/bin/send_pushover.sh " ✅ Communication OK - FROM LUIGI" " ✅ Communication between server/client is OK |  Current battery level : ${BATTERY_LEVEL}%"
  167  cd /usr/local/bin
  168  ls
  169  ls
  170  ls -la
  171  chmod +x send_pushover.sh
  172  ls -la
  173  send_pushover.sh " ✅ Communication OK - FROM LUIGI" " ✅ Communication between server/client is OK |  Current battery level : ${BATTERY_LEVEL}%"
  174  cd /etc/nut/
  175  ls
  176  nano upssched-cmd
  177  nano upssched-cmd
  178  nano upssched-cmd
  179  nano /usr/sbin/upsmon
  180  nano upssched-cmd
  181  /usr/local/bin/send_pushover.sh " ✅ Communication OK - FROM LUIGI" " ✅ Communication between server/client is OK |  Current battery level : ${BATTERY_LEVEL}%"
  182  clear
  183  ls -la
  184  nano nut.conf
  185  nano upsmon.conf
  186  nano upssched-cmd
  187  nano upssched.conf
  188  cleat
  189  clear
  190  ls -la
  191  cat nut.conf
  192  cat upsmon.conf
  193  cat upssched-cmd
  194  cat upssched.conf
  195  ls
  196  nano /usr/bin/upssched-cmd
  197  ls
  198  nano upssched-cmd
  199  nano /usr/bin/upssched-cmd
  200  nano upssched.conf
  201  nano upssched.conf
  202  nano /usr/bin/upssched-cmd
  203  systemctl status nut-client
  204  systemctl restart nut-client
  205  systemctl status nut-client
  206  systemctl status nut-client
  207  systemctl status nut-client
  208  nano upsmon.conf
  209  systemctl status nut-monitor
  210  systemctl restart nut-client
  211  systemctl status nut-monitor
  212  systemctl restart nut-client
  213  systemctl status nut-monitor
  214  systemctl status nut-monitor
  215  reboot
  216  cd /etc/nut/
  217  ls
  218  clear
  219  ls- la
  220  ls -la
  221  systemctl status nut-monitor
  222  cd /etc/nut/
  223  ls
  224  nano upssched.conf
  225  nano /usr/bin/upssched-cmd
  226  nano upssched.conf
  227  ls
  228  ls
  229  cd ..
  230  cd /etc/nut/
  231  ls
  232  clear
  233  nano upssched
  234  ls
  235  rm -r upssched-cmd
  236  nano upssched.
  237  nano upssched.conf
  238  nano /usr/bin/upssched-cmd
  239  upsc eaton1600-luigi@192.168.1.213
  240  clear
  241  nano /usr/bin/upssched-cmd
  242  upsc eaton1600-luigi@192.168.1.213
  243  clear
  244  nano /usr/bin/upssched-cmd
  245  nano /usr/bin/upssched-cmd
  246  systemctl restart nut-client
  247  systemctl status nut-client
  248  systemctl status nut-client
  249  qm list
  250  lsblk
  251  clear
  252  lsblk
  253  mkfs.ext4 sdc
  254  mkfs.ext4 /dev/sdc
  255  fdisk /dev/sdc
  256  lsblk
  257  mkdir mnt/bakcup-nvme
  258  mkdir /mnt/backup-nvme
  259  ls
  260  ls /mnt/
  261  ls /mnt/backup-temp/
  262  mount /dev/sdc1 /mnt/backup-nvme/
  263  lsblk
  264  blkid
  265  clear
  266  lsblk
  267  mkfs.ext4 /dev/sdc1
  268  lsblk
  269  mount /dev/sdc1 /mnt/backup-nvme/
  270  lsblk
  271  blkid
  272  nano /etc/fstab
  273  nano /etc/fstab
  274  mount -a
  275  df -h
  276  echo "Test email from Proxmox: $(hostname)" | /usr/bin/proxmox-mail-forward
  277  cat /var/log/mail.log
  278  apt install libsasl2-modules -y
  279  nano /etc/postfix/main.cf
  280  nano /etc/postfix/sasl_passwd
  281  postmap hash:/etc/postfix/sasl_passwd
  282  chmod 600 /etc/postfix/sasl_passwd
  283  systemctl restart postfix
  284  echo "Test email from Proxmox: $(hostname)" | mail -s "Proxmox Testing" <%youraccount%>@gmail.com
  285  echo "Test email from Proxmox: $(hostname)" | mail -s "Proxmox Testing" henri.rion@gmail.com
  286  echo "Test email from Proxmox: $(hostname)" | /usr/bin/proxmox-mail-forward
  287  lsblk
  288  exit
  289  exit
  290  less /var/log/kern.log
  291  clear
  292  /var/log/kern.log
  293  less /var/log/kern.log
  294  journalctl -b -1
  295  history

I don't see anything relevant (?)

Here is the journalctl -b -1

The crash occured on the Feb 14 20:04:00 (I know this thanks to my grafana / prometheus dashboard).
So I don't see relevant logs again...

Feb 14 20:39:56 pve kernel: Linux version 5.15.136-1-pve (build@proxmox) (gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP PVE 5.15.136-1 (2024-01-23T11:10Z) ()
Feb 14 20:39:56 pve kernel: Command line: BOOT_IMAGE=/boot/vmlinuz-5.15.136-1-pve root=/dev/mapper/pve-root ro quiet amd_iommu=on
Feb 14 20:39:56 pve kernel: KERNEL supported cpus:
Feb 14 20:39:56 pve kernel:   Intel GenuineIntel
Feb 14 20:39:56 pve kernel:   AMD AuthenticAMD
Feb 14 20:39:56 pve kernel:   Hygon HygonGenuine
Feb 14 20:39:56 pve kernel:   Centaur CentaurHauls
Feb 14 20:39:56 pve kernel:   zhaoxin   Shanghai 
Feb 14 20:39:56 pve kernel: BIOS-provided physical RAM map:
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009ffff] usable
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x00000000000a0000-0x00000000000fffff] reserved
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x0000000000100000-0x0000000009afefff] usable
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x0000000009aff000-0x0000000009ffffff] reserved
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x000000000a000000-0x000000000a1fffff] usable
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x000000000a200000-0x000000000a20ffff] ACPI NVS
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x000000000a210000-0x000000000affffff] usable
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x000000000b000000-0x000000000b020fff] reserved
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x000000000b021000-0x000000006db07fff] usable
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x000000006db08000-0x0000000074353fff] reserved
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x0000000074354000-0x00000000744d1fff] ACPI data
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x00000000744d2000-0x00000000764d1fff] ACPI NVS
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x00000000764d2000-0x0000000079e7efff] reserved
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x0000000079e7f000-0x0000000079ffefff] type 20
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x0000000079fff000-0x000000007bff8fff] usable
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x000000007bff9000-0x000000007bffdfff] reserved
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x000000007bffe000-0x000000007bffffff] usable
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x000000007c000000-0x000000007cffffff] reserved
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x000000007d7f3000-0x000000007fffffff] reserved
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x00000000e0000000-0x00000000efffffff] reserved
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x00000000fd000000-0x00000000ffffffff] reserved
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x0000000100000000-0x000000205de7ffff] usable
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x000000205eec0000-0x00000020a01fffff] reserved
Feb 14 20:39:56 pve kernel: BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved
Feb 14 20:39:56 pve kernel: NX (Execute Disable) protection: active
Feb 14 20:39:56 pve kernel: efi: EFI v2.80 by American Megatrends
Feb 14 20:39:56 pve kernel: efi: ACPI=0x744d1000 ACPI 2.0=0x744d1014 TPMFinalLog=0x76488000 SMBIOS=0x794ba000 SMBIOS 3.0=0x794b9000 MEMATTR=0x65dec018 ESRT=0x699f2e98
Feb 14 20:39:56 pve kernel: secureboot: Secure boot disabled
Feb 14 20:39:56 pve kernel: SMBIOS 3.5.0 present.
Feb 14 20:39:56 pve kernel: DMI: ASUS System Product Name/TUF GAMING B650-PLUS, BIOS 1811 10/07/2023
Feb 14 20:39:56 pve kernel: tsc: Fast TSC calibration using PIT
Feb 14 20:39:56 pve kernel: tsc: Detected 4491.474 MHz processor
Feb 14 20:39:56 pve kernel: e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
Feb 14 20:39:56 pve kernel: e820: remove [mem 0x000a0000-0x000fffff] usable
Feb 14 20:39:56 pve kernel: last_pfn = 0x205de80 max_arch_pfn = 0x400000000
Feb 14 20:39:56 pve kernel: x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT 
Feb 14 20:39:56 pve kernel: e820: update [mem 0x80000000-0xffffffff] usable ==> reserved
Feb 14 20:39:56 pve kernel: last_pfn = 0x7c000 max_arch_pfn = 0x400000000
Feb 14 20:39:56 pve kernel: esrt: Reserving ESRT space from 0x00000000699f2e98 to 0x00000000699f2ed0.
Feb 14 20:39:56 pve kernel: e820: update [mem 0x699f2000-0x699f2fff] usable ==> reserved
Feb 14 20:39:56 pve kernel: Using GB pages for direct mapping
Feb 14 20:39:56 pve kernel: secureboot: Secure boot disabled
Feb 14 20:39:56 pve kernel: RAMDISK: [mem 0x30a6d000-0x3452dfff]
Feb 14 20:39:56 pve kernel: ACPI: Early table checksum verification disabled
Feb 14 20:39:56 pve kernel: ACPI: RSDP 0x00000000744D1014 000024 (v02 ALASKA)
Feb 14 20:39:56 pve kernel: ACPI: XSDT 0x00000000744D0728 0000EC (v01 ALASKA A M I    01072009 AMI  01000013)
Feb 14 20:39:56 pve kernel: ACPI: FACP 0x00000000744C6000 000114 (v06 ALASKA A M I    01072009 AMI  00010013)
Feb 14 20:39:56 pve kernel: ACPI: DSDT 0x00000000744A6000 00EAB2 (v73 ALASKA A M I    01072009 INTL 20230331)
Feb 14 20:39:56 pve kernel: ACPI: FACS 0x00000000764B8000 000040
Feb 14 20:39:56 pve kernel: ACPI: SSDT 0x00000000744C8000 007F64 (v02 AMD    Splinter 00000002 MSFT 04000000)
Feb 14 20:39:56 pve kernel: ACPI: SSDT 0x00000000744C7000 0003F1 (v02 ALASKA CPUSSDT  01072009 AMI  01072009)
Feb 14 20:39:56 pve kernel: ACPI: FIDT 0x00000000744BB000 00009C (v01 ALASKA A M I    01072009 AMI  00010013)
Feb 14 20:39:56 pve kernel: ACPI: MCFG 0x00000000744B9000 00003C (v01 ALASKA A M I    01072009 MSFT 00010013)
Feb 14 20:39:56 pve kernel: ACPI: HPET 0x00000000744B8000 000038 (v01 ALASKA A M I    01072009 AMI  00000005)
Feb 14 20:39:56 pve kernel: ACPI: WDRT 0x00000000744B7000 000047 (v01 ALASKA A M I    01072009 AMI  00000005)
Feb 14 20:39:56 pve kernel: ACPI: FPDT 0x00000000744B5000 000044 (v01 ALASKA A M I    01072009 AMI  01000013)
Feb 14 20:39:56 pve kernel: ACPI: VFCT 0x000000007449B000 00AE84 (v01 ALASKA A M I    00000001 AMD  33504F47)

 

[henriii]

You would need to provide more details in how you installed it, also why not install most recent v8.1 since this is a fresh install apparently?

Could you post your history and journalctl -b -1 into [ CODE ] [ /CODE ] tags or attach as a file here?

I tried to install the v8 version, but I had issues with specific drivers (don't remember exactly which one), and I read on forums that there was no issues using the v7...

 

[esi_y]

The system is still booting. When I restart it, everything goes normal (for how long?).

history (the name of my server is luigi)

Why not run apt full-upgrade? It will stay on v7.

Here is the journalctl -b -1

The crash occured on the Feb 14 20:04:00 (I know this thanks to my grafana / prometheus dashboard).
So I don't see relevant logs again...

So this looks like there was at least one more reboot since? You can check journalctl --list-boots and then show respective -b $numb which will include the time of crash. It's the boot prior, probably -2.

 

[henriii]

Why not run apt full-upgrade? It will stay on v7.



So this looks like there was at least one more reboot since? You can check journalctl --list-boots and then show respective -b $numb which will include the time of crash. It's the boot prior, probably -2.

apt full-upgrade --> done yesterday already

root@pve:~# journalctl --list-boots
-23 f5478212787048a4815cf7f6706e0b70 Sun 2024-02-04 17:04:17 CET—Sun 2024-02-04 17:10:24 CET
-22 0b5764aea2cf4bfc925b56a51a83463f Sun 2024-02-04 17:15:32 CET—Sun 2024-02-04 17:24:29 CET
-21 86253ba149ba4aaab08513c0b08fafff Sun 2024-02-04 17:25:02 CET—Sun 2024-02-04 17:28:50 CET
-20 6f42e92fc4e146288d79d9e48f26e878 Sun 2024-02-04 17:29:22 CET—Sun 2024-02-04 18:24:22 CET
-19 5edcb638c7dc4db687e39710dd02f78d Sun 2024-02-04 18:25:50 CET—Sun 2024-02-04 18:27:48 CET
-18 68e522d455eb41d1995e08ac866fc160 Sun 2024-02-04 18:28:38 CET—Sun 2024-02-04 18:32:01 CET
-17 ebd8b4b65e37406cad5326f1054800cb Sun 2024-02-04 18:32:52 CET—Sun 2024-02-04 20:39:01 CET
-16 9d9bbd5a1f0548d4a4fad313ea128d25 Sun 2024-02-04 20:39:40 CET—Sun 2024-02-04 20:40:26 CET
-15 31521127376740818c3276972209eb44 Mon 2024-02-05 11:53:30 CET—Mon 2024-02-05 22:53:23 CET
-14 d04b72f43b6b418ab3e573c984be61be Wed 2024-02-07 18:44:13 CET—Wed 2024-02-07 21:44:16 CET
-13 c8a333e611074c6f9e0282f6e3c62528 Thu 2024-02-08 02:17:18 CET—Sat 2024-02-10 09:25:48 CET
-12 d62f980e4736444795bfed135561bb30 Sat 2024-02-10 09:27:30 CET—Sat 2024-02-10 09:38:01 CET
-11 e1000d07b93a41a3bf43880fee92b118 Sat 2024-02-10 09:40:37 CET—Sat 2024-02-10 10:00:28 CET
-10 cef0c3eef5d4482396ae97123f1f5ef2 Sat 2024-02-10 11:20:40 CET—Sat 2024-02-10 11:53:09 CET
 -9 3841b1bb70a4401bbfa70ba64a86c8cc Sat 2024-02-10 11:53:42 CET—Sat 2024-02-10 14:47:47 CET
 -8 c69bd12722bd464bb5c6ea20579c44b9 Sat 2024-02-10 14:48:21 CET—Sat 2024-02-10 14:54:12 CET
 -7 121841fdc7764945b13a11f35a1b0953 Sat 2024-02-10 14:55:35 CET—Sat 2024-02-10 15:31:36 CET
 -6 56227b05fa57409baa71f700f3401adb Sat 2024-02-10 15:35:35 CET—Sat 2024-02-10 15:43:54 CET
 -5 84e67328f64e471b82af31d67c1be8fb Sat 2024-02-10 15:45:36 CET—Sat 2024-02-10 16:08:45 CET
 -4 a31d2b4adeb64ddcbe47fa3950564e6e Sat 2024-02-10 16:15:43 CET—Sun 2024-02-11 21:52:21 CET
 -3 c2d7df2dd7d948ea80fd076635743300 Sun 2024-02-11 21:53:01 CET—Tue 2024-02-13 21:01:13 CET
 -2 2f08738eb0614cf5bcef6c1eea58236d Tue 2024-02-13 21:35:31 CET—Wed 2024-02-14 19:59:44 CET
 -1 7407ff862e1b4153989bf3555219ea0f Wed 2024-02-14 20:39:56 CET—Wed 2024-02-14 21:29:51 CET
  0 a4d508d9840042d0b51ed68a1cccbbb5 Wed 2024-02-14 21:30:21 CET—Wed 2024-02-14 22:19:38 CET
root@pve:~# journalctl --list-boots -b $numb
-23 f5478212787048a4815cf7f6706e0b70 Sun 2024-02-04 17:04:17 CET—Sun 2024-02-04 17:10:24 CET
-22 0b5764aea2cf4bfc925b56a51a83463f Sun 2024-02-04 17:15:32 CET—Sun 2024-02-04 17:24:29 CET
-21 86253ba149ba4aaab08513c0b08fafff Sun 2024-02-04 17:25:02 CET—Sun 2024-02-04 17:28:50 CET
-20 6f42e92fc4e146288d79d9e48f26e878 Sun 2024-02-04 17:29:22 CET—Sun 2024-02-04 18:24:22 CET
-19 5edcb638c7dc4db687e39710dd02f78d Sun 2024-02-04 18:25:50 CET—Sun 2024-02-04 18:27:48 CET
-18 68e522d455eb41d1995e08ac866fc160 Sun 2024-02-04 18:28:38 CET—Sun 2024-02-04 18:32:01 CET
-17 ebd8b4b65e37406cad5326f1054800cb Sun 2024-02-04 18:32:52 CET—Sun 2024-02-04 20:39:01 CET
-16 9d9bbd5a1f0548d4a4fad313ea128d25 Sun 2024-02-04 20:39:40 CET—Sun 2024-02-04 20:40:26 CET
-15 31521127376740818c3276972209eb44 Mon 2024-02-05 11:53:30 CET—Mon 2024-02-05 22:53:23 CET
-14 d04b72f43b6b418ab3e573c984be61be Wed 2024-02-07 18:44:13 CET—Wed 2024-02-07 21:44:16 CET
-13 c8a333e611074c6f9e0282f6e3c62528 Thu 2024-02-08 02:17:18 CET—Sat 2024-02-10 09:25:48 CET
-12 d62f980e4736444795bfed135561bb30 Sat 2024-02-10 09:27:30 CET—Sat 2024-02-10 09:38:01 CET
-11 e1000d07b93a41a3bf43880fee92b118 Sat 2024-02-10 09:40:37 CET—Sat 2024-02-10 10:00:28 CET
-10 cef0c3eef5d4482396ae97123f1f5ef2 Sat 2024-02-10 11:20:40 CET—Sat 2024-02-10 11:53:09 CET
 -9 3841b1bb70a4401bbfa70ba64a86c8cc Sat 2024-02-10 11:53:42 CET—Sat 2024-02-10 14:47:47 CET
 -8 c69bd12722bd464bb5c6ea20579c44b9 Sat 2024-02-10 14:48:21 CET—Sat 2024-02-10 14:54:12 CET
 -7 121841fdc7764945b13a11f35a1b0953 Sat 2024-02-10 14:55:35 CET—Sat 2024-02-10 15:31:36 CET
 -6 56227b05fa57409baa71f700f3401adb Sat 2024-02-10 15:35:35 CET—Sat 2024-02-10 15:43:54 CET
 -5 84e67328f64e471b82af31d67c1be8fb Sat 2024-02-10 15:45:36 CET—Sat 2024-02-10 16:08:45 CET
 -4 a31d2b4adeb64ddcbe47fa3950564e6e Sat 2024-02-10 16:15:43 CET—Sun 2024-02-11 21:52:21 CET
 -3 c2d7df2dd7d948ea80fd076635743300 Sun 2024-02-11 21:53:01 CET—Tue 2024-02-13 21:01:13 CET
 -2 2f08738eb0614cf5bcef6c1eea58236d Tue 2024-02-13 21:35:31 CET—Wed 2024-02-14 19:59:44 CET
 -1 7407ff862e1b4153989bf3555219ea0f Wed 2024-02-14 20:39:56 CET—Wed 2024-02-14 21:29:51 CET
  0 a4d508d9840042d0b51ed68a1cccbbb5 Wed 2024-02-14 21:30:21 CET—Wed 2024-02-14 22:19:38 CET
root@pve:~# journalctl --list-boots -b -2
-23 f5478212787048a4815cf7f6706e0b70 Sun 2024-02-04 17:04:17 CET—Sun 2024-02-04 17:10:24 CET
-22 0b5764aea2cf4bfc925b56a51a83463f Sun 2024-02-04 17:15:32 CET—Sun 2024-02-04 17:24:29 CET
-21 86253ba149ba4aaab08513c0b08fafff Sun 2024-02-04 17:25:02 CET—Sun 2024-02-04 17:28:50 CET
-20 6f42e92fc4e146288d79d9e48f26e878 Sun 2024-02-04 17:29:22 CET—Sun 2024-02-04 18:24:22 CET
-19 5edcb638c7dc4db687e39710dd02f78d Sun 2024-02-04 18:25:50 CET—Sun 2024-02-04 18:27:48 CET
-18 68e522d455eb41d1995e08ac866fc160 Sun 2024-02-04 18:28:38 CET—Sun 2024-02-04 18:32:01 CET
-17 ebd8b4b65e37406cad5326f1054800cb Sun 2024-02-04 18:32:52 CET—Sun 2024-02-04 20:39:01 CET
-16 9d9bbd5a1f0548d4a4fad313ea128d25 Sun 2024-02-04 20:39:40 CET—Sun 2024-02-04 20:40:26 CET
-15 31521127376740818c3276972209eb44 Mon 2024-02-05 11:53:30 CET—Mon 2024-02-05 22:53:23 CET
-14 d04b72f43b6b418ab3e573c984be61be Wed 2024-02-07 18:44:13 CET—Wed 2024-02-07 21:44:16 CET
-13 c8a333e611074c6f9e0282f6e3c62528 Thu 2024-02-08 02:17:18 CET—Sat 2024-02-10 09:25:48 CET
-12 d62f980e4736444795bfed135561bb30 Sat 2024-02-10 09:27:30 CET—Sat 2024-02-10 09:38:01 CET
-11 e1000d07b93a41a3bf43880fee92b118 Sat 2024-02-10 09:40:37 CET—Sat 2024-02-10 10:00:28 CET
-10 cef0c3eef5d4482396ae97123f1f5ef2 Sat 2024-02-10 11:20:40 CET—Sat 2024-02-10 11:53:09 CET
 -9 3841b1bb70a4401bbfa70ba64a86c8cc Sat 2024-02-10 11:53:42 CET—Sat 2024-02-10 14:47:47 CET
 -8 c69bd12722bd464bb5c6ea20579c44b9 Sat 2024-02-10 14:48:21 CET—Sat 2024-02-10 14:54:12 CET
 -7 121841fdc7764945b13a11f35a1b0953 Sat 2024-02-10 14:55:35 CET—Sat 2024-02-10 15:31:36 CET
 -6 56227b05fa57409baa71f700f3401adb Sat 2024-02-10 15:35:35 CET—Sat 2024-02-10 15:43:54 CET
 -5 84e67328f64e471b82af31d67c1be8fb Sat 2024-02-10 15:45:36 CET—Sat 2024-02-10 16:08:45 CET
 -4 a31d2b4adeb64ddcbe47fa3950564e6e Sat 2024-02-10 16:15:43 CET—Sun 2024-02-11 21:52:21 CET
 -3 c2d7df2dd7d948ea80fd076635743300 Sun 2024-02-11 21:53:01 CET—Tue 2024-02-13 21:01:13 CET
 -2 2f08738eb0614cf5bcef6c1eea58236d Tue 2024-02-13 21:35:31 CET—Wed 2024-02-14 19:59:44 CET
 -1 7407ff862e1b4153989bf3555219ea0f Wed 2024-02-14 20:39:56 CET—Wed 2024-02-14 21:29:51 CET
  0 a4d508d9840042d0b51ed68a1cccbbb5 Wed 2024-02-14 21:30:21 CET—Wed 2024-02-14 22:19:38 CET

 

[esi_y]

So this looks like there was at least one more reboot since? You can check journalctl --list-boots and then show respective -b $numb which will include the time of crash. It's the boot prior, probably -2.

Say you get this from --list-boots:

 -4 a31d2b4adeb64ddcbe47fa3950564e6e Sat 2024-02-10 16:15:43 CET—Sun 2024-02-11 21:52:21 CET
 -3 c2d7df2dd7d948ea80fd076635743300 Sun 2024-02-11 21:53:01 CET—Tue 2024-02-13 21:01:13 CET
 -2 2f08738eb0614cf5bcef6c1eea58236d Tue 2024-02-13 21:35:31 CET—Wed 2024-02-14 19:59:44 CET
 -1 7407ff862e1b4153989bf3555219ea0f Wed 2024-02-14 20:39:56 CET—Wed 2024-02-14 21:29:51 CET
  0 a4d508d9840042d0b51ed68a1cccbbb5 Wed 2024-02-14 21:30:21 CET—Wed 2024-02-14 22:19:38 CET

Your crash occured 20:04 so the boot at 20:39 is past is, you crashed in the boot prior, so you can then check journalctl -b -2, please include it all, so easiest e.g. journalctl -b -2 > boot-2.txt. You may attach the file. Note you may want to sanitize the output when it comes to IPs or email addresses.

 

[henriii]

Say you get this from --list-boots:

 -4 a31d2b4adeb64ddcbe47fa3950564e6e Sat 2024-02-10 16:15:43 CET—Sun 2024-02-11 21:52:21 CET
 -3 c2d7df2dd7d948ea80fd076635743300 Sun 2024-02-11 21:53:01 CET—Tue 2024-02-13 21:01:13 CET
 -2 2f08738eb0614cf5bcef6c1eea58236d Tue 2024-02-13 21:35:31 CET—Wed 2024-02-14 19:59:44 CET
 -1 7407ff862e1b4153989bf3555219ea0f Wed 2024-02-14 20:39:56 CET—Wed 2024-02-14 21:29:51 CET
  0 a4d508d9840042d0b51ed68a1cccbbb5 Wed 2024-02-14 21:30:21 CET—Wed 2024-02-14 22:19:38 CET

Your crash occured 20:04 so the boot at 20:39 is past is, you crashed in the boot prior, so you can then check journalctl -b -2, please include it all, so easiest e.g. journalctl -b -2 > boot-2.txt. You may attach the file. Note you may want to sanitize the output when it comes to IPs or email addresses.

Here you are, after running journalctl -b -2 > boot-2.txt. Unfortunately, I don't see logs at crash time...

 

[esi_y]

I think you are having hardware/firmware/compatibility issues rather than something PVE related.

First of all, there's a newer BIOS for your system:
https://www.asus.com/motherboards-c...helpdesk_bios?model2Name=TUF-GAMING-B650-PLUS

Then, I would run memtest:
https://www.memtest86.com/download.htm

Further you might be better off searching hardware specific forums on this. Did this system run for you for some time on some other system with this hardware config or is this a new setup, etc.

 

[henriii]

I think you are having hardware/firmware/compatibility issues rather than something PVE related.

First of all, there's a newer BIOS for your system:
https://www.asus.com/motherboards-c...helpdesk_bios?model2Name=TUF-GAMING-B650-PLUS

Then, I would run memtest:
https://www.memtest86.com/download.htm

Further you might be better off searching hardware specific forums on this. Did this system run for you for some time on some other system with this hardware config or is this a new setup, etc.

damned :-(.
OK, I will proceed to the motherboard BIOS update then.

OK, for the memtest.

Do you have anything about the CPU ? is there any way to test it ?

The system is brand new. Ordered 1 month ago... :-(

 

[esi_y]

damned :-(.

It's not the first or the last time with consumer hardware. It's not there's something faulty necessarily, it's the compatibility or drivers or some device which would be best disabled, power saving features sometimes. Since this is a new system I would start with updated firmware and memtest. See from there.

OK, I will proceed to the motherboard BIOS update then.

OK, for the memtest.

You may also check compatibility list of ASUS for that memory. You can unseat/reseat the modules, you can even have it run with one at a time (it's very unlikely you got all 4 modules bad). Since this is a new system, I would actually forget about PVE for the time being and instead e.g. install Ubuntu Desktop (PVE uses their kernels anyhow) and see how it fares there. You can run some load tests, run virt-manager, etc.

Do you have anything about the CPU ? is there any way to test it ?

I personally have no experience with this one. Sometimes it's interesting to disable cstates and see if that has not resolved the issue by miracle. Generic tips you would find around. You are much more likely to find someone with similar issue on e.g. Ubuntu forum if it was a hardware problem or compatibility thing, it's just very likely others tried to run it on the hardware and ran into the same. Then extrapolate for PVE.

The system is brand new. Ordered 1 month ago... :-(

Some people if facing this situation would do a return, if possible. The time then comes to do full research and re-order different set of components for instance. If that's not possible, you have to find out what's wrong. Worst case you RMA a bad component or have to sell off perfectly good but incompatible ones.

NB I noticed some rather high temperatures on the /dev/sdd, but nothing before your kernel panics. Maybe you were shredding it with something knowingly, I do not think it's the issue. But be aware also btw:
https://www.reddit.com/r/buildapc/comments/1857wpw/samsung_990_pro_firmware_0b2qjxg7_is_it_okay/

I don't think it's related to your freezes, but it will save you TBWs on those 990s to have the latest one.

 

[esi_y]

Oh and one more thing - if you edit the thread title to include your hardware (MB + CPU) I suspect you run much higher chance of attracting someone's attention who runs the same and has been through this if a known issue.