PVE 8.2.2: After Update and reboot NMI: PCI system error

[Adi M]

Running Proxmox PVE 8.2.2.
After Update and reboot I get error "NMI: PCI system error (SERR) for reason b1 on CPU 0." on server HP ProLiant DL380 G6.

What I see: the first 3 of 8 hdd is ZFS-Storage for operationsystem and only one or two led is active. Until grup all Leds ar active

In the post of https://forum.proxmox.com/threads/kernel-crashes-on-dl360.26066/ there is something like

intel_idle.max_cstate=0

I there a way to edit /etc/default/grub and update-grub by rescue access?

 

[MRoman]

I got same problem on HP ProLiant DL380 G6 with proxmox 8.2.2.

On HP ProLiant DL380 G7, HP ProLiant DL380 G8 got ext4 file system problems, it could not be mounted as RW, only RO, I can't login via SSH, but it's ping.

EXT4-fs error (device dm-2) in ext4_mb_clear_bb:6534: error 95
EXT4-fs (dm-2): Remounting filesystem read-only

Workaround to make nodes alive is to choose another linux kernel like: 6.5.13-5-pve
It is seems to me 6.8.4-2-pve is broken for old hardware.

 

[Adi M]

At the moment, I am not possible to change the kernel because the GRUB is not detecting usb keyboard

 

[MRoman]

For manipulations with servers I used HP iLo.
Also found info how to downgrade kernel https://forum.proxmox.com/threads/revert-to-prior-kernel.100310/post-433030
UEFI?

$ proxmox-boot-tool kernel list
$ proxmox-boot-tool kernel add|remove <kernel-version>
$ proxmox-boot-tool refresh

Legacy https://forum.proxmox.com/threads/revert-to-prior-kernel.100310/post-434580

 

[alessiobravi]

Hi all,

same problem here, last usable kernel found in repositories is 6.5.13-5-pve (at date), all nodes in this dev cluster have been forced to boot with such kernel via GRUB_DEFAULT - proxmox-boot-tool also used to manually keep the kernel version bootable.

Even if I know such boxes are old, in a test dev they still have their value, hope to find a solution that will let me upgrade PVE also with new kernels soon

Has anyone got a better solution in the meanwhile?


Ciao!

 

[Nicolas Simond]

Hello, same problem here on a MicroServer gen8.
Reverted to 6.5 branch since the very latest kernel put the server in a bootloop.

Old boxes doesn't only have value for test dev. At home, many people use proxmox and a lot of us run this on old hardware.

 

[artnet]

The solution is to disable iommu. Edit /etc/default/grub and add or modify line GRUB_CMDLINE_LINUX_DEFAULT to:
GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=off"
Run update-grub
You can also edit grub parameters before boot by pressing "e" in grub menu.

 

[linuxgeek1035]

The solution is to disable iommu. Edit /etc/default/grub and add or modify line GRUB_CMDLINE_LINUX_DEFAULT to:
GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=off"
Run update-grub
You can also edit grub parameters before boot by pressing "e" in grub menu.

I wanted to try the above suggestion by editing the grub parameter during bootup, but did not find this line "GRUB_CMDLINE_LINUX_DEFAULT", do I have to add it? Thanks.

 

[artnet]

I wanted to try the above suggestion by editing the grub parameter during bootup, but did not find this line "GRUB_CMDLINE_LINUX_DEFAULT", do I have to add it? Thanks.

If you are editing during boot, just add intel_iommu=off at the end of the linux line.
Example:
linux /boot/vmlinuz-6.9-x86_64 root=UUID=beb6d712-c5f1-4a32-b7fc-f60ab43d216f rw quiet splash intel_iommu=off

 

[linuxgeek1035]

Thank you very much, it works. Awesome.