Strange error with my proxmox server

[-RYknow]

Not really sure why I'm suddenly getting this error. It reads;

do_IRQ: 4.34 No irq handler for vector

I get this error 3-4 times per minute. We ended up having a power outage yesterday, but my server (dell R610) is on a battery backup. I was home when the power went out, and I went down into my basement and just shut the machine down (shutdown command). When I rebooted the machine, I started getting these errors.

I read through this link, and followed the instructions to add the line "pci=nomsi,noaer", which I've done. I updated grub as suggested, but the problem continues. The machine has been flawless since I got it setup about a year ago. The only physical change to the machine was about a month ago I added 32gb of ram. Since adding the ram the machine had been working fine and without errors.

Any thoughts?
-RYknow

 

[ulf.kosack]

Hi RYknow,

you're not alone. After upgrade from pve 4.X to 5.2 I've the same errors on console:

Message from syslogd@XXXXXXXS006 at Jun  2 10:18:27 ...
 kernel:[ 1363.829234] do_IRQ: 2.39 No irq handler for vector

Message from syslogd@XXXXXXXS006 at Jun  2 10:18:37 ...
 kernel:[ 1374.001308] do_IRQ: 2.39 No irq handler for vector

Message from syslogd@XXXXXXXS006 at Jun  2 10:18:57 ...
 kernel:[ 1394.181973] do_IRQ: 2.39 No irq handler for vector

Message from syslogd@XXXXXXXS006 at Jun  2 10:19:52 ...
 kernel:[ 1448.886861] do_IRQ: 2.39 No irq handler for vector

Message from syslogd@XXXXXXXS006 at Jun  2 10:20:22 ...
 kernel:[ 1479.241896] do_IRQ: 2.39 No irq handler for vector

Message from syslogd@XXXXXXXS006 at Jun  2 10:20:25 ...
 kernel:[ 1482.244370] do_IRQ: 2.39 No irq handler for vector

My versions:

proxmox-ve: 5.2-2 (running kernel: 4.15.17-1-pve)
pve-manager: 5.2-1 (running version: 5.2-1/0fcd7879)
pve-kernel-4.15: 5.2-1
pve-kernel-4.15.17-1-pve: 4.15.17-9
pve-kernel-4.4.117-2-pve: 4.4.117-110
pve-kernel-4.4.98-3-pve: 4.4.98-103
pve-kernel-4.4.95-1-pve: 4.4.95-99
pve-kernel-4.4.76-1-pve: 4.4.76-94
pve-kernel-4.4.67-1-pve: 4.4.67-92
pve-kernel-4.4.35-2-pve: 4.4.35-79
pve-kernel-4.4.35-1-pve: 4.4.35-77
pve-kernel-4.4.21-1-pve: 4.4.21-71
pve-kernel-4.4.19-1-pve: 4.4.19-66
pve-kernel-4.4.16-1-pve: 4.4.16-64
pve-kernel-4.4.13-2-pve: 4.4.13-58
pve-kernel-4.4.6-1-pve: 4.4.6-48
corosync: 2.4.2-pve5
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.0-8
libpve-apiclient-perl: 2.0-4
libpve-common-perl: 5.0-31
libpve-guest-common-perl: 2.0-16
libpve-http-server-perl: 2.0-8
libpve-storage-perl: 5.0-23
libqb0: 1.0.1-1
lvm2: 2.02.168-pve6
lxc-pve: 3.0.0-3
lxcfs: 3.0.0-1
novnc-pve: 0.6-4
proxmox-widget-toolkit: 1.0-18
pve-cluster: 5.0-27
pve-container: 2.0-23
pve-docs: 5.2-4
pve-firewall: 3.0-9
pve-firmware: 2.0-4
pve-ha-manager: 2.0-5
pve-i18n: 1.0-5
pve-libspice-server1: 0.12.8-3
pve-qemu-kvm: 2.11.1-5
pve-xtermjs: 1.0-5
qemu-server: 5.0-26
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.8-pve1~bpo9

Anybody out there to?

Thanks
Ulf

 

[ulf.kosack]

To inspect the error I've configured temporary rsyslog, because you cannot not search, write, if every 20-60 seconds two messages are shown in the console.

I've created a file /etc/rsyslog.d/NoIrqHandler.conf

if $msg contains 'do_IRQ: 2.39 No irq handler for vector' then /var/log/noIrqHandler.log
& stop

and restarted rsyslog. After that the messages are still there, but not on the console. Only in /var/log/noIrqHandler.log

 

[ulf.kosack]

The often found solution to extend the GRUB_CMDLINE_LINUX doesn't helps.

Others have the same problem with kernel 4.15: NVIDIA 396.* driver drops support for Fermi-based GPUs

I had to adjust the above rsyslog configuration, because after restart with changed GRUB_CMDLINE_LINUX the IRQ number is changed.

Perhaps it is something with the graphics driver, my card is

01:03.0 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200eW WPCM450 (rev 0a)

on a Supermicro X8DT3-F.

And the rsyslog configuration only helps for a ssh session, not for the local console session :-(

Perhaps this can help the Proxmox staff: [Linux] “0.107 No irq handler for vector” aka “Problems with my PCIe Port”

 

[ulf.kosack]

We are really not alone: https://bugzilla.redhat.com/show_bug.cgi?id=1551605

 

[-RYknow]

It's interesting, it was suggested (via the homelab subreddit) to update all system BIOS and firmware. While this didn't fix the issue, the frequency in which the error is thrown has dropped dramatically. Originally I was seeing the error 3-4 times per minute (extremely frustrating while trying to edit files via vim, as the info would populate there 3-4 times per minute). Since doing all my server upgrades, the error is now being thrown maybe 20-25 times per day. It's dropped dramatically, but I'm hoping for a fix.

I'll verify tonight what video adapter my R610 is running.

EDIT: Just looked back and noticed the frequency that the error is showing up on your server. It's incredibly frustrating... :-(

-RYknow

 

[Beat Arnet]

Not sure if this is related, but when I install apcupsd on my fresh 5.2-5 install, I get this message when the apcupsd service is running.

kernel:[ 2065.534899] do_IRQ: 3.36 No irq handler for vector

 

[ulf.kosack]

I've installed apcupsd too. I will check this next week. I'm on the way to short holidays. Thanks

 

[-RYknow]

Not sure if this is related, but when I install apcupsd on my fresh 5.2-5 install, I get this message when the apcupsd service is running.

That interesting. I'll check on this tonight when I get home. I believe I did install apcupsd. Thanks for the input.

-RYknow

 

[joulester]

Hi did you guys solve this? I get this message when I try to install apcupsd.

 

[-RYknow]

Hi did you guys solve this? I get this message when I try to install apcupsd.

I forgot to reply... and it's been awhile. I want to say that I did NOT have apcupsd installed, and I'm still getting the error. I'll double check tonight and try to remember to respond.

-RYknow

 

[thenightfighter]

To inspect the error I've configured temporary rsyslog, because you cannot not search, write, if every 20-60 seconds two messages are shown in the console.

I've created a file /etc/rsyslog.d/NoIrqHandler.conf

if $msg contains 'do_IRQ: 2.39 No irq handler for vector' then /var/log/noIrqHandler.log
& stop

and restarted rsyslog. After that the messages are still there, but not on the console. Only in /var/log/noIrqHandler.log

Works fine... Thanks!!

And I could say on my end it was apcupsd which caused the problem... A simple systemctl stop apcupsd.service and systemctl disable apcupsd.service and things are done... No more crap which fires into mi vi session

 

[VladVons]

This error causes kernel 4.15 with apcupsd. (ProxMox 5.2)
Older kernel 4.13 works good. (ProxMox 5.1)

GRUB_CMDLINE_LINUX = "pci=nomsi,noaer"
This strongly slowes my console. So doesnt work

I didnt solve problem with 4.15, so i downgraded to 4.13
On kernel 4.17.7 x86_64 messages are still present....

 

[joulester]

Hi I solved this with installing apcupsd, stop the service, setup the settings for apcupsd and then starting it again. I don´t get the messages any more once it is configured.

 

[VladVons]

Yes it works.

I installed "apcupsd" on another computer, configured it and copied ready "conf" file to problematic target.
Then restarted.
Hope in future this bug will be pached in kernel

 

[tofi]

Hello,

i've solved this error by replacing the lines

UPSTYPE usb
DEVICE /dev/ttyS0

by
UPSTYPE usb
DEVICE

in /etc/apcupsd/apcupsd.conf

For what reason ever, it's configured with usb and non-blank-device by default, which is not working.

 

[-RYknow]

Hello,

i've solved this error by replacing the lines

UPSTYPE usb
DEVICE /dev/ttyS0

by
UPSTYPE usb
DEVICE

in /etc/apcupsd/apcupsd.conf

For what reason ever, it's configured with usb and non-blank-device by default, which is not working.

That's good to know. Thanks for reporting in. I had just written it off and disabled apcupsd. What version of Proxmox are you running?

-RYknow

 

[tofi]

What version of Proxmox are you running?

Linux 4.15.18-24-pve #1 SMP PVE 4.15.18-52 (Thu, 05 Dec 2019 10:14:17 +0100)

pve-manager/5.4-13/aee6f0ec

Glad to help