SNMP error

[hellstorm]

Hi! I have this error after every start. After restart it became normal but after 24 hours it appears again.
Snmp service locks, And I could not start it agait.
Its error with hardware or vm or LXC or something else
In messages get I got:
May 19 09:54:49 pve9 kernel: [126148.397501] ? _cond_resched+0x19/0x30
May 19 09:54:49 pve9 kernel: [126148.397521] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000013
May 19 09:56:50 pve9 kernel: [126269.227241] schedule_timeout+0x205/0x300
May 19 09:56:50 pve9 kernel: [126269.227263] cancel_delayed_work_sync+0x13/0x20
May 19 09:56:50 pve9 kernel: [126269.227285] do_filp_open+0x93/0x100
May 19 09:56:50 pve9 kernel: [126269.227308] RAX: ffffffffffffffda RBX: 0000000000000013 RCX: 00007f8c76e17c8b
May 19 09:58:51 pve9 kernel: [126390.056952] ? ttwu_do_activate+0x5a/0x70
May 19 09:58:51 pve9 kernel: [126390.056961] ? work_busy+0x90/0x90
May 19 09:58:51 pve9 kernel: [126390.056977] blkdev_get+0xe0/0x140
May 19 09:58:51 pve9 kernel: [126390.056992] do_filp_open+0x93/0x100
May 19 09:58:51 pve9 kernel: [126390.057009] RIP: 0033:0x7f8c76e17c8b
May 19 10:00:52 pve9 kernel: [126510.886820] schedule+0x33/0xa0
May 19 10:00:52 pve9 kernel: [126510.886852] cancel_delayed_work_sync+0x13/0x20
May 19 10:00:52 pve9 kernel: [126510.886895] do_filp_open+0x93/0x100
May 19 10:00:52 pve9 kernel: [126510.886926] RAX: ffffffffffffffda RBX: 0000000000000013 RCX: 00007f8c76e17c8b
May 19 10:02:52 pve9 kernel: [126631.716484] Call Trace:
May 19 10:02:52 pve9 kernel: [126631.716514] ? exact_lock+0x11/0x20
May 19 10:02:52 pve9 kernel: [126631.716538] ? __schedule+0x2ee/0x700
May 19 10:02:52 pve9 kernel: [126631.716565] Code: Bad RIP value.
May 19 10:04:53 pve9 kernel: [126752.550230] schedule_timeout+0x205/0x300
May 19 10:04:53 pve9 kernel: [126752.550254] disk_block_events+0x78/0x80
May 19 10:04:53 pve9 kernel: [126752.550278] ? __alloc_fd+0x46/0x150
May 19 10:04:53 pve9 kernel: [126752.550300] RDX: 0000000000000800 RSI: 00007ffc93bf59a0 RDI: 00000000ffffff9c
May 19 10:06:54 pve9 kernel: [126873.375954] schedule_timeout+0x205/0x300
May 19 10:06:54 pve9 kernel: [126873.375977] cancel_delayed_work_sync+0x13/0x20
May 19 10:06:54 pve9 kernel: [126873.376000] do_filp_open+0x93/0x100
May 19 10:06:54 pve9 kernel: [126873.376023] RAX: ffffffffffffffda RBX: 0000000000000013 RCX: 00007f8c76e17c8b
May 19 10:08:55 pve9 kernel: [126994.205666] Call Trace:
May 19 10:08:55 pve9 kernel: [126994.205692] ? work_busy+0x90/0x90
May 19 10:08:55 pve9 kernel: [126994.205714] do_dentry_open+0x143/0x3a0
May 19 10:08:55 pve9 kernel: [126994.205732] __x64_sys_openat+0x20/0x30
May 19 10:08:55 pve9 kernel: [126994.205750] R13: 00007f8c774712e0 R14: 0000000000000010 R15: 0000000000060012
May 19 10:10:56 pve9 kernel: [127115.035403] ? wake_up_q+0x80/0x80
May 19 10:10:56 pve9 kernel: [127115.035424] __blkdev_get+0x72/0x560
May 19 10:10:56 pve9 kernel: [127115.035445] do_sys_open+0x177/0x280
May 19 10:10:56 pve9 kernel: [127115.035467] RBP: 00007ffc93bf59a0 R08: 0000000000000000 R09: 00007f8c76a44e80
========
service snmpd status:

snmpd.service - Simple Network Management Protocol (SNMP) Daemon.
Loaded: loaded (/lib/systemd/system/snmpd.service; enabled; vendor preset: enabled)
Active: active (running) since Sun 2020-05-17 22:52:43 MSK; 1 day 12h ago
Process: 3021 ExecStartPre=/bin/mkdir -p /var/run/agentx (code=exited, status=0/SUCCESS)
Main PID: 3041 (snmpd)
Tasks: 1 (limit: 4915)
Memory: 45.6M
CGroup: /system.slice/snmpd.service
└─3041 /usr/sbin/snmpd -Lsd -Lf /dev/null -u Debian-snmp -g Debian-snmp -I -smux mteTrigger mteTriggerConf -f -p /run/snmpd.pid

May 18 21:49:56 pve9 snmpd[3041]: inetNetToMediaTable:_add_or_update_arpentry: unsupported address type, len = 0
May 18 21:49:56 pve9 snmpd[3041]: inetNetToMediaTable:_add_or_update_arpentry: unsupported address type, len = 0
May 18 21:50:02 pve9 snmpd[3041]: inetNetToMediaTable:_add_or_update_arpentry: unsupported address type, len = 0
May 18 21:50:02 pve9 snmpd[3041]: inetNetToMediaTable:_add_or_update_arpentry: unsupported address type, len = 0
May 19 03:50:00 pve9 snmpd[3041]: inetNetToMediaTable:_add_or_update_arpentry: unsupported address type, len = 0
May 19 03:50:00 pve9 snmpd[3041]: inetNetToMediaTable:_add_or_update_arpentry: unsupported address type, len = 0
May 19 03:50:00 pve9 snmpd[3041]: inetNetToMediaTable:_add_or_update_arpentry: unsupported address type, len = 0
May 19 03:50:00 pve9 snmpd[3041]: inetNetToMediaTable:_add_or_update_arpentry: unsupported address type, len = 0
May 19 03:50:00 pve9 snmpd[3041]: inetNetToMediaTable:_add_or_update_arpentry: unsupported address type, len = 0
May 19 03:50:00 pve9 snmpd[3041]: inetNetToMediaTable:_add_or_update_arpentry: unsupported address type, len = 0

Today I got the same error
May 20 03:13:47 pve9 kernel: [51354.356792] __flush_work+0x131/0x1e0
May 20 03:13:47 pve9 kernel: [51354.356809] ? blkdev_get_by_dev+0x50/0x50
May 20 03:13:47 pve9 kernel: [51354.356826] ? _cond_resched+0x19/0x30
May 20 03:13:47 pve9 kernel: [51354.356854] RBP: 00007ffdb663cb50 R08: 0000000000000000 R09: 00007fa3d06d0e80
May 20 03:15:48 pve9 kernel: [51475.182404] Tainted: P OE 5.4.34-1-pve #1
May 20 03:15:48 pve9 kernel: [51475.182495] Call Trace:
May 20 03:15:48 pve9 kernel: [51475.182552] __blkdev_get+0x72/0x560
May 20 03:15:48 pve9 kernel: [51475.182561] ? blkdev_get_by_dev+0x50/0x50
May 20 03:17:49 pve9 kernel: [51596.012192] schedule+0x33/0xa0
May 20 03:17:49 pve9 kernel: [51596.012211] ? kobj_lookup+0xec/0x160
May 20 03:17:49 pve9 kernel: [51596.012228] ? __schedule+0x2ee/0x700
May 20 03:17:49 pve9 kernel: [51596.012246] RIP: 0033:0x7fa3d0aa3c8b
May 20 03:19:50 pve9 kernel: [51716.841840] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
May 20 03:19:50 pve9 kernel: [51716.841883] __flush_work+0x131/0x1e0
May 20 03:19:50 pve9 kernel: [51716.841900] ? blkdev_get_by_dev+0x50/0x50
May 20 03:19:50 pve9 kernel: [51716.841916] ? _cond_resched+0x19/0x30
May 20 03:19:50 pve9 kernel: [51716.841933] RBP: 00007ffdb663cb50 R08: 0000000000000000 R09: 00007fa3d06d0e80
May 20 03:21:50 pve9 kernel: [51837.671652] schedule_timeout+0x205/0x300
May 20 03:21:50 pve9 kernel: [51837.671672] disk_block_events+0x78/0x80
May 20 03:21:50 pve9 kernel: [51837.671692] ? __alloc_fd+0x46/0x150
May 20 03:21:50 pve9 kernel: [51837.671710] RDX: 0000000000000800 RSI: 00007ffdb663cb50 RDI: 00000000ffffff9c
May 20 03:23:51 pve9 kernel: [51958.501374] __flush_work+0x131/0x1e0
May 20 03:23:51 pve9 kernel: [51958.501392] blkdev_open+0x87/0xa0
May 20 03:23:51 pve9 kernel: [51958.501410] __x64_sys_openat+0x20/0x30
May 20 03:25:52 pve9 kernel: [52079.331061] do_filp_open+0x93/0x100
May 20 03:25:52 pve9 kernel: [52079.331082] RAX: ffffffffffffffda RBX: 0000000000000013 RCX: 00007fa3d0aa3c8b
May 20 03:27:53 pve9 kernel: [52200.160755] Call Trace:
May 20 03:27:53 pve9 kernel: [52200.160780] ? exact_lock+0x11/0x20
May 20 03:27:53 pve9 kernel: [52200.160799] ? __schedule+0x2ee/0x700
May 20 03:27:53 pve9 kernel: [52200.160820] Code: Bad RIP value.
May 20 03:29:54 pve9 kernel: [52320.990504] schedule+0x33/0xa0
May 20 03:29:54 pve9 kernel: [52320.990523] ? kobj_lookup+0xec/0x160
May 20 03:29:54 pve9 kernel: [52320.990542] ? put_device+0x13/0x20
May 20 03:29:54 pve9 kernel: [52320.990563] RSP: 002b:00007ffdb663cac0 EFLAGS: 00000246 ORIG_RAX: 0000000000000101
May 20 03:31:55 pve9 kernel: [52441.824152] snmpd D 0 2543 1 0x00000000
May 20 03:31:55 pve9 kernel: [52441.824177] __cancel_work_timer+0x115/0x190
May 20 03:31:55 pve9 kernel: [52441.824197] path_openat+0x2e9/0x16f0
May 20 03:31:55 pve9 kernel: [52441.824214] entry_SYSCALL_64_after_hwframe+0x44/0xa9

Zabbix from the zabbix-agent shows nothing.
No nfs share is mounted.

FS I use for LXC and VM is zfs.
Thanx for answers

 

[Stoiko Ivanov]

The posted logs lack some valuable information - check the journal and paste the complete trace and a few lines above an below.

I would guess that something does not finish fast enough based on:

echo 0 > /proc/sys/kernel/hung_task_timeout_secs"

and that it might be related to your disks based on:

May 20 03:13:47 pve9 kernel: [51354.356809] ? blkdev_get_by_dev+0x50/0x50

I hope this helps!