Hello all,
I was getting some warnings that my local storage was filling up and that seemed very odd to me so I investigated. Turns out my node that has only been running a few weeks had a 22G syslog and 12G messages file. There seems to be lots of log spam about devices I have passed through (a USB Conbee II stick and something with one of the pic devices. Logs below. Would love any advice!
lspci
The Nvidia GPU, Intel iGPU and LSI are passed through to VMs
lsusb
tail -n 100 syslog
I was getting some warnings that my local storage was filling up and that seemed very odd to me so I investigated. Turns out my node that has only been running a few weeks had a 22G syslog and 12G messages file. There seems to be lots of log spam about devices I have passed through (a USB Conbee II stick and something with one of the pic devices. Logs below. Would love any advice!
lspci
Code:
00:00.0 Host bridge: Intel Corporation Device a700 (rev 01)
00:01.0 PCI bridge: Intel Corporation Device a70d (rev 01)
00:01.1 PCI bridge: Intel Corporation Device a72d (rev 01)
00:02.0 VGA compatible controller: Intel Corporation Device a780 (rev 04)
00:14.0 USB controller: Intel Corporation Device 7ae0 (rev 11)
00:14.2 RAM memory: Intel Corporation Device 7aa7 (rev 11)
00:14.3 Network controller: Intel Corporation Device 7af0 (rev 11)
00:15.0 Serial bus controller [0c80]: Intel Corporation Device 7acc (rev 11)
00:15.1 Serial bus controller [0c80]: Intel Corporation Device 7acd (rev 11)
00:15.2 Serial bus controller [0c80]: Intel Corporation Device 7ace (rev 11)
00:15.3 Serial bus controller [0c80]: Intel Corporation Device 7acf (rev 11)
00:16.0 Communication controller: Intel Corporation Device 7ae8 (rev 11)
00:17.0 SATA controller: Intel Corporation Device 7ae2 (rev 11)
00:19.0 Serial bus controller [0c80]: Intel Corporation Device 7afc (rev 11)
00:19.1 Serial bus controller [0c80]: Intel Corporation Device 7afd (rev 11)
00:1c.0 PCI bridge: Intel Corporation Device 7ab8 (rev 11)
00:1c.2 PCI bridge: Intel Corporation Device 7aba (rev 11)
00:1c.4 PCI bridge: Intel Corporation Device 7abc (rev 11)
00:1d.0 PCI bridge: Intel Corporation Device 7ab0 (rev 11)
00:1f.0 ISA bridge: Intel Corporation Device 7a84 (rev 11)
00:1f.3 Audio device: Intel Corporation Device 7ad0 (rev 11)
00:1f.4 SMBus: Intel Corporation Device 7aa3 (rev 11)
00:1f.5 Serial bus controller [0c80]: Intel Corporation Device 7aa4 (rev 11)
01:00.0 VGA compatible controller: NVIDIA Corporation GA102 [GeForce RTX 3090] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GA102 High Definition Audio Controller (rev a1)
02:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS2308 PCI-Express Fusion-MPT SAS-2 (rev 05)
03:00.0 Ethernet controller: Aquantia Corp. Device 14c0 (rev 03)
04:00.0 Ethernet controller: Intel Corporation Ethernet Controller I225-V (rev 03)
05:00.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
06:00.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
06:01.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
06:02.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
06:03.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
07:00.0 USB controller: Intel Corporation Thunderbolt 4 NHI [Maple Ridge 4C 2020]
3e:00.0 USB controller: Intel Corporation Thunderbolt 4 USB Controller [Maple Ridge 4C 2020]
75:00.0 Ethernet controller: Mellanox Technologies MT27500 Family [ConnectX-3]
The Nvidia GPU, Intel iGPU and LSI are passed through to VMs
lsusb
Code:
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 002: ID 0bda:0411 Realtek Semiconductor Corp. Hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 008: ID 1b1c:0c10 Corsair Commander PRO
Bus 001 Device 009: ID 0781:5571 SanDisk Corp. Cruzer Fit
Bus 001 Device 007: ID 0bda:5411 Realtek Semiconductor Corp. RTS5411 Hub
Bus 001 Device 063: ID 1cf1:0030 Dresden Elektronik ZigBee gateway [ConBee II]
Bus 001 Device 002: ID 0414:a00b Giga-Byte Technology Co., Ltd USB Audio
Bus 001 Device 012: ID 8087:0033 Intel Corp.
Bus 001 Device 013: ID 048d:5702 Integrated Technology Express, Inc. ITE Device
Bus 001 Device 011: ID 05e3:0608 Genesys Logic, Inc. Hub
Bus 001 Device 010: ID 05e3:0608 Genesys Logic, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
tail -n 100 syslog
Code:
Dec 14 23:01:24 pve-nas pvedaemon[2818]: starting 1 worker(s)
Dec 14 23:01:24 pve-nas pvedaemon[2818]: worker 4023818 started
Dec 14 23:01:35 pve-nas kernel: [82817.867449] pci_bus 0000:06: Allocating resources
Dec 14 23:01:54 pve-nas kernel: [82836.962628] pci_bus 0000:06: Allocating resources
Dec 14 23:01:54 pve-nas pmxcfs[2605]: [status] notice: received log
Dec 14 23:02:13 pve-nas kernel: [82855.881841] pci_bus 0000:06: Allocating resources
Dec 14 23:02:30 pve-nas pmxcfs[2605]: [status] notice: received log
Dec 14 23:02:32 pve-nas kernel: [82875.061046] pci_bus 0000:06: Allocating resources
Dec 14 23:02:47 pve-nas systemd[1]: session-56.scope: Succeeded.
Dec 14 23:02:47 pve-nas pmxcfs[2605]: [status] notice: received log
Dec 14 23:02:48 pve-nas pmxcfs[2605]: [status] notice: received log
Dec 14 23:02:48 pve-nas systemd[1]: Started Session 57 of user root.
Dec 14 23:02:51 pve-nas kernel: [82893.924291] pci_bus 0000:06: Allocating resources
Dec 14 23:03:05 pve-nas pmxcfs[2605]: [status] notice: received log
Dec 14 23:03:05 pve-nas pmxcfs[2605]: [status] notice: received log
Dec 14 23:03:06 pve-nas systemd[1]: Started Session 58 of user root.
Dec 14 23:03:10 pve-nas kernel: [82913.155400] pci_bus 0000:06: Allocating resources
Dec 14 23:03:27 pve-nas kernel: [82930.598688] pci_bus 0000:06: Allocating resources
Dec 14 23:03:46 pve-nas kernel: [82949.657885] pci_bus 0000:06: Allocating resources
Dec 14 23:04:05 pve-nas kernel: [82968.725084] pci_bus 0000:06: Allocating resources
Dec 14 23:04:25 pve-nas kernel: [82987.896302] pci_bus 0000:06: Allocating resources
Dec 14 23:04:42 pve-nas kernel: [83005.843536] pci_bus 0000:06: Allocating resources
Dec 14 23:05:02 pve-nas kernel: [83024.962861] pci_bus 0000:06: Allocating resources
Dec 14 23:05:03 pve-nas pveproxy[3922434]: worker exit
Dec 14 23:05:03 pve-nas pveproxy[2827]: worker 3922434 finished
Dec 14 23:05:03 pve-nas pveproxy[2827]: starting 1 worker(s)
Dec 14 23:05:03 pve-nas pveproxy[2827]: worker 4036362 started
Dec 14 23:05:21 pve-nas kernel: [83044.041987] pci_bus 0000:06: Allocating resources
Dec 14 23:05:40 pve-nas kernel: [83063.141226] pci_bus 0000:06: Allocating resources
Dec 14 23:05:58 pve-nas kernel: [83081.660394] pci_bus 0000:06: Allocating resources
Dec 14 23:06:11 pve-nas systemd[1]: session-58.scope: Succeeded.
Dec 14 23:06:11 pve-nas systemd[1]: session-58.scope: Consumed 1.619s CPU time.
Dec 14 23:06:11 pve-nas pmxcfs[2605]: [status] notice: received log
Dec 14 23:06:17 pve-nas kernel: [83100.827593] pci_bus 0000:06: Allocating resources
Dec 14 23:06:34 pve-nas kernel: [83117.690898] pci_bus 0000:06: Allocating resources
Dec 14 23:06:52 pve-nas kernel: [83135.226226] pci_bus 0000:06: Allocating resources
Dec 14 23:07:11 pve-nas kernel: [83154.389359] pci_bus 0000:06: Allocating resources
Dec 14 23:07:30 pve-nas kernel: [83173.568562] pci_bus 0000:06: Allocating resources
Dec 14 23:07:48 pve-nas systemd[1]: session-57.scope: Succeeded.
Dec 14 23:07:48 pve-nas pmxcfs[2605]: [status] notice: received log
Dec 14 23:07:49 pve-nas pmxcfs[2605]: [status] notice: received log
Dec 14 23:07:49 pve-nas kernel: [83191.963798] pci_bus 0000:06: Allocating resources
Dec 14 23:07:49 pve-nas systemd[1]: Started Session 59 of user root.
Dec 14 23:08:08 pve-nas kernel: [83211.083003] pci_bus 0000:06: Allocating resources
Dec 14 23:08:27 pve-nas kernel: [83230.254207] pci_bus 0000:06: Allocating resources
Dec 14 23:08:46 pve-nas kernel: [83249.393476] pci_bus 0000:06: Allocating resources
Dec 14 23:09:05 pve-nas kernel: [83268.556654] pci_bus 0000:06: Allocating resources
Dec 14 23:09:23 pve-nas kernel: [83285.979961] pci_bus 0000:06: Allocating resources
Dec 14 23:09:42 pve-nas kernel: [83305.015218] pci_bus 0000:06: Allocating resources
Dec 14 23:09:59 pve-nas kernel: [83322.018460] pci_bus 0000:06: Allocating resources
Dec 14 23:10:14 pve-nas kernel: [83337.689865] pci_bus 0000:06: Allocating resources
Dec 14 23:10:34 pve-nas kernel: [83356.860981] pci_bus 0000:06: Allocating resources
Dec 14 23:10:53 pve-nas kernel: [83376.084144] pci_bus 0000:06: Allocating resources
Dec 14 23:11:09 pve-nas pmxcfs[2605]: [status] notice: received log
Dec 14 23:11:09 pve-nas pmxcfs[2605]: [status] notice: received log
Dec 14 23:11:10 pve-nas systemd[1]: Started Session 60 of user root.
Dec 14 23:11:12 pve-nas kernel: [83395.047387] pci_bus 0000:06: Allocating resources
Dec 14 23:11:31 pve-nas kernel: [83414.270590] pci_bus 0000:06: Allocating resources
Dec 14 23:11:50 pve-nas kernel: [83433.449755] pci_bus 0000:06: Allocating resources
Dec 14 23:12:09 pve-nas kernel: [83452.616950] pci_bus 0000:06: Allocating resources
Dec 14 23:12:28 pve-nas kernel: [83471.788230] pci_bus 0000:06: Allocating resources
Dec 14 23:12:37 pve-nas pmxcfs[2605]: [status] notice: received log
Dec 14 23:12:48 pve-nas kernel: [83490.895338] pci_bus 0000:06: Allocating resources
Dec 14 23:12:49 pve-nas systemd[1]: session-59.scope: Succeeded.
Dec 14 23:12:49 pve-nas pmxcfs[2605]: [status] notice: received log
Dec 14 23:12:50 pve-nas pmxcfs[2605]: [status] notice: received log
Dec 14 23:12:50 pve-nas systemd[1]: Started Session 61 of user root.
Dec 14 23:13:06 pve-nas kernel: [83509.666594] pci_bus 0000:06: Allocating resources
Dec 14 23:13:25 pve-nas kernel: [83528.457834] pci_bus 0000:06: Allocating resources
Dec 14 23:13:43 pve-nas kernel: [83545.945129] pci_bus 0000:06: Allocating resources
Dec 14 23:14:02 pve-nas kernel: [83565.176277] pci_bus 0000:06: Allocating resources
Dec 14 23:14:20 pve-nas kernel: [83583.779515] pci_bus 0000:06: Allocating resources
Dec 14 23:14:38 pve-nas kernel: [83601.630762] pci_bus 0000:06: Allocating resources
Dec 14 23:14:55 pve-nas kernel: [83618.162041] pci_bus 0000:06: Allocating resources
Dec 14 23:15:14 pve-nas kernel: [83637.389341] pci_bus 0000:06: Allocating resources
Dec 14 23:15:33 pve-nas kernel: [83656.556605] pci_bus 0000:06: Allocating resources
Dec 14 23:15:52 pve-nas kernel: [83674.935762] pci_bus 0000:06: Allocating resources
Dec 14 23:16:10 pve-nas kernel: [83693.566986] pci_bus 0000:06: Allocating resources
Dec 14 23:16:28 pve-nas kernel: [83711.418250] pci_bus 0000:06: Allocating resources
Dec 14 23:16:47 pve-nas kernel: [83730.341475] pci_bus 0000:06: Allocating resources
Dec 14 23:16:54 pve-nas pmxcfs[2605]: [status] notice: received log
Dec 14 23:17:01 pve-nas CRON[4072036]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Dec 14 23:17:05 pve-nas kernel: [83748.424651] pci_bus 0000:06: Allocating resources
Dec 14 23:17:24 pve-nas kernel: [83767.467839] pci_bus 0000:06: Allocating resources
Dec 14 23:17:30 pve-nas pmxcfs[2605]: [status] notice: received log
Dec 14 23:17:43 pve-nas kernel: [83786.443025] pci_bus 0000:06: Allocating resources
Dec 14 23:17:50 pve-nas systemd[1]: session-61.scope: Succeeded.
Dec 14 23:17:50 pve-nas pmxcfs[2605]: [status] notice: received log
Dec 14 23:17:50 pve-nas pmxcfs[2605]: [status] notice: received log
Dec 14 23:17:51 pve-nas systemd[1]: Started Session 63 of user root.
Dec 14 23:18:02 pve-nas kernel: [83805.674327] pci_bus 0000:06: Allocating resources
Dec 14 23:18:18 pve-nas kernel: [83821.657595] pci_bus 0000:06: Allocating resources
Dec 14 23:18:34 pve-nas kernel: [83837.600994] pci_bus 0000:06: Allocating resources
Dec 14 23:18:54 pve-nas kernel: [83856.828131] pci_bus 0000:06: Allocating resources
Dec 14 23:19:10 pve-nas kernel: [83873.495495] pci_bus 0000:06: Allocating resources
Dec 14 23:19:29 pve-nas kernel: [83892.214648] pci_bus 0000:06: Allocating resources
Dec 14 23:19:48 pve-nas kernel: [83911.313923] pci_bus 0000:06: Allocating resources
Dec 14 23:20:06 pve-nas kernel: [83929.601133] pci_bus 0000:06: Allocating resources
Dec 14 23:20:26 pve-nas kernel: [83948.832383] pci_bus 0000:06: Allocating resources
Dec 14 23:20:44 pve-nas kernel: [83966.903624] pci_bus 0000:06: Allocating resources