Hardware: Intel Dragon Canyon NUC 12 Extreme i9 64GB RAM INTEL UHD 770
Proxmox Server: 8.1.3
Primary Use: LXC Plex Server
Storage: NFS mounts from the LXC Plex Server to Synology NAS for media (TV,Movies etc)
Symptom: Within 48 hours the server will just crash/halt. The console screen will just be stuck, cursor is not flashing.
So I need some help to try and diagnose this. It's very odd. The server just hard crashes. I know it has todo with Plex, when I set the plex service to restart every 2 days the LXC rarely will crash. If I do not restart plex it will eventually crash usually within 48-72 hours.
I have no idea where to look. The syslogs from the Proxmox host and the Plex LXC haven't given any clues. Where else should I look?
When I go through the syslog there are not many clues.
Syslog from Proxmox server:
Plex LXC Syslog:
Last lines of Plex log:
Proxmox Server: 8.1.3
Primary Use: LXC Plex Server
Storage: NFS mounts from the LXC Plex Server to Synology NAS for media (TV,Movies etc)
Symptom: Within 48 hours the server will just crash/halt. The console screen will just be stuck, cursor is not flashing.
So I need some help to try and diagnose this. It's very odd. The server just hard crashes. I know it has todo with Plex, when I set the plex service to restart every 2 days the LXC rarely will crash. If I do not restart plex it will eventually crash usually within 48-72 hours.
I have no idea where to look. The syslogs from the Proxmox host and the Plex LXC haven't given any clues. Where else should I look?
When I go through the syslog there are not many clues.
Syslog from Proxmox server:
Code:
Jan 17 10:56:26 pve11 postfix/qmgr[1462]: E0A051802A: from=<root@pve11.localdomain>, size=4126, nrcpt=1 (queue active)
Jan 17 10:56:59 pve11 postfix/smtp[431217]: connect to aspmx.l.google.com[142.250.31.26]:25: Connection timed out
Jan 17 10:56:59 pve11 postfix/smtp[431217]: connect to aspmx.l.google.com[2607:f8b0:4004:c06::1a]:25: Network is unreachable
Jan 17 10:57:29 pve11 postfix/smtp[431217]: connect to alt1.aspmx.l.google.com[209.85.202.27]:25: Connection timed out
Jan 17 10:57:29 pve11 postfix/smtp[431217]: connect to alt1.aspmx.l.google.com[2a00:1450:400b:c00::1a]:25: Network is unreachable
Jan 17 10:57:59 pve11 postfix/smtp[431217]: connect to alt2.aspmx.l.google.com[64.233.184.27]:25: Connection timed out
Jan 17 10:57:59 pve11 postfix/smtp[431217]: E0A051802A: to=<michael@******.net>, relay=none, delay=120385, delays=120293/0.01/93/0, dsn=4.4.1, status=deferred (connect to alt2.aspmx.l.google.com[64.233.184.27]:25: Connection timed out)
Jan 17 11:17:01 pve11 CRON[435098]: pam_unix(cron:session): session opened for user root(uid=0) by (uid=0)
Jan 17 11:17:01 pve11 CRON[435099]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)
Jan 17 11:17:01 pve11 CRON[435098]: pam_unix(cron:session): session closed for user root
Jan 17 11:46:26 pve11 postfix/qmgr[1462]: E446B18628: from=<root@pve11.localdomain>, size=4192, nrcpt=1 (queue active)
Jan 17 11:47:01 pve11 postfix/smtp[440333]: connect to aspmx.l.google.com[142.251.167.27]:25: Connection timed out
Jan 17 11:47:01 pve11 postfix/smtp[440333]: connect to aspmx.l.google.com[2607:f8b0:4004:c09::1a]:25: Network is unreachable
Jan 17 11:47:31 pve11 postfix/smtp[440333]: connect to alt1.aspmx.l.google.com[209.85.202.26]:25: Connection timed out
Jan 17 11:47:31 pve11 postfix/smtp[440333]: connect to alt1.aspmx.l.google.com[2a00:1450:400b:c00::1b]:25: Network is unreachable
Jan 17 11:48:01 pve11 postfix/smtp[440333]: connect to alt2.aspmx.l.google.com[64.233.184.26]:25: Connection timed out
Jan 17 11:48:01 pve11 postfix/smtp[440333]: E446B18628: to=<michael@******.net>, relay=none, delay=91051, delays=90956/0.01/95/0, dsn=4.4.1, status=deferred (connect to alt2.aspmx.l.google.com[64.233.184.26]:25: Connection timed out)
Jan 17 11:49:48 pve11 pmxcfs[2069]: [dcdb] notice: data verification successful
Jan 17 11:56:09 pve11 systemd[1]: Starting man-db.service - Daily man-db regeneration...
Jan 17 11:56:09 pve11 systemd[1]: man-db.service: Deactivated successfully.
Jan 17 11:56:09 pve11 systemd[1]: Finished man-db.service - Daily man-db regeneration.
Jan 17 12:06:26 pve11 postfix/qmgr[1462]: E0A051802A: from=<root@pve11.localdomain>, size=4126, nrcpt=1 (queue active)
Jan 17 12:06:29 pve11 postfix/smtp[443952]: connect to aspmx.l.google.com[2607:f8b0:4004:c08::1a]:25: Network is unreachable
Jan 17 12:06:59 pve11 postfix/smtp[443952]: connect to aspmx.l.google.com[142.251.167.27]:25: Connection timed out
Jan 17 12:07:29 pve11 postfix/smtp[443952]: connect to alt1.aspmx.l.google.com[209.85.202.27]:25: Connection timed out
Jan 17 12:07:29 pve11 postfix/smtp[443952]: connect to alt1.aspmx.l.google.com[2a00:1450:400b:c00::1b]:25: Network is unreachable
Jan 17 12:07:29 pve11 postfix/smtp[443952]: connect to alt2.aspmx.l.google.com[2a00:1450:400c:c0b::1b]:25: Network is unreachable
Jan 17 12:07:30 pve11 postfix/smtp[443952]: E0A051802A: to=<michael@******.net>, relay=none, delay=124556, delays=124493/0.01/63/0, dsn=4.4.1, status=deferred (connect to alt2.aspmx.l.google.com[2a00:1450:400c:c0b::1b]:25: Network is unreachable)
Jan 17 12:17:01 pve11 CRON[445817]: pam_unix(cron:session): session opened for user root(uid=0) by (uid=0)
Jan 17 12:17:01 pve11 CRON[445818]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)
Jan 17 12:17:01 pve11 CRON[445817]: pam_unix(cron:session): session closed for user root
Jan 17 12:49:48 pve11 pmxcfs[2069]: [dcdb] notice: data verification successful
Jan 17 12:56:27 pve11 postfix/qmgr[1462]: E446B18628: from=<root@pve11.localdomain>, size=4192, nrcpt=1 (queue active)
Jan 17 12:56:32 pve11 postfix/smtp[453001]: connect to aspmx.l.google.com[2607:f8b0:4004:c09::1b]:25: Network is unreachable
Jan 17 12:57:02 pve11 postfix/smtp[453001]: connect to aspmx.l.google.com[142.251.167.26]:25: Connection timed out
Jan 17 12:57:32 pve11 postfix/smtp[453001]: connect to alt1.aspmx.l.google.com[209.85.202.27]:25: Connection timed out
Jan 17 12:57:32 pve11 postfix/smtp[453001]: connect to alt1.aspmx.l.google.com[2a00:1450:400b:c00::1b]:25: Network is unreachable
Jan 17 12:57:32 pve11 postfix/smtp[453001]: connect to alt2.aspmx.l.google.com[2a00:1450:400c:c0b::1b]:25: Network is unreachable
Jan 17 12:57:32 pve11 postfix/smtp[453001]: E446B18628: to=<michael@******.net>, relay=none, delay=95221, delays=95156/0.01/65/0, dsn=4.4.1, status=deferred (connect to alt2.aspmx.l.google.com[2a00:1450:400c:c0b::1b]:25: Network is unreachable)
-- Reboot --Plex LXC Syslog:
Code:
Jan 17 10:21:01 plex CRON[11085]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Jan 17 11:21:01 plex CRON[11651]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Jan 17 11:54:09 plex systemd[1]: Starting Message of the Day...
Jan 17 11:54:09 plex systemd[11834]: motd-news.service: Failed to locate executable /etc/update-motd.d/50-motd-news: Permission denied
Jan 17 11:54:09 plex systemd[11834]: motd-news.service: Failed at step EXEC spawning /etc/update-motd.d/50-motd-news: Permission denied
Jan 17 11:54:09 plex systemd[1]: motd-news.service: Main process exited, code=exited, status=203/EXEC
Jan 17 11:54:09 plex systemd[1]: motd-news.service: Failed with result 'exit-code'.
Jan 17 11:54:09 plex systemd[1]: Failed to start Message of the Day.
-> Last line before crash
Jan 17 12:21:01 plex CRON[12020]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Jan 17 20:27:05 plex keyboard-setup.sh[55]: Couldn't get a file descriptor referring to the console.
Jan 17 20:27:05 plex keyboard-setup.sh[57]: Couldn't get a file descriptor referring to the console.
Jan 17 20:27:05 plex kernel: [ 0.000000] Linux version 6.5.11-7-pve (build@proxmox) (gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 SMP PREEMPT_DYNAMIC PMX 6.5.11-7 (2023-12-05T09:44Z) ()
Jan 17 20:27:05 plex keyboard-setup.sh[59]: Couldn't get a file descriptor referring to the console
Jan 17 20:27:05 plex kernel: [ 0.000000] Command line: initrd=\EFI\proxmox\6.5.11-7-pve\initrd.img-6.5.11-7-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs
Jan 17 20:27:05 plex kernel: [ 0.000000] KERNEL supported cpus:
Jan 17 20:27:05 plex kernel: [ 0.000000] Intel GenuineIntel
Jan 17 20:27:05 plex kernel: [ 0.000000] AMD AuthenticAMD
Jan 17 20:27:05 plex kernel: [ 0.000000] Hygon HygonGenuine
Jan 17 20:27:05 plex kernel: [ 0.000000] Centaur CentaurHauls
Jan 17 20:27:05 plex systemd[1]: Starting Flush Journal to Persistent Storage...
Jan 17 20:27:05 plex kernel: [ 0.000000] zhaoxin Shanghai
Jan 17 20:27:05 plex kernel: [ 0.000000] x86/split lock detection: #AC: crashing the kernel on kernel split_locks and warning on user-space split_locks
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-provided physical RAM map:
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009dfff] usable
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-e820: [mem 0x000000000009e000-0x000000000009efff] reserved
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-e820: [mem 0x000000000009f000-0x000000000009ffff] usable
Jan 17 20:27:05 plex keyboard-setup.sh[62]: setupcon: We are not on the console, the console is left unconfigured.
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-e820: [mem 0x00000000000a0000-0x00000000000fffff] reserved
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000003a854fff] usable
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-e820: [mem 0x000000003a855000-0x0000000040e67fff] reserved
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-e820: [mem 0x0000000040e68000-0x0000000040f49fff] ACPI data
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-e820: [mem 0x0000000040f4a000-0x0000000041120fff] ACPI NVS
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-e820: [mem 0x0000000041121000-0x00000000447fefff] reserved
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-e820: [mem 0x00000000447ff000-0x00000000447fffff] usable
Jan 17 20:27:05 plex systemd[1]: Finished Set the console keyboard layout.
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-e820: [mem 0x0000000044800000-0x000000004affffff] reserved
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-e820: [mem 0x000000004b200000-0x000000004b3fffff] reserved
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-e820: [mem 0x000000004be00000-0x00000000527fffff] reserved
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-e820: [mem 0x00000000c0000000-0x00000000cfffffff] reserved
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-e820: [mem 0x00000000fe000000-0x00000000fe010fff] reserved
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-e820: [mem 0x00000000fec00000-0x00000000fec00fff] reserved
Jan 17 20:27:05 plex systemd[1]: Finished Create System Users.
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-e820: [mem 0x00000000fed00000-0x00000000fed00fff] reserved
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-e820: [mem 0x00000000fed20000-0x00000000fed7ffff] reserved
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-e820: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
Jan 17 20:27:05 plex kernel: [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000010ad7fffff] usable
Jan 17 20:27:05 plex kernel: [ 0.000000] NX (Execute Disable) protection: active
Jan 17 20:27:05 plex kernel: [ 0.000000] e820: update [mem 0x291ae018-0x291bfa57] usable ==> usable
Jan 17 20:27:05 plex apparmor.systemd[81]: Restarting AppArmor
Jan 17 20:27:05 plex kernel: [ 0.000000] e820: update [mem 0x291ae018-0x291bfa57] usable ==> usable
Jan 17 20:27:05 plex kernel: [ 0.000000] extended physical RAM map:
Jan 17 20:27:05 plex apparmor.systemd[81]: Reloading AppArmor profiles
Jan 17 20:27:05 plex kernel: [ 0.000000] reserve setup_data: [mem 0x0000000000000000-0x000000000009dfff] usable
Jan 17 20:27:05 plex kernel: [ 0.000000] reserve setup_data: [mem 0x000000000009e000-0x000000000009efff] reserved
Jan 17 20:27:05 plex kernel: [ 0.000000] reserve setup_data: [mem 0x000000000009f000-0x000000000009ffff] usable
Jan 17 20:27:05 plex kernel: [ 0.000000] reserve setup_data: [mem 0x00000000000a0000-0x00000000000fffff] reserved
Jan 17 20:27:05 plex kernel: [ 0.000000] reserve setup_data: [mem 0x0000000000100000-0x00000000291ae017] usable
Jan 17 20:27:05 plex kernel: [ 0.000000] reserve setup_data: [mem 0x00000000291ae018-0x00000000291bfa57] usable
Jan 17 20:27:05 plex systemd[1]: Starting Create Static Device Nodes in /dev...
Jan 17 20:27:05 plex kernel: [ 0.000000] reserve setup_data: [mem 0x00000000291bfa58-0x000000003a854fff] usable
Jan 17 20:27:05 plex kernel: [ 0.000000] reserve setup_data: [mem 0x000000003a855000-0x0000000040e67fff] reserved
Jan 17 20:27:05 plex kernel: [ 0.000000] reserve setup_data: [mem 0x0000000040e68000-0x0000000040f49fff] ACPI data
Jan 17 20:27:05 plex kernel: [ 0.000000] reserve setup_data: [mem 0x0000000040f4a000-0x0000000041120fff] ACPI NVS
Jan 17 20:27:05 plex kernel: [ 0.000000] reserve setup_data: [mem 0x0000000041121000-0x00000000447fefff] reserved
Jan 17 20:27:05 plex systemd[1]: Finished nftables.
Jan 17 20:27:05 plex systemd[1]: Finished Create Static Device Nodes in /dev.
Jan 17 20:27:05 plex systemd[1]: Reached target Preparation for Local File Systems.
Jan 17 20:27:05 plex kernel: [ 0.000000] reserve setup_data: [mem 0x00000000447ff000-0x00000000447fffff] usable
Jan 17 20:27:05 plex systemd[1]: Reached target Local File Systems.
Jan 17 20:27:05 plex systemd[1]: Starting Load AppArmor profiles...Last lines of Plex log:
Code:
Jan 17, 2024 12:58:46.894 [139960618945336] ERROR - [CreditsDetectionManager] BufferingLineReader: failed to read line (error: -1)
Jan 17, 2024 12:58:46.894 [139960750123832] DEBUG - Jobs: '/usr/lib/plexmediaserver/Plex Media Scanner' exit code for process 13134 is 0 (success)
Jan 17, 2024 12:58:46.894 [139960618945336] DEBUG - [CreditsDetectionManager] Activity: updated activity 22280597-3fa3-41e5-a3a0-ac2bbb9a5879 - completed 75.0% - Detecting Credits
Jan 17, 2024 12:58:46.894 [139960618945336] DEBUG - [CreditsDetectionManager] Updating part with ID=194330 [/mnt/tv/Finding Your Roots with Henry Louis Gates, Jr/Season 09/Finding Your Roots with Henry Louis Gates, Jr. (2012) - S09E07 - Chosen [WEBDL-1080p][8bit][h264][AAC 2.0]-4f8c4100292.mkv]
Jan 17, 2024 12:58:46.896 [139960618945336] DEBUG - [CreditsDetectionManager] Successfully processed item 190696
Jan 17, 2024 12:58:46.897 [139960618945336] DEBUG - [CreditsDetectionManager] Activity: Ended activity 22280597-3fa3-41e5-a3a0-ac2bbb9a5879.
Jan 17, 2024 12:58:46.897 [139960618945336] DEBUG - Activity: ended sub-activity d932a489-3bf5-4a67-a324-d2ffcfc5b59c parent: dd68521f-1744-4d35-a3bb-01b8d2578f5f overall progress: 100.0% (1007/1007)
Jan 17, 2024 12:58:46.897 [139960618945336] DEBUG - Activity: updated activity dd68521f-1744-4d35-a3bb-01b8d2578f5f - completed 100.0% - Butler tasks
Jan 17, 2024 12:58:46.897 [139960618945336] DEBUG - Activity: registered new sub-activity aa8a9297-34c9-455f-934d-780ac59d355b - "ButlerTaskGenerateCreditsMarkers" parent: dd68521f-1744-4d35-a3bb-01b8d2578f5f overall progress: 99.9% (1007/1008)
Jan 17, 2024 12:58:46.897 [139960618945336] DEBUG - Activity: updated activity dd68521f-1744-4d35-a3bb-01b8d2578f5f - completed 99.9% - Butler tasks
Jan 17, 2024 12:58:46.897 [139960618945336] DEBUG - Butler: Scheduling credits marker creation for: 190695
Jan 17, 2024 12:58:46.899 [139960618945336] DEBUG - [CreditsDetectionManager] Running credits detection for item 190695
Jan 17, 2024 12:58:46.899 [139960618945336] DEBUG - [CreditsDetectionManager] Activity: registered new activity 7ffc74c3-18ba-4407-82e2-e5ac708f6904 - "Detecting Credits"
Jan 17, 2024 12:58:46.958 [139960618945336] DEBUG - [CreditsDetectionManager/JobRunner] Job running: FFMPEG_EXTERNAL_LIBS='/var/lib/plexmediaserver/Library/Application\ Support/Plex\ Media\ Server/Codecs/1d5bf5d-4660-linux-x86_64/' X_PLEX_TOKEN=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx "/usr/lib/plexmediaserver/Plex Media Scanner" -C -f "/mnt/tv/Finding Your Roots with Henry Louis Gates, Jr/Season 09/Finding Your Roots with Henry Louis Gates, Jr. (2012) - S09E06 - Family Lost and Found [WEBDL-1080p][8bit][h264][AAC 2.0]-4f8c4100292.mkv" --log-file-suffix " Credits" --creditsTempDataPath /tmp/PlexCreditsDetection-4d082551-e57e-40f0-b762-3f51efa1db1e
Jan 17, 2024 12:58:46.958 [139960618945336] DEBUG - [CreditsDetectionManager/JobRunner] Jobs: Starting child process with pid 13297
Jan 17, 2024 12:58:48.885 [139960527022904] DEBUG - Push: Queueing up 2 sections to process for new content.
Last edited:
