Hallo,
Mein PBS ist über PVE virtualisiert und ich lasse einmal in der Woche ein Backup Job mit STOP für den PBS auf dem PVE laufen. Leider wird der PBS nicht vernünftig runter gefahren. Er bleibt in einem inkosistenten Zustand stehen. Ich muss dann immer hart STOP machen über den PVE.
Auffällige Log Meldungen zu der Zeit des Job Startes sind vorallem
Jemand zufällig eine Idee?
Grüße
Mein PBS ist über PVE virtualisiert und ich lasse einmal in der Woche ein Backup Job mit STOP für den PBS auf dem PVE laufen. Leider wird der PBS nicht vernünftig runter gefahren. Er bleibt in einem inkosistenten Zustand stehen. Ich muss dann immer hart STOP machen über den PVE.
Auffällige Log Meldungen zu der Zeit des Job Startes sind vorallem
Code:
Jan 31 12:15:06 p-pbs postfix/postfix-script[2221133]: stopping the Postfix mail system
Jan 31 12:15:06 p-pbs postfix/master[977]: terminating on signal 15
Jan 31 12:15:22 p-pbs kernel: [TTM] Buffer eviction failed
Jan 31 12:15:37 p-pbs kernel: [TTM] Buffer eviction failed
Jan 31 12:15:52 p-pbs kernel: [TTM] Buffer eviction failed
Jan 31 12:16:08 p-pbs kernel: [TTM] Buffer eviction failed
Jan 31 12:16:23 p-pbs kernel: [TTM] Buffer eviction failed
Jan 31 12:16:38 p-pbs kernel: [TTM] Buffer eviction failed
Jan 31 12:16:54 p-pbs kernel: [TTM] Buffer eviction failed
Jan 31 12:17:09 p-pbs kernel: [TTM] Buffer eviction failed
Jan 31 12:17:24 p-pbs kernel: [TTM] Buffer eviction failed
Jan 31 12:17:40 p-pbs kernel: [TTM] Buffer eviction failed
Jan 31 12:17:55 p-pbs kernel: [TTM] Buffer eviction failed
Jan 31 12:18:11 p-pbs kernel: [TTM] Buffer eviction failed
Jan 31 12:18:26 p-pbs kernel: [TTM] Buffer eviction failed
Jan 31 12:18:37 p-pbs kernel: INFO: task systemd:1 blocked for more than 122 seconds.
Jan 31 12:18:37 p-pbs kernel: Tainted: P O 6.17.4-2-pve #1
Jan 31 12:18:37 p-pbs kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 31 12:18:37 p-pbs kernel: task:systemd state:D stack:0 pid:1 tgid:1 ppid:0 task_flags:0x400100 flags:0x00004002
Jan 31 12:18:37 p-pbs kernel: Call Trace:
Jan 31 12:18:37 p-pbs kernel: <TASK>
Jan 31 12:18:37 p-pbs kernel: __schedule+0x468/0x1310
Jan 31 12:18:37 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:18:37 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:18:37 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:18:37 p-pbs kernel: ? add_wait_queue+0x76/0xa0
Jan 31 12:18:37 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:18:37 p-pbs kernel: schedule+0x27/0xf0
Jan 31 12:18:37 p-pbs kernel: schedule_preempt_disabled+0x15/0x30
Jan 31 12:18:37 p-pbs kernel: __ww_mutex_lock.constprop.0+0x7fd/0xda0
Jan 31 12:18:37 p-pbs kernel: __ww_mutex_lock_slowpath+0x16/0x30
Jan 31 12:18:37 p-pbs kernel: ww_mutex_lock+0xec/0x100
Jan 31 12:18:37 p-pbs kernel: drm_modeset_lock+0x5f/0xf0
Jan 31 12:18:37 p-pbs kernel: drm_atomic_get_plane_state+0x93/0x190
Jan 31 12:18:37 p-pbs kernel: drm_client_modeset_commit_atomic+0xb9/0x240
Jan 31 12:18:37 p-pbs kernel: drm_client_modeset_commit_locked+0x5b/0x170
Jan 31 12:18:37 p-pbs kernel: ? mutex_lock+0x12/0x50
Jan 31 12:18:37 p-pbs kernel: drm_fb_helper_pan_display+0x113/0x280
Jan 31 12:18:37 p-pbs kernel: fb_pan_display+0x8b/0x160
Jan 31 12:18:37 p-pbs kernel: bit_update_start+0x20/0x50
Jan 31 12:18:37 p-pbs kernel: fbcon_switch+0x469/0x620
Jan 31 12:18:37 p-pbs kernel: csi_J+0x2a7/0x2f0
Jan 31 12:18:37 p-pbs kernel: do_con_write+0x1405/0x2450
Jan 31 12:18:37 p-pbs kernel: con_write+0x14/0x50
Jan 31 12:18:37 p-pbs kernel: n_tty_write+0x154/0x550
Jan 31 12:18:37 p-pbs kernel: ? __pfx_woken_wake_function+0x10/0x10
Jan 31 12:18:37 p-pbs kernel: file_tty_write.isra.0+0x181/0x2d0
Jan 31 12:18:37 p-pbs kernel: tty_write+0x11/0x20
Jan 31 12:18:37 p-pbs kernel: vfs_write+0x274/0x490
Jan 31 12:18:37 p-pbs kernel: ? _raw_spin_unlock+0xe/0x40
Jan 31 12:18:37 p-pbs kernel: ksys_write+0x6f/0xf0
Jan 31 12:18:37 p-pbs kernel: __x64_sys_write+0x19/0x30
Jan 31 12:18:37 p-pbs kernel: x64_sys_call+0x79/0x2330
Jan 31 12:18:37 p-pbs kernel: do_syscall_64+0x80/0xa30
Jan 31 12:18:37 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:18:37 p-pbs kernel: ? x64_sys_call+0x1742/0x2330
Jan 31 12:18:37 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:18:37 p-pbs kernel: ? do_syscall_64+0xb8/0xa30
Jan 31 12:18:37 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:18:37 p-pbs kernel: ? __x64_sys_read+0x19/0x30
Jan 31 12:18:37 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:18:37 p-pbs kernel: ? x64_sys_call+0x1e95/0x2330
Jan 31 12:18:37 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:18:37 p-pbs kernel: ? do_syscall_64+0xb8/0xa30
Jan 31 12:18:37 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:18:37 p-pbs kernel: ? x64_sys_call+0x1151/0x2330
Jan 31 12:18:37 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:18:37 p-pbs kernel: ? do_syscall_64+0xb8/0xa30
Jan 31 12:18:37 p-pbs kernel: ? exc_page_fault+0x90/0x1b0
Jan 31 12:18:37 p-pbs kernel: entry_SYSCALL_64_after_hwframe+0x76/0x7e
Jan 31 12:18:37 p-pbs kernel: RIP: 0033:0x79d02ba9a687
Jan 31 12:18:37 p-pbs kernel: RSP: 002b:00007ffca6436320 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
Jan 31 12:18:37 p-pbs kernel: RAX: ffffffffffffffda RBX: 000079d02c04fe00 RCX: 000079d02ba9a687
Jan 31 12:18:37 p-pbs kernel: RDX: 000000000000000c RSI: 000079d02bf27cc4 RDI: 000000000000003e
Jan 31 12:18:37 p-pbs kernel: RBP: 000000000000003e R08: 0000000000000000 R09: 0000000000000000
Jan 31 12:18:37 p-pbs kernel: R10: 0000000000000000 R11: 0000000000000202 R12: 00000000000186a0
Jan 31 12:18:37 p-pbs kernel: R13: 0000008c83613c66 R14: 000000000000000c R15: 000000000000003e
Jan 31 12:18:37 p-pbs kernel: </TASK>
Jan 31 12:18:37 p-pbs kernel: INFO: task systemd:1 is blocked on a mutex likely owned by task kworker/1:2:2220351.
Jan 31 12:18:37 p-pbs kernel: task:kworker/1:2 state:D stack:0 pid:2220351 tgid:2220351 ppid:2 task_flags:0x4208060 flags:0x00004000
Jan 31 12:18:37 p-pbs kernel: Workqueue: events drm_fb_helper_damage_work
Jan 31 12:18:37 p-pbs kernel: Call Trace:
Jan 31 12:18:37 p-pbs kernel: <TASK>
Jan 31 12:18:37 p-pbs kernel: __schedule+0x468/0x1310
Jan 31 12:18:37 p-pbs kernel: ? lock_timer_base+0x73/0xa0
Jan 31 12:18:37 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:18:37 p-pbs kernel: ? _raw_spin_unlock_irqrestore+0x11/0x60
Jan 31 12:18:37 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:18:37 p-pbs kernel: schedule+0x27/0xf0
Jan 31 12:18:37 p-pbs kernel: schedule_timeout+0x89/0x110
Jan 31 12:18:37 p-pbs kernel: ? __pfx_process_timeout+0x10/0x10
Jan 31 12:18:37 p-pbs kernel: qxl_fence_wait+0xfc/0x1c0 [qxl]
Jan 31 12:18:37 p-pbs kernel: ? __pfx_autoremove_wake_function+0x10/0x10
Jan 31 12:18:37 p-pbs kernel: dma_fence_wait_timeout+0x67/0x170
Jan 31 12:18:37 p-pbs kernel: dma_resv_wait_timeout+0xbc/0x1e0
Jan 31 12:18:37 p-pbs kernel: ttm_bo_wait_ctx+0x53/0x90 [ttm]
Jan 31 12:18:37 p-pbs kernel: qxl_bo_move+0x4f/0x110 [qxl]
Jan 31 12:18:37 p-pbs kernel: ttm_bo_handle_move_mem+0xd3/0x1b0 [ttm]
Jan 31 12:18:37 p-pbs kernel: ttm_bo_evict+0x135/0x190 [ttm]
Jan 31 12:18:37 p-pbs kernel: ttm_bo_evict_cb+0x9a/0x110 [ttm]
Jan 31 12:18:37 p-pbs kernel: ttm_lru_walk_for_evict+0xc0/0x230 [ttm]
Jan 31 12:18:37 p-pbs kernel: ttm_bo_alloc_resource+0x1ce/0x5e0 [ttm]
Jan 31 12:18:37 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:18:37 p-pbs kernel: ttm_bo_validate+0x9a/0x180 [ttm]
Jan 31 12:18:37 p-pbs kernel: ttm_bo_init_reserved+0x157/0x1c0 [ttm]
Jan 31 12:18:37 p-pbs kernel: qxl_bo_create+0x177/0x240 [qxl]
Jan 31 12:18:37 p-pbs kernel: ? __pfx_qxl_ttm_bo_destroy+0x10/0x10 [qxl]
Jan 31 12:18:37 p-pbs kernel: qxl_alloc_bo_reserved+0x4b/0xc0 [qxl]
Jan 31 12:18:37 p-pbs kernel: qxl_image_alloc_objects+0x102/0x1c0 [qxl]
Jan 31 12:18:37 p-pbs kernel: qxl_draw_dirty_fb+0x1be/0x530 [qxl]
Jan 31 12:18:37 p-pbs kernel: ? ww_mutex_lock_interruptible+0x30/0x100
Jan 31 12:18:37 p-pbs kernel: qxl_framebuffer_surface_dirty+0x10c/0x1f0 [qxl]
Jan 31 12:18:37 p-pbs kernel: drm_fbdev_ttm_helper_fb_dirty+0x2ef/0x3ab [drm_ttm_helper]
Jan 31 12:18:37 p-pbs kernel: drm_fb_helper_damage_work+0x92/0x180
Jan 31 12:18:37 p-pbs kernel: process_one_work+0x18b/0x370
Jan 31 12:18:37 p-pbs kernel: worker_thread+0x33a/0x480
Jan 31 12:18:37 p-pbs kernel: ? _raw_spin_unlock_irqrestore+0x11/0x60
Jan 31 12:18:37 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:18:37 p-pbs kernel: ? __pfx_worker_thread+0x10/0x10
Jan 31 12:18:37 p-pbs kernel: kthread+0x10b/0x220
Jan 31 12:18:37 p-pbs kernel: ? _raw_spin_unlock_irq+0xe/0x60
Jan 31 12:18:37 p-pbs kernel: ? __pfx_kthread+0x10/0x10
Jan 31 12:18:37 p-pbs kernel: ret_from_fork+0x208/0x240
Jan 31 12:18:37 p-pbs kernel: ? __pfx_kthread+0x10/0x10
Jan 31 12:18:37 p-pbs kernel: ret_from_fork_asm+0x1a/0x30
Jan 31 12:18:37 p-pbs kernel: </TASK>
Jan 31 12:18:41 p-pbs kernel: [TTM] Buffer eviction failed
Jan 31 12:18:57 p-pbs kernel: [TTM] Buffer eviction failed
Jan 31 12:19:12 p-pbs kernel: [TTM] Buffer eviction failed
Jan 31 12:19:27 p-pbs kernel: [TTM] Buffer eviction failed
Jan 31 12:19:43 p-pbs kernel: [TTM] Buffer eviction failed
Jan 31 12:19:58 p-pbs kernel: [TTM] Buffer eviction failed
Jan 31 12:20:13 p-pbs kernel: [TTM] Buffer eviction failed
Jan 31 12:20:29 p-pbs kernel: [TTM] Buffer eviction failed
Jan 31 12:20:40 p-pbs kernel: INFO: task systemd:1 blocked for more than 245 seconds.
Jan 31 12:20:40 p-pbs kernel: Tainted: P O 6.17.4-2-pve #1
Jan 31 12:20:40 p-pbs kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 31 12:20:40 p-pbs kernel: task:systemd state:D stack:0 pid:1 tgid:1 ppid:0 task_flags:0x400100 flags:0x00004002
Jan 31 12:20:40 p-pbs kernel: Call Trace:
Jan 31 12:20:40 p-pbs kernel: <TASK>
Jan 31 12:20:40 p-pbs kernel: __schedule+0x468/0x1310
Jan 31 12:20:40 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:20:40 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:20:40 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:20:40 p-pbs kernel: ? add_wait_queue+0x76/0xa0
Jan 31 12:20:40 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:20:40 p-pbs kernel: schedule+0x27/0xf0
Jan 31 12:20:40 p-pbs kernel: schedule_preempt_disabled+0x15/0x30
Jan 31 12:20:40 p-pbs kernel: __ww_mutex_lock.constprop.0+0x7fd/0xda0
Jan 31 12:20:40 p-pbs kernel: __ww_mutex_lock_slowpath+0x16/0x30
Jan 31 12:20:40 p-pbs kernel: ww_mutex_lock+0xec/0x100
Jan 31 12:20:40 p-pbs kernel: drm_modeset_lock+0x5f/0xf0
Jan 31 12:20:40 p-pbs kernel: drm_atomic_get_plane_state+0x93/0x190
Jan 31 12:20:40 p-pbs kernel: drm_client_modeset_commit_atomic+0xb9/0x240
Jan 31 12:20:40 p-pbs kernel: drm_client_modeset_commit_locked+0x5b/0x170
Jan 31 12:20:40 p-pbs kernel: ? mutex_lock+0x12/0x50
Jan 31 12:20:40 p-pbs kernel: drm_fb_helper_pan_display+0x113/0x280
Jan 31 12:20:40 p-pbs kernel: fb_pan_display+0x8b/0x160
Jan 31 12:20:40 p-pbs kernel: bit_update_start+0x20/0x50
Jan 31 12:20:40 p-pbs kernel: fbcon_switch+0x469/0x620
Jan 31 12:20:40 p-pbs kernel: csi_J+0x2a7/0x2f0
Jan 31 12:20:40 p-pbs kernel: do_con_write+0x1405/0x2450
Jan 31 12:20:40 p-pbs kernel: con_write+0x14/0x50
Jan 31 12:20:40 p-pbs kernel: n_tty_write+0x154/0x550
Jan 31 12:20:40 p-pbs kernel: ? __pfx_woken_wake_function+0x10/0x10
Jan 31 12:20:40 p-pbs kernel: file_tty_write.isra.0+0x181/0x2d0
Jan 31 12:20:40 p-pbs kernel: tty_write+0x11/0x20
Jan 31 12:20:40 p-pbs kernel: vfs_write+0x274/0x490
Jan 31 12:20:40 p-pbs kernel: ? _raw_spin_unlock+0xe/0x40
Jan 31 12:20:40 p-pbs kernel: ksys_write+0x6f/0xf0
Jan 31 12:20:40 p-pbs kernel: __x64_sys_write+0x19/0x30
Jan 31 12:20:40 p-pbs kernel: x64_sys_call+0x79/0x2330
Jan 31 12:20:40 p-pbs kernel: do_syscall_64+0x80/0xa30
Jan 31 12:20:40 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:20:40 p-pbs kernel: ? x64_sys_call+0x1742/0x2330
Jan 31 12:20:40 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:20:40 p-pbs kernel: ? do_syscall_64+0xb8/0xa30
Jan 31 12:20:40 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:20:40 p-pbs kernel: ? __x64_sys_read+0x19/0x30
Jan 31 12:20:40 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:20:40 p-pbs kernel: ? x64_sys_call+0x1e95/0x2330
Jan 31 12:20:40 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:20:40 p-pbs kernel: ? do_syscall_64+0xb8/0xa30
Jan 31 12:20:40 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:20:40 p-pbs kernel: ? x64_sys_call+0x1151/0x2330
Jan 31 12:20:40 p-pbs kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Jan 31 12:20:40 p-pbs kernel: ? do_syscall_64+0xb8/0xa30
Jan 31 12:20:40 p-pbs kernel: ? exc_page_fault+0x90/0x1b0
Jan 31 12:20:40 p-pbs kernel: entry_SYSCALL_64_after_hwframe+0x76/0x7e
Jan 31 12:20:40 p-pbs kernel: RIP: 0033:0x79d02ba9a687
Jan 31 12:20:40 p-pbs kernel: RSP: 002b:00007ffca6436320 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
Jan 31 12:20:40 p-pbs kernel: RAX: ffffffffffffffda RBX: 000079d02c04fe00 RCX: 000079d02ba9a687
Jan 31 12:20:40 p-pbs kernel: RDX: 000000000000000c RSI: 000079d02bf27cc4 RDI: 000000000000003e
Jan 31 12:20:40 p-pbs kernel: RBP: 000000000000003e R08: 0000000000000000 R09: 0000000000000000
Jan 31 12:20:40 p-pbs kernel: R10: 0000000000000000 R11: 0000000000000202 R12: 00000000000186a0
Jan 31 12:20:40 p-pbs kernel: R13: 0000008c83613c66 R14: 000000000000000c R15: 000000000000003e
Jan 31 12:20:40 p-pbs kernel: </TASK>
Jan 31 12:20:40 p-pbs kernel: INFO: task systemd:1 is blocked on a mutex likely owned by task kworker/1:2:2220351.
Jan 31 12:20:40 p-pbs kernel: task:kworker/1:2 state:D stack:0 pid:2220351 tgid:2220351 ppid:2 task_flags:0x4208060 flags:0x00004000
Jan 31 12:20:40 p-pbs kernel: Workqueue: events drm_fb_helper_damage_work
Jan 31 12:20:40 p-pbs kernel: Call Trace:
Jan 31 12:20:40 p-pbs kernel: <TASK>
Jemand zufällig eine Idee?
Grüße