Feb 21 03:00:02 pve pvescheduler[2273299]: <root@pam> starting task UPID:pve:0022B014:024213CA:699911A2:vzdump:114:root@pam:
Feb 21 03:00:07 pve pvescheduler[2273300]: INFO: starting new backup job: vzdump 114 --notification-mode notification-system --node pve --fleecing 0 --compress zstd --notes-template '{{guestname}}' --storage T3_2_VM --prune-backups 'keep-monthly=1,keep-weekly=1' --quiet 1 --mode snapshot
Feb 21 03:00:07 pve pvescheduler[2273300]: INFO: Starting Backup of VM 114 (qemu)
Feb 21 03:00:07 pve pvestatd[1204]: status update time (6.732 seconds)
Feb 21 03:01:08 pve pvestatd[1204]: status update time (6.894 seconds)
Feb 21 03:01:13 pve pvestatd[1204]: got timeout
Feb 21 03:03:59 pve kernel: INFO: task nfsd:1002813 blocked for more than 122 seconds.
Feb 21 03:03:59 pve kernel: Tainted: P S O 6.17.9-1-pve #1
Feb 21 03:03:59 pve kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Feb 21 03:03:59 pve kernel: task:nfsd state:D stack:0 pid:1002813 tgid:1002813 ppid:2 task_flags:0x300040 flags:0x00004000
Feb 21 03:03:59 pve kernel: Call Trace:
Feb 21 03:03:59 pve kernel: <TASK>
Feb 21 03:03:59 pve kernel: __schedule+0x468/0x1310
Feb 21 03:03:59 pve kernel: schedule+0x27/0xf0
Feb 21 03:03:59 pve kernel: schedule_preempt_disabled+0x15/0x30
Feb 21 03:03:59 pve kernel: rwsem_down_write_slowpath+0x268/0x6d0
Feb 21 03:03:59 pve kernel: ? nfsd_current_rqst+0x3b/0x50 [nfsd]
Feb 21 03:03:59 pve kernel: ? aa_file_perm+0x60/0x3b0
Feb 21 03:03:59 pve kernel: down_write+0x5b/0x80
Feb 21 03:03:59 pve kernel: ext4_buffered_write_iter+0x36/0x150
Feb 21 03:03:59 pve kernel: ext4_file_write_iter+0xb1/0x900
Feb 21 03:03:59 pve kernel: ? apparmor_file_permission+0x1f/0x30
Feb 21 03:03:59 pve kernel: ? security_file_permission+0x36/0x60
Feb 21 03:03:59 pve kernel: ? rw_verify_area+0x57/0x190
Feb 21 03:03:59 pve kernel: vfs_iocb_iter_write+0xd7/0x230
Feb 21 03:03:59 pve kernel: nfsd_vfs_write+0x264/0x670 [nfsd]
Feb 21 03:03:59 pve kernel: nfsd4_write+0x124/0x1b0 [nfsd]
Feb 21 03:03:59 pve kernel: nfsd4_proc_compound+0x3b0/0x770 [nfsd]
Feb 21 03:03:59 pve kernel: nfsd_dispatch+0xcd/0x220 [nfsd]
Feb 21 03:03:59 pve kernel: svc_process_common+0x4d3/0x6c0 [sunrpc]
Feb 21 03:03:59 pve kernel: ? __pfx_nfsd_dispatch+0x10/0x10 [nfsd]
Feb 21 03:03:59 pve kernel: svc_process+0x136/0x1f0 [sunrpc]
Feb 21 03:03:59 pve kernel: svc_recv+0x7e8/0x9c0 [sunrpc]
Feb 21 03:03:59 pve kernel: ? __pfx_nfsd+0x10/0x10 [nfsd]
Feb 21 03:03:59 pve kernel: nfsd+0x90/0xf0 [nfsd]
Feb 21 03:03:59 pve kernel: kthread+0x108/0x220
Feb 21 03:03:59 pve kernel: ? __pfx_kthread+0x10/0x10
Feb 21 03:03:59 pve kernel: ret_from_fork+0x205/0x240
Feb 21 03:03:59 pve kernel: ? __pfx_kthread+0x10/0x10
Feb 21 03:03:59 pve kernel: ret_from_fork_asm+0x1a/0x30
Feb 21 03:03:59 pve kernel: </TASK>
Feb 21 03:03:59 pve kernel: INFO: task nfsd:1002813 <writer> blocked on an rw-semaphore likely owned by task nfsd:1002819 <writer>
Feb 21 03:03:59 pve kernel: task:nfsd state:D stack:0 pid:1002819 tgid:1002819 ppid:2 task_flags:0x300840 flags:0x00004000
Feb 21 03:03:59 pve kernel: Call Trace:
Feb 21 03:03:59 pve kernel: <TASK>
Feb 21 03:03:59 pve kernel: __schedule+0x468/0x1310
Feb 21 03:03:59 pve kernel: ? rpc_run_task+0x10f/0x180 [sunrpc]
Feb 21 03:03:59 pve kernel: schedule+0x27/0xf0
Feb 21 03:03:59 pve kernel: wait_on_commit+0x9e/0xe0 [nfs]
Feb 21 03:03:59 pve kernel: ? __pfx_var_wake_function+0x10/0x10
Feb 21 03:03:59 pve kernel: __nfs_commit_inode+0xd2/0x1d0 [nfs]
Feb 21 03:03:59 pve kernel: nfs_wb_folio+0xbb/0x1e0 [nfs]
Feb 21 03:03:59 pve kernel: nfs_release_folio+0x72/0x110 [nfs]
Feb 21 03:03:59 pve kernel: filemap_release_folio+0x62/0xa0
Feb 21 03:03:59 pve kernel: __folio_split+0x679/0x1800
Feb 21 03:03:59 pve kernel: ? __isolate_free_page+0x1dc/0x430
Feb 21 03:03:59 pve kernel: split_huge_page_to_list_to_order+0x2e/0xb0
Feb 21 03:03:59 pve kernel: split_folio_to_list+0x10/0x20
Feb 21 03:03:59 pve kernel: migrate_pages_batch+0x463/0xec0
Feb 21 03:03:59 pve kernel: ? __pfx_compaction_alloc+0x10/0x10
Feb 21 03:03:59 pve kernel: ? __pfx_compaction_free+0x10/0x10
Feb 21 03:03:59 pve kernel: migrate_pages+0x9a7/0xda0
Feb 21 03:03:59 pve kernel: ? folio_add_lru+0x49/0x60
Feb 21 03:03:59 pve kernel: ? __pfx_compaction_free+0x10/0x10
Feb 21 03:03:59 pve kernel: ? __pfx_compaction_alloc+0x10/0x10
Feb 21 03:03:59 pve kernel: compact_zone+0xb8e/0x1150
Feb 21 03:03:59 pve kernel: compact_zone_order+0xa4/0x100
Feb 21 03:03:59 pve kernel: try_to_compact_pages+0xde/0x2b0
Feb 21 03:03:59 pve kernel: __alloc_pages_direct_compact+0x90/0x210
Feb 21 03:03:59 pve kernel: __alloc_frozen_pages_noprof+0x5ba/0x12a0
Feb 21 03:03:59 pve kernel: ? xas_load+0x15/0xf0
Feb 21 03:03:59 pve kernel: ? policy_nodemask+0x111/0x190
Feb 21 03:03:59 pve kernel: alloc_pages_mpol+0x80/0x180
Feb 21 03:03:59 pve kernel: folio_alloc_noprof+0x5b/0xc0
Feb 21 03:03:59 pve kernel: filemap_alloc_folio_noprof+0xe1/0xf0
Feb 21 03:03:59 pve kernel: __filemap_get_folio+0x18c/0x340
Feb 21 03:03:59 pve kernel: ext4_da_write_begin+0x143/0x360
Feb 21 03:03:59 pve kernel: generic_perform_write+0x134/0x2d0
Feb 21 03:03:59 pve kernel: ext4_buffered_write_iter+0x6d/0x150
Feb 21 03:03:59 pve kernel: ext4_file_write_iter+0xb1/0x900
Feb 21 03:03:59 pve kernel: ? apparmor_file_permission+0x1f/0x30
Feb 21 03:03:59 pve kernel: ? security_file_permission+0x36/0x60
Feb 21 03:03:59 pve kernel: ? rw_verify_area+0x57/0x190
Feb 21 03:03:59 pve kernel: vfs_iocb_iter_write+0xd7/0x230
Feb 21 03:03:59 pve kernel: nfsd_vfs_write+0x264/0x670 [nfsd]
Feb 21 03:03:59 pve kernel: nfsd4_write+0x124/0x1b0 [nfsd]
Feb 21 03:03:59 pve kernel: nfsd4_proc_compound+0x3b0/0x770 [nfsd]
Feb 21 03:03:59 pve kernel: nfsd_dispatch+0xcd/0x220 [nfsd]
Feb 21 03:03:59 pve kernel: svc_process_common+0x4d3/0x6c0 [sunrpc]
Feb 21 03:03:59 pve kernel: ? __pfx_nfsd_dispatch+0x10/0x10 [nfsd]
Feb 21 03:03:59 pve kernel: svc_process+0x136/0x1f0 [sunrpc]
Feb 21 03:03:59 pve kernel: svc_recv+0x7e8/0x9c0 [sunrpc]
Feb 21 03:03:59 pve kernel: ? __pfx_nfsd+0x10/0x10 [nfsd]
Feb 21 03:03:59 pve kernel: nfsd+0x90/0xf0 [nfsd]
Feb 21 03:03:59 pve kernel: kthread+0x108/0x220
Feb 21 03:03:59 pve kernel: ? __pfx_kthread+0x10/0x10
Feb 21 03:03:59 pve kernel: ret_from_fork+0x205/0x240
Feb 21 03:03:59 pve kernel: ? __pfx_kthread+0x10/0x10
Feb 21 03:03:59 pve kernel: ret_from_fork_asm+0x1a/0x30
Feb 21 03:03:59 pve kernel: </TASK>
.
.
.
.
Feb 21 03:04:00 pve kernel: Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings
Feb 21 03:10:01 pve CRON[2274974]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
Feb 21 03:10:01 pve CRON[2274976]: (root) CMD (test -e /run/systemd/system || SERVICE_MODE=1 /sbin/e2scrub_all -A -r)
Feb 21 03:10:01 pve CRON[2274974]: pam_unix(cron:session): session closed for user root
Feb 21 03:17:01 pve CRON[2275933]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
Feb 21 03:17:01 pve CRON[2275935]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)
Feb 21 03:17:01 pve CRON[2275933]: pam_unix(cron:session): session closed for user root
Feb 21 03:17:11 pve systemd[1]: Starting pve-daily-update.service - Daily PVE download activities...
Feb 21 03:17:12 pve pveupdate[2275960]: <root@pam> starting task UPID:pve:0022BA7D:0243A5FE:699915A8:aptupdate::root@pam:
Feb 21 03:17:13 pve pveupdate[2275965]: update new package list: /var/lib/pve-manager/pkgupdates
Feb 21 03:17:16 pve pveupdate[2275960]: <root@pam> end task UPID:pve:0022BA7D:0243A5FE:699915A8:aptupdate::root@pam: OK
Feb 21 03:17:16 pve systemd[1]: pve-daily-update.service: Deactivated successfully.
Feb 21 03:17:16 pve systemd[1]: Finished pve-daily-update.service - Daily PVE download activities.
Feb 21 03:17:16 pve systemd[1]: pve-daily-update.service: Consumed 4.503s CPU time, 324.8M memory peak.
Feb 21 03:20:48 pve pmxcfs[1003]: [status] notice: received log
Feb 21 03:20:51 pve pmxcfs[1003]: [status] notice: received log
Feb 21 03:21:57 pve pmxcfs[1003]: [dcdb] notice: data verification successful
Feb 21 03:31:36 pve postfix/qmgr[1149]: 6DB46360609: from=<adminno@pve.hp>, size=450, nrcpt=1 (queue active)
Feb 21 03:31:36 pve postfix/local[2277951]: error: open database /etc/aliases.db: No such file or directory
Feb 21 03:31:36 pve postfix/local[2277951]: warning: hash:/etc/aliases is unavailable. open database /etc/aliases.db: No such file or directory
Feb 21 03:31:36 pve postfix/local[2277951]: warning: hash:/etc/aliases: lookup of 'root' failed
Feb 21 03:31:36 pve postfix/local[2277951]: 6DB46360609: to=<root@pve.hp>, orig_to=<root>, relay=local, delay=382028, delays=382028/0.01/0/0.01, dsn=4.3.0, status=deferred (alias database unavailable)
Feb 21 03:37:14 pve kernel: nfs: server pve.lan not responding, timed out
Feb 21 03:37:14 pve kernel: nfs: server pve.lan not responding, timed out
Feb 21 03:37:14 pve kernel: nfs: server pve.lan not responding, timed out
Feb 21 03:37:14 pve kernel: nfs: server pve.lan not responding, timed out
Feb 21 03:37:14 pve kernel: nfs: server pve.lan not responding, timed out
Feb 21 03:37:14 pve kernel: nfs: server pve.lan not responding, timed out
Feb 21 03:37:14 pve kernel: nfs: server pve.lan not responding, timed out
Feb 21 03:37:14 pve kernel: nfs: server pve.lan not responding, timed out
Feb 21 03:37:14 pve kernel: nfs: server pve.lan not responding, timed out
Feb 21 03:37:14 pve kernel: nfs: server pve.lan not responding, timed out
Feb 21 03:46:39 pve kernel: nfs: server pve.lan not responding, timed out
Feb 21 03:46:39 pve kernel: nfs: server pve.lan not responding, timed out
Feb 21 03:46:39 pve kernel: rpc_check_timeout: 261 callbacks suppressed
Feb 21 03:47:44 pve pvestatd[1204]: got timeout
Feb 21 03:47:44 pve pvestatd[1204]: unable to activate storage 'T3_2_VM' - directory '/mnt/pve/T3_2_VM' does not exist or is unreachable
Feb 21 03:47:54 pve pvestatd[1204]: got timeout
Feb 21 03:47:54 pve pvestatd[1204]: unable to activate storage 'T3_2_VM' - directory '/mnt/pve/T3_2_VM' does not exist or is unreachable
Feb 21 03:48:09 pve pvestatd[1204]: got timeout
Feb 21 03:48:09 pve pvestatd[1204]: unable to activate storage 'T3_2_VM' - directory '/mnt/pve/T3_2_VM' does not exist or is unreachable
Feb 21 03:48:09 pve pvestatd[1204]: status update time (6.773 seconds)
Feb 21 03:48:15 pve pvestatd[1204]: got timeout