node proxmox is offline but. vm is on line

[fgrazioli]

Hello!
First of all I am using Google Translate.

I have the proxmox VE installed, but it has shown some errors.
the proxmox node appears as inactive and with a question mark, but the vms work.
To be able to return I need to restart the server. and it's not working very well.
Could you help me please?

 

[fgrazioli]

pveverison -v
-bash: pveverison: command not found
root@proxmox:~# pveversion -v
proxmox-ve: 8.2.0 (running kernel: 6.8.8-2-pve)
pve-manager: 8.2.4 (running version: 8.2.4/faa83925c9641325)
proxmox-kernel-helper: 8.1.0
proxmox-kernel-6.8: 6.8.8-2
proxmox-kernel-6.8.8-2-pve-signed: 6.8.8-2
proxmox-kernel-6.8.8-1-pve-signed: 6.8.8-1
proxmox-kernel-6.5.13-5-pve-signed: 6.5.13-5
proxmox-kernel-6.5: 6.5.13-5
proxmox-kernel-6.5.11-8-pve-signed: 6.5.11-8
ceph-fuse: 17.2.7-pve2
corosync: 3.1.7-pve3
criu: 3.17.1-2
glusterfs-client: 10.3-5
ifupdown2: 3.2.0-1+pmx8
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-4
libknet1: 1.28-pve1
libproxmox-acme-perl: 1.5.1
libproxmox-backup-qemu0: 1.4.1
libproxmox-rs-perl: 0.3.3
libpve-access-control: 8.1.4
libpve-apiclient-perl: 3.3.2
libpve-cluster-api-perl: 8.0.7
libpve-cluster-perl: 8.0.7
libpve-common-perl: 8.2.1
libpve-guest-common-perl: 5.1.3
libpve-http-server-perl: 5.1.0
libpve-network-perl: 0.9.8
libpve-rs-perl: 0.8.9
libpve-storage-perl: 8.2.3
libspice-server1: 0.15.1-1
lvm2: 2.03.16-2
lxc-pve: 6.0.0-1
lxcfs: 6.0.0-pve2
novnc-pve: 1.4.0-3
proxmox-backup-client: 3.2.4-1
proxmox-backup-file-restore: 3.2.4-1
proxmox-firewall: 0.4.2
proxmox-kernel-helper: 8.1.0
proxmox-mail-forward: 0.2.3
proxmox-mini-journalreader: 1.4.0
proxmox-offline-mirror-helper: 0.6.6
proxmox-widget-toolkit: 4.2.3
pve-cluster: 8.0.7
pve-container: 5.1.12
pve-docs: 8.2.2
pve-edk2-firmware: 4.2023.08-4
pve-esxi-import-tools: 0.7.1
pve-firewall: 5.0.7
pve-firmware: 3.12-1
pve-ha-manager: 4.0.5
pve-i18n: 3.2.2
pve-qemu-kvm: 9.0.0-3
pve-xtermjs: 5.3.0-3
qemu-server: 8.2.1
smartmontools: 7.3-pve1
spiceterm: 3.3.0
swtpm: 0.8.0+pve1
vncterm: 1.8.0
zfsutils-linux: 2.2.4-pve1

 

[fgrazioli]

DMESG command
[ 3.756599] systemd[1]: modprobe@dm_mod.service: Deactivated successfully.
[ 3.756764] systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod.
[ 3.756997] systemd[1]: modprobe@drm.service: Deactivated successfully.
[ 3.757160] systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm.
[ 3.757417] systemd[1]: modprobe@efi_pstore.service: Deactivated successfully.
[ 3.757580] systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore.
[ 3.757815] systemd[1]: modprobe@fuse.service: Deactivated successfully.
[ 3.757977] systemd[1]: Finished modprobe@fuse.service - Load Kernel Module fuse.
[ 3.758200] systemd[1]: modprobe@loop.service: Deactivated successfully.
[ 3.758378] systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop.
[ 3.759657] systemd[1]: Mounting sys-fs-fuse-connections.mount - FUSE Control File System...
[ 3.760707] systemd[1]: Mounting sys-kernel-config.mount - Kernel Configuration File System...
[ 3.760796] systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met.
[ 3.761572] systemd[1]: Mounted sys-fs-fuse-connections.mount - FUSE Control File System.
[ 3.762435] systemd[1]: Mounted sys-kernel-config.mount - Kernel Configuration File System.
[ 3.779278] EXT4-fs (dm-1): re-mounted d5a34542-c1bc-4f6e-b5bc-efdf73e9f44a r/w. Quota mode: none.
[ 3.780479] systemd[1]: Finished systemd-remount-fs.service - Remount Root and Kernel File Systems.
[ 3.780616] systemd[1]: systemd-firstboot.service - First Boot Wizard was skipped because of an unmet condition check (ConditionFirstBoot=yes).
[ 3.780652] systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore).
[ 3.792801] spl: loading out-of-tree module taints kernel.
[ 3.798749] systemd[1]: Starting systemd-random-seed.service - Load/Save Random Seed...
[ 3.800297] systemd[1]: Starting systemd-sysusers.service - Create System Users...
[ 3.825020] systemd[1]: Started systemd-journald.service - Journal Service.
[ 3.846820] zfs: module license 'CDDL' taints kernel.
[ 3.846826] Disabling lock debugging due to kernel taint
[ 3.846847] zfs: module license taints kernel.
[ 3.854382] systemd-journald[439]: Received client request to flush runtime journal.
[ 4.489563] input: PC Speaker as /devices/platform/pcspkr/input/input4
[ 4.491651] ccp 0000:09:00.2: ccp: unable to access the device: you might be running a broken BIOS.
[ 4.492800] ccp 0000:09:00.2: tee enabled
[ 4.492885] ccp 0000:09:00.2: psp enabled
[ 4.542985] RAPL PMU: API unit is 2^-32 Joules, 1 fixed counters, 163840 ms ovfl timer
[ 4.542991] RAPL PMU: hw unit of domain package 2^-16 Joules
[ 4.555640] Adding 8388604k swap on /dev/mapper/pve-swap. Priority:-2 extents:1 across:8388604k SS
[ 4.650778] cryptd: max_cpu_qlen set to 1000
[ 4.659212] AVX2 version of gcm_enc/dec engaged.
[ 4.659309] AES CTR mode by8 optimization enabled
[ 4.668365] input: HD-Audio Generic HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:08.1/0000:09:00.1/sound/card0/input5
[ 4.668427] input: HD-Audio Generic HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:08.1/0000:09:00.1/sound/card0/input6
[ 4.668481] input: HD-Audio Generic HDMI/DP,pcm=8 as /devices/pci0000:00/0000:00:08.1/0000:09:00.1/sound/card0/input7
[ 4.668721] input: HD-Audio Generic HDMI/DP,pcm=9 as /devices/pci0000:00/0000:00:08.1/0000:09:00.1/sound/card0/input8
[ 4.705490] snd_hda_codec_realtek hdaudioC1D0: autoconfig for ALC897: line_outs=1 (0x14/0x0/0x0/0x0/0x0) type:line
[ 4.705500] snd_hda_codec_realtek hdaudioC1D0: speaker_outs=0 (0x0/0x0/0x0/0x0/0x0)
[ 4.705504] snd_hda_codec_realtek hdaudioC1D0: hp_outs=1 (0x1b/0x0/0x0/0x0/0x0)
[ 4.705508] snd_hda_codec_realtek hdaudioC1D0: mono: mono_out=0x0
[ 4.705511] snd_hda_codec_realtek hdaudioC1D0: inputs:
[ 4.705514] snd_hda_codec_realtek hdaudioC1D0: Front Mic=0x19
[ 4.705517] snd_hda_codec_realtek hdaudioC1D0: Rear Mic=0x18
[ 4.705520] snd_hda_codec_realtek hdaudioC1D0: Line=0x1a
[ 4.725526] input: HD-Audio Generic Front Mic as /devices/pci0000:00/0000:00:08.1/0000:09:00.6/sound/card1/input9
[ 4.725602] input: HD-Audio Generic Rear Mic as /devices/pci0000:00/0000:00:08.1/0000:09:00.6/sound/card1/input10
[ 4.725669] input: HD-Audio Generic Line as /devices/pci0000:00/0000:00:08.1/0000:09:00.6/sound/card1/input11
[ 4.725821] input: HD-Audio Generic Line Out as /devices/pci0000:00/0000:00:08.1/0000:09:00.6/sound/card1/input12
[ 4.725918] input: HD-Audio Generic Front Headphone as /devices/pci0000:00/0000:00:08.1/0000:09:00.6/sound/card1/input13
[ 5.424210] ZFS: Loaded module v2.2.4-pve1, ZFS pool version 5000, ZFS filesystem version 5
[ 7.016584] [drm] amdgpu kernel modesetting enabled.
[ 7.017063] amdgpu: Virtual CRAT table created for CPU
[ 7.017092] amdgpu: Topology: Add CPU node
[ 7.017277] [drm] initializing kernel modesetting (RENOIR 0x1002:0x1636 0x1565:0x170B 0xC9).
[ 7.017289] [drm] register mmio base: 0xFCC00000
[ 7.017290] [drm] register mmio size: 524288
[ 7.020299] [drm] add ip block number 0 <soc15_common>
[ 7.020301] [drm] add ip block number 1 <gmc_v9_0>
[ 7.020303] [drm] add ip block number 2 <vega10_ih>
[ 7.020304] [drm] add ip block number 3 <psp>
[ 7.020306] [drm] add ip block number 4 <smu>
[ 7.020307] [drm] add ip block number 5 <dm>
[ 7.020309] [drm] add ip block number 6 <gfx_v9_0>
[ 7.020311] [drm] add ip block number 7 <sdma_v4_0>
[ 7.020312] [drm] add ip block number 8 <vcn_v2_0>
[ 7.020314] [drm] add ip block number 9 <jpeg_v2_0>
[ 7.023471] kvm_amd: TSC scaling supported
[ 7.023475] kvm_amd: Nested Virtualization enabled
[ 7.023476] kvm_amd: Nested Paging enabled
[ 7.023492] kvm_amd: Virtual VMLOAD VMSAVE supported
[ 7.023493] kvm_amd: Virtual GIF supported
[ 7.023494] kvm_amd: LBR virtualization supported
[ 7.023763] [drm] BIOS signature incorrect 0 0
[ 7.023793] amdgpu 0000:09:00.0: amdgpu: Fetched VBIOS from ROM BAR
[ 7.023797] amdgpu: ATOM BIOS: 113-RENOIR-035
[ 7.037845] MCE: In-kernel MCE decoding enabled.
[ 7.041968] [drm] VCN decode is enabled in VM mode
[ 7.041971] [drm] VCN encode is enabled in VM mode
[ 7.043955] [drm] JPEG decode is enabled in VM mode
[ 7.049479] Console: switching to colour dummy device 80x25
[ 7.061976] amdgpu 0000:09:00.0: vgaarb: deactivate vga console
[ 7.061984] amdgpu 0000:09:00.0: amdgpu: Trusted Memory Zone (TMZ) feature enabled
[ 7.061989] amdgpu 0000:09:00.0: amdgpu: MODE2 reset
[ 7.062470] [drm] vm size is 262144 GB, 4 levels, block size is 9-bit, fragment size is 9-bit
[ 7.062480] amdgpu 0000:09:00.0: amdgpu: VRAM: 512M 0x000000F400000000 - 0x000000F41FFFFFFF (512M used)
[ 7.062484] amdgpu 0000:09:00.0: amdgpu: GART: 1024M 0x0000000000000000 - 0x000000003FFFFFFF
[ 7.062496] [drm] Detected VRAM RAM=512M, BAR=512M
[ 7.062498] [drm] RAM width 128bits DDR4
[ 7.062637] [drm] amdgpu: 512M of VRAM memory ready
[ 7.062640] [drm] amdgpu: 7684M of GTT memory ready.
[ 7.062657] [drm] GART: num cpu pages 262144, num gpu pages 262144
[ 7.062792] [drm] PCIE GART of 1024M enabled.
[ 7.062794] [drm] PTB located at 0x000000F41FC00000
[ 7.063265] [drm] Loading DMUB firmware via PSP: version=0x01010028
[ 7.063845] [drm] Found VCN firmware Version ENC: 1.21 DEC: 7 VEP: 0 Revision: 3
[ 7.063861] amdgpu 0000:09:00.0: amdgpu: Will use PSP to load VCN firmware
[ 7.066677] intel_rapl_common: Found RAPL domain package
[ 7.066682] intel_rapl_common: Found RAPL domain core
[ 7.748514] [drm] reserve 0x400000 from 0xf41f800000 for PSP TMR
[ 7.833793] amdgpu 0000:09:00.0: amdgpu: RAS: optional ras ta ucode is not available
[ 7.844575] amdgpu 0000:09:00.0: amdgpu: RAP: optional rap ta ucode is not available
[ 7.850575] [drm] psp gfx command LOAD_TA(0x1) failed and response status is (0x7)
[ 7.850800] [drm] psp gfx command INVOKE_CMD(0x3) failed and response status is (0x4)
[ 7.850802] amdgpu 0000:09:00.0: amdgpu: Secure display: Generic Failure.
[ 7.850804] amdgpu 0000:09:00.0: amdgpu: SECUREDISPLAY: query securedisplay TA failed. ret 0x0
[ 7.851110] amdgpu 0000:09:00.0: amdgpu: SMU is initialized successfully!
[ 7.851420] [drm] Display Core v3.2.266 initialized on DCN 2.1
[ 7.851422] [drm] DP-HDMI FRL PCON supported
[ 7.851961] [drm] DMUB hardware initialized: version=0x01010028
[ 7.872773] snd_hda_intel 0000:09:00.1: bound 0000:09:00.0 (ops amdgpu_dm_audio_component_bind_ops [amdgpu])
[ 7.875222] [drm] kiq ring mec 2 pipe 1 q 0
[ 7.878142] [drm] VCN decode and encode initialized successfully(under DPG Mode).
[ 7.878153] [drm] JPEG decode initialized successfully.
[ 7.890714] amdgpu: HMM registered 512MB device memory
[ 7.892450] kfd kfd: amdgpu: Allocated 3969056 bytes on gart
[ 7.892465] kfd kfd: amdgpu: Total number of KFD nodes to be created: 1
[ 7.892598] amdgpu: Virtual CRAT table created for GPU
[ 7.893115] amdgpu: Topology: Add dGPU node [0x1636:0x1002]
[ 7.893117] kfd kfd: amdgpu: added device 1002:1636
[ 7.893178] amdgpu 0000:09:00.0: amdgpu: SE 1, SH per SE 1, CU per SH 8, active_cu_number 7
[ 7.893183] amdgpu 0000:09:00.0: amdgpu: ring gfx uses VM inv eng 0 on hub 0
[ 7.893185] amdgpu 0000:09:00.0: amdgpu: ring gfx_low uses VM inv eng 1 on hub 0
[ 7.893187] amdgpu 0000:09:00.0: amdgpu: ring gfx_high uses VM inv eng 4 on hub 0
[ 7.893189] amdgpu 0000:09:00.0: amdgpu: ring comp_1.0.0 uses VM inv eng 5 on hub 0
[ 7.893191] amdgpu 0000:09:00.0: amdgpu: ring comp_1.1.0 uses VM inv eng 6 on hub 0
[ 7.893193] amdgpu 0000:09:00.0: amdgpu: ring comp_1.2.0 uses VM inv eng 7 on hub 0
[ 7.893194] amdgpu 0000:09:00.0: amdgpu: ring comp_1.3.0 uses VM inv eng 8 on hub 0
[ 7.893196] amdgpu 0000:09:00.0: amdgpu: ring comp_1.0.1 uses VM inv eng 9 on hub 0
[ 7.893198] amdgpu 0000:09:00.0: amdgpu: ring comp_1.1.1 uses VM inv eng 10 on hub 0
[ 7.893200] amdgpu 0000:09:00.0: amdgpu: ring comp_1.2.1 uses VM inv eng 11 on hub 0
[ 7.893202] amdgpu 0000:09:00.0: amdgpu: ring comp_1.3.1 uses VM inv eng 12 on hub 0
[ 7.893204] amdgpu 0000:09:00.0: amdgpu: ring kiq_0.2.1.0 uses VM inv eng 13 on hub 0
[ 7.893206] amdgpu 0000:09:00.0: amdgpu: ring sdma0 uses VM inv eng 0 on hub 8
[ 7.893208] amdgpu 0000:09:00.0: amdgpu: ring vcn_dec uses VM inv eng 1 on hub 8
[ 7.893210] amdgpu 0000:09:00.0: amdgpu: ring vcn_enc0 uses VM inv eng 4 on hub 8
[ 7.893212] amdgpu 0000:09:00.0: amdgpu: ring vcn_enc1 uses VM inv eng 5 on hub 8
[ 7.893214] amdgpu 0000:09:00.0: amdgpu: ring jpeg_dec uses VM inv eng 6 on hub 8
[ 7.894693] [drm] Initialized amdgpu 3.57.0 20150101 for 0000:09:00.0 on minor 1
[ 7.896507] amdgpu 0000:09:00.0: [drm] Cannot find any crtc or sizes
[ 7.896523] [drm] DSC precompute is not needed.
[ 8.093472] audit: type=1400 audit(1719843370.173:2): apparmor="STATUS" operation="profile_load" profile="unconfined" name="pve-container-mounthotplug" pid=720 comm="apparmor_parser"
[ 8.093529] audit: type=1400 audit(1719843370.173:3): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/bin/lxc-start" pid=723 comm="apparmor_parser"
[ 8.093543] audit: type=1400 audit(1719843370.173:4): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/bin/lxc-copy" pid=722 comm="apparmor_parser"
[ 8.094740] audit: type=1400 audit(1719843370.174:5): apparmor="STATUS" operation="profile_load" profile="unconfined" name="swtpm" pid=725 comm="apparmor_parser"
[ 8.097609] audit: type=1400 audit(1719843370.177:6): apparmor="STATUS" operation="profile_load" profile="unconfined" name="nvidia_modprobe" pid=719 comm="apparmor_parser"
[ 8.097615] audit: type=1400 audit(1719843370.177:7): apparmor="STATUS" operation="profile_load" profile="unconfined" name="nvidia_modprobe//kmod" pid=719 comm="apparmor_parser"
[ 8.097976] audit: type=1400 audit(1719843370.177:8): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lsb_release" pid=717 comm="apparmor_parser"
[ 8.100133] audit: type=1400 audit(1719843370.179:9): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/bin/man" pid=724 comm="apparmor_parser"
[ 8.100137] audit: type=1400 audit(1719843370.179:10): apparmor="STATUS" operation="profile_load" profile="unconfined" name="man_filter" pid=724 comm="apparmor_parser"
[ 8.100140] audit: type=1400 audit(1719843370.179:11): apparmor="STATUS" operation="profile_load" profile="unconfined" name="man_groff" pid=724 comm="apparmor_parser"
[ 8.122246] RPC: Registered named UNIX socket transport module.
[ 8.122250] RPC: Registered udp transport module.
[ 8.122252] RPC: Registered tcp transport module.
[ 8.122253] RPC: Registered tcp-with-tls transport module.
[ 8.122254] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 8.175421] softdog: initialized. soft_noboot=0 soft_margin=60 sec soft_panic=0 (nowayout=0)
[ 8.175426] softdog: soft_reboot_cmd=<not set> soft_active_on_boot=0
[ 9.021479] vmbr0: port 1(enp5s0) entered blocking state
[ 9.021484] vmbr0: port 1(enp5s0) entered disabled state
[ 9.021500] r8169 0000:05:00.0 enp5s0: entered allmulticast mode
[ 9.021560] r8169 0000:05:00.0 enp5s0: entered promiscuous mode
[ 9.051399] Generic FE-GE Realtek PHY r8169-0-500:00: attached PHY driver (mii_bushy_addr=r8169-0-500:00, irq=MAC)
[ 9.215758] r8169 0000:05:00.0 enp5s0: Link is Down
[ 9.219249] vmbr0: port 1(enp5s0) entered blocking state
[ 9.219256] vmbr0: port 1(enp5s0) entered forwarding state
[ 10.044427] vmbr0: port 1(enp5s0) entered disabled state
[ 11.950042] r8169 0000:05:00.0 enp5s0: Link is Up - 1Gbps/Full - flow control off
[ 11.950064] vmbr0: port 1(enp5s0) entered blocking state
[ 11.950067] vmbr0: port 1(enp5s0) entered forwarding state
[ 15.295585] tap102i0: entered promiscuous mode
[ 15.357734] vmbr0: port 2(fwpr102p0) entered blocking state
[ 15.357741] vmbr0: port 2(fwpr102p0) entered disabled state
[ 15.357765] fwpr102p0: entered allmulticast mode
[ 15.357819] fwpr102p0: entered promiscuous mode
[ 15.357870] vmbr0: port 2(fwpr102p0) entered blocking state
[ 15.357873] vmbr0: port 2(fwpr102p0) entered forwarding state
[ 15.369303] fwbr102i0: port 1(fwln102i0) entered blocking state
[ 15.369310] fwbr102i0: port 1(fwln102i0) entered disabled state
[ 15.369335] fwln102i0: entered allmulticast mode
[ 15.369413] fwln102i0: entered promiscuous mode
[ 15.369470] fwbr102i0: port 1(fwln102i0) entered blocking state
[ 15.369473] fwbr102i0: port 1(fwln102i0) entered forwarding state
[ 15.380875] fwbr102i0: port 2(tap102i0) entered blocking state
[ 15.380882] fwbr102i0: port 2(tap102i0) entered disabled state
[ 15.380896] tap102i0: entered allmulticast mode
[ 15.380966] fwbr102i0: port 2(tap102i0) entered blocking state
[ 15.380969] fwbr102i0: port 2(tap102i0) entered forwarding state
[ 15.428934] kvm: SMP vm created on host with unstable TSC; guest TSC will not be reliable

 

[fgrazioli]

Jul 01 12:40:03 proxmox pvestatd[1076]: qemu status update error: metrics send error 'Proxmox': failed to send metrics: Connection refused
Jul 01 12:40:13 proxmox pvestatd[1076]: node status update error: metrics send error 'Proxmox': failed to send metrics: Connection refused
Jul 01 12:40:13 proxmox pvestatd[1076]: qemu status update error: metrics send error 'Proxmox': failed to send metrics: Connection refused
Jul 01 12:40:13 proxmox pvestatd[1076]: command '/sbin/vgs --separator : --noheadings --units b --unbuffered --nosuffix --options vg_name,vg_size,vg_free,lv_count' failed: open3: pipe(GLOB(0x638e41c06c78), GLOB(0x638e41c3c4f0)) failed: Too many open files at /usr/share/perl5/PVE/Tools.pm line 494.
Jul 01 12:40:13 proxmox pvestatd[1076]: command '/sbin/vgscan --ignorelockingfailure --mknodes' failed: open3: pipe(GLOB(0x638e41c32818), GLOB(0x638e41c31e78)) failed: Too many open files at /usr/share/perl5/PVE/Tools.pm line 494.
Jul 01 12:40:13 proxmox pvestatd[1076]: command '/sbin/lvs --separator : --noheadings --units b --unbuffered --nosuffix --config 'report/time_format="%s"' --options vg_name,lv_name,lv_size,lv_attr,pool_lv,data_percent,metadata_percent,snap_percent,uuid,tags,metadata_size,time' failed: open3: pipe(GLOB(0x638e41c3b6a0), GLOB(0x638e41c0e088)) failed: Too many open files at /usr/share/perl5/PVE/Tools.pm line 494.
Jul 01 12:40:23 proxmox pvestatd[1076]: node status update error: metrics send error 'Proxmox': failed to send metrics: Connection refused
Jul 01 12:40:23 proxmox pvestatd[1076]: qemu status update error: metrics send error 'Proxmox': failed to send metrics: Connection refused
Jul 01 12:40:23 proxmox pvestatd[1076]: command '/sbin/vgs --separator : --noheadings --units b --unbuffered --nosuffix --options vg_name,vg_size,vg_free,lv_count' failed: open3: pipe(IO::File=GLOB(0x638e41c38420), GLOB(0x638e41c4aa80)) failed: Too many open files at /usr/share/perl5/PVE/Tools.pm line 494.
Jul 01 12:40:23 proxmox pvestatd[1076]: command '/sbin/vgscan --ignorelockingfailure --mknodes' failed: open3: pipe(IO::File=GLOB(0x638e41c31e78), GLOB(0x638e41c32ba8)) failed: Too many open files at /usr/share/perl5/PVE/Tools.pm line 494.
Jul 01 12:40:23 proxmox pvestatd[1076]: command '/sbin/lvs --separator : --noheadings --units b --unbuffered --nosuffix --config 'report/time_format="%s"' --options vg_name,lv_name,lv_size,lv_attr,pool_lv,data_percent,metadata_percent,snap_percent,uuid,tags,metadata_size,time' failed: open3: pipe(IO::File=GLOB(0x638e41c4a930), GLOB(0x638e41c3c868)) failed: Too many open files at /usr/share/perl5/PVE/Tools.pm line 494.
Jul 01 12:40:33 proxmox pvestatd[1076]: node status update error: metrics send error 'Proxmox': failed to send metrics: Connection refused
Jul 01 12:40:33 proxmox pvestatd[1076]: qemu status update error: metrics send error 'Proxmox': failed to send metrics: Connection refused
Jul 01 12:40:33 proxmox pvestatd[1076]: command '/sbin/vgs --separator : --noheadings --units b --unbuffered --nosuffix --options vg_name,vg_size,vg_free,lv_count' failed: open3: pipe(IO::File=GLOB(0x638e41c4b0b0), GLOB(0x638e41c43828)) failed: Too many open files at /usr/share/perl5/PVE/Tools.pm line 494.
Jul 01 12:40:33 proxmox pvestatd[1076]: command '/sbin/vgscan --ignorelockingfailure --mknodes' failed: open3: pipe(IO::File=GLOB(0x638e41c3be68), GLOB(0x638e41c3d180)) failed: Too many open files at /usr/share/perl5/PVE/Tools.pm line 494.
Jul 01 12:40:33 proxmox pvestatd[1076]: command '/sbin/lvs --separator : --noheadings --units b --unbuffered --nosuffix --config 'report/time_format="%s"' --options vg_name,lv_name,lv_size,lv_attr,pool_lv,data_percent,metadata_percent,snap_percent,uuid,tags,metadata_size,time' failed: open3: pipe(IO::File=GLOB(0x638e41c32530), GLOB(0x638e41c3cee0)) failed: Too many open files at /usr/share/perl5/PVE/Tools.pm line 494.
Jul 01 12:40:43 proxmox pvestatd[1076]: node status update error: metrics send error 'Proxmox': failed to send metrics: Connection refused
Jul 01 12:40:43 proxmox pvestatd[1076]: qemu status update error: metrics send error 'Proxmox': failed to send metrics: Connection refused
Jul 01 12:40:43 proxmox pvestatd[1076]: command '/sbin/vgs --separator : --noheadings --units b --unbuffered --nosuffix --options vg_name,vg_size,vg_free,lv_count' failed: open3: pipe(GLOB(0x638e41c3ba48), IO::File=GLOB(0x638e41c38528)) failed: Too many open files at /usr/share/perl5/PVE/Tools.pm line 494.
Jul 01 12:40:43 proxmox pvestatd[1076]: command '/sbin/vgscan --ignorelockingfailure --mknodes' failed: open3: pipe(GLOB(0x638e41c31e78), IO::File=GLOB(0x638e41c3bce8)) failed: Too many open files at /usr/share/perl5/PVE/Tools.pm line 494.
Jul 01 12:40:43 proxmox pvestatd[1076]: command '/sbin/lvs --separator : --noheadings --units b --unbuffered --nosuffix --config 'report/time_format="%s"' --options vg_name,lv_name,lv_size,lv_attr,pool_lv,data_percent,metadata_percent,snap_percent,uuid,tags,metadata_size,time' failed: open3: pipe(GLOB(0x638e41c3bc40), IO::File=GLOB(0x638e41c58040)) failed: Too many open files at /usr/share/perl5/PVE/Tools.pm line 494.
Jul 01 12:40:43 proxmox pvestatd[1076]: Can't call method "reader" on an undefined value at /usr/share/perl5/PVE/Tools.pm line 1046.
Jul 01 12:40:43 proxmox pvestatd[12698]: Can't call method "writer" on an undefined value at /usr/share/perl5/PVE/Tools.pm line 1025.
Jul 01 12:40:43 proxmox pvestatd[1076]: lxc console cleanup error: file '/proc/1/cmdline' exists but open for reading failed - Too many open files
Jul 01 12:40:43 proxmox pvestatd[12698]: lxc console cleanup error: file '/proc/1/cmdline' exists but open for reading failed - Too many open files
Jul 01 12:40:43 proxmox pvestatd[12698]: ipcc_send_rec[4] failed: Too many open files
Jul 01 12:40:53 proxmox pvestatd[1076]: node status update error: metrics send error 'Proxmox': failed to send metrics: Connection refused
Jul 01 12:40:53 proxmox pvestatd[1076]: qemu status update error: file '/proc/uptime' exists but open for reading failed - Too many open files
Jul 01 12:40:53 proxmox pvestatd[1076]: lxc status update error: file '/proc/uptime' exists but open for reading failed - Too many open files
Jul 01 12:40:53 proxmox pvestatd[1076]: lxc cpuset rebalance error: file '/sys/fs/cgroup/cpuset.cpus.effective' exists but open for reading failed - Too many open files
Jul 01 12:40:53 proxmox pvestatd[12698]: ipcc_send_rec[1] failed: Too many open files
Jul 01 12:40:53 proxmox pvestatd[1076]: Can't call method "reader" on an undefined value at /usr/share/perl5/PVE/Tools.pm line 1046.
Jul 01 12:40:53 proxmox pvestatd[12716]: Can't call method "writer" on an undefined value at /usr/share/perl5/PVE/Tools.pm line 1025.
Jul 01 12:40:53 proxmox pvestatd[12698]: ipcc_send_rec[2] failed: Too many open files
Jul 01 12:40:53 proxmox pvestatd[12716]: command '/sbin/vgs --separator : --noheadings --units b --unbuffered --nosuffix --options vg_name,vg_size,vg_free,lv_count' failed: open3: pipe(GLOB(0x638e41c14c78), IO::File=GLOB(0x638e41c16400)) failed: Too many open files at /usr/share/perl5/PVE/Tools.pm line 494.
Jul 01 12:40:53 proxmox pvestatd[1076]: command '/sbin/vgs --separator : --noheadings --units b --unbuffered --nosuffix --options vg_name,vg_size,vg_free,lv_count' failed: open3: pipe(GLOB(0x638e41c14c78), IO::File=GLOB(0x638e41c16400)) failed: Too many open files at /usr/share/perl5/PVE/Tools.pm line 494.
Jul 01 12:40:53 proxmox pvestatd[12716]: command '/sbin/vgscan --ignorelockingfailure --mknodes' failed: open3: pipe(GLOB(0x638e41c4aff0), IO::File=GLOB(0x638e41c3c670)) failed: Too many open files at /usr/share/perl5/PVE/Tools.pm line 494.
Jul 01 12:40:53 proxmox pvestatd[1076]: command '/sbin/vgscan --ignorelockingfailure --mknodes' failed: open3: pipe(GLOB(0x638e41c4aff0), IO::File=GLOB(0x638e41c3c670)) failed: Too many open files at /usr/share/perl5/PVE/Tools.pm line 494.
Jul 01 12:40:53 proxmox pvestatd[12716]: command '/sbin/lvs --separator : --noheadings --units b --unbuffered --nosuffix --config 'report/time_format="%s"' --options vg_name,lv_name,lv_size,lv_attr,pool_lv,data_percent,metadata_percent,snap_percent,uuid,tags,metadata_size,time' failed: open3: pipe(GLOB(0x638e415f3d08), IO::File=GLOB(0x638e41c3dcf8)) failed: Too many open files at /usr/share/perl5/PVE/Tools.pm line 494.
Jul 01 12:40:53 proxmox pvestatd[1076]: command '/sbin/lvs --separator : --noheadings --units b --unbuffered --nosuffix --config 'report/time_format="%s"' --options vg_name,lv_name,lv_size,lv_attr,pool_lv,data_percent,metadata_percent,snap_percent,uuid,tags,metadata_size,time' failed: open3: pipe(GLOB(0x638e415f3d08), IO::File=GLOB(0x638e41c3dcf8)) failed: Too many open files at /usr/share/perl5/PVE/Tools.pm line 494.
Jul 01 12:40:53 proxmox pvestatd[12716]: storage status update error: Unrecognised protocol udp at /usr/share/perl5/PVE/Status/Graphite.pm line 104.
Jul 01 12:40:53 proxmox pvestatd[1076]: storage status update error: Unrecognised protocol udp at /usr/share/perl5/PVE/Status/Graphite.pm line 104.
Jul 01 12:40:53 proxmox pvestatd[12716]: lxc console cleanup error: file '/proc/uptime' exists but open for reading failed - Too many open files
Jul 01 12:40:53 proxmox pvestatd[1076]: lxc console cleanup error: file '/proc/uptime' exists but open for reading failed - Too many open files
Jul 01 12:40:53 proxmox pvestatd[12716]: sdn status update error: file '/etc/network/interfaces.d/sdn' exists but open for reading failed - Too many open files
Jul 01 12:40:53 proxmox pvestatd[1076]: sdn status update error: file '/etc/network/interfaces.d/sdn' exists but open for reading failed - Too many open files
Jul 01 12:40:53 proxmox pvestatd[12716]: ERROR: file '/proc/12716/statm' exists but open for reading failed - Too many open files
Jul 01 12:40:53 proxmox pvestatd[1076]: ERROR: file '/proc/1076/statm' exists but open for reading failed - Too many open files
Jul 01 12:40:53 proxmox pvestatd[1076]: server closing
Jul 01 12:40:53 proxmox pvestatd[12716]: server closing
Jul 01 12:40:53 proxmox pvestatd[12716]: server stopped
Jul 01 12:40:53 proxmox pvestatd[1076]: server stopped
Jul 01 12:40:53 proxmox pvestatd[12698]: ipcc_send_rec[3] failed: Too many open files
Jul 01 12:40:53 proxmox pvestatd[12698]: got inotify poll request in wrong process - disabling inotify
Jul 01 12:40:53 proxmox pvestatd[12698]: ipcc_send_rec[4] failed: Too many open files
Jul 01 12:40:53 proxmox pvestatd[12698]: status update error: Too many open files
Jul 01 12:40:54 proxmox pvestatd[12698]: received signal TERM
Jul 01 12:40:54 proxmox pvestatd[12698]: server closing
Jul 01 12:40:54 proxmox pvestatd[12698]: server stopped
Jul 01 12:40:54 proxmox systemd[1]: pvestatd.service: Deactivated successfully.
Jul 01 12:40:54 proxmox systemd[1]: pvestatd.service: Consumed 31.993s CPU time.
Jul 01 12:41:23 proxmox pvedaemon[1097]: <root@pam> successful auth for user 'root@pam'
Jul 01 12:45:10 proxmox pveproxy[11120]: worker exit
Jul 01 12:45:10 proxmox pveproxy[1875]: worker 11120 finished
Jul 01 12:45:10 proxmox pveproxy[1875]: starting 1 worker(s)
Jul 01 12:45:10 proxmox pveproxy[1875]: worker 13192 started
Jul 01 12:47:51 proxmox pvedaemon[1095]: <root@pam> successful auth for user 'root@pam'
Jul 01 12:48:40 proxmox pveproxy[10009]: worker exit
Jul 01 12:48:40 proxmox pveproxy[1875]: worker 10009 finished
Jul 01 12:48:40 proxmox pveproxy[1875]: starting 1 worker(s)
Jul 01 12:48:40 proxmox pveproxy[1875]: worker 13576 started
Jul 01 12:56:17 proxmox pvedaemon[14418]: starting termproxy UPIDroxmox:00003852:00092DAD:6682D1A1:vncshell::root@pam:
Jul 01 12:56:17 proxmox pvedaemon[1095]: <root@pam> starting task UPIDroxmox:00003852:00092DAD:6682D1A1:vncshell::root@pam:
Jul 01 12:56:17 proxmox pvedaemon[1097]: <root@pam> successful auth for user 'root@pam'
Jul 01 12:56:17 proxmox login[14421]: pam_unix(login:session): session opened for user root(uid=0) by root(uid=0)
Jul 01 12:56:17 proxmox systemd[1]: Created slice user-0.slice - User Slice of UID 0.
Jul 01 12:56:17 proxmox systemd[1]: Starting user-runtime-dir@0.service - User Runtime Directory /run/user/0...
Jul 01 12:56:17 proxmox systemd-logind[746]: New session 7 of user root.
Jul 01 12:56:17 proxmox systemd[1]: Finished user-runtime-dir@0.service - User Runtime Directory /run/user/0.
Jul 01 12:56:17 proxmox systemd[1]: Starting user@0.service - User Manager for UID 0...
Jul 01 12:56:17 proxmox (systemd)[14427]: pam_unix(systemd-user:session): session opened for user root(uid=0) by (uid=0)
Jul 01 12:56:17 proxmox systemd[14427]: Queued start job for default target default.target.
Jul 01 12:56:17 proxmox systemd[14427]: Created slice app.slice - User Application Slice.
Jul 01 12:56:17 proxmox systemd[14427]: Reached target paths.target - Paths.
Jul 01 12:56:17 proxmox systemd[14427]: Reached target timers.target - Timers.
Jul 01 12:56:17 proxmox systemd[14427]: Listening on dirmngr.socket - GnuPG network certificate management daemon.
Jul 01 12:56:17 proxmox systemd[14427]: Listening on gpg-agent-browser.socket - GnuPG cryptographic agent and passphrase cache (access for web browsers).
Jul 01 12:56:17 proxmox systemd[14427]: Listening on gpg-agent-extra.socket - GnuPG cryptographic agent and passphrase cache (restricted).
Jul 01 12:56:17 proxmox systemd[14427]: Listening on gpg-agent-ssh.socket - GnuPG cryptographic agent (ssh-agent emulation).
Jul 01 12:56:17 proxmox systemd[14427]: Listening on gpg-agent.socket - GnuPG cryptographic agent and passphrase cache.
Jul 01 12:56:17 proxmox systemd[14427]: Reached target sockets.target - Sockets.
Jul 01 12:56:17 proxmox systemd[14427]: Reached target basic.target - Basic System.
Jul 01 12:56:17 proxmox systemd[14427]: Reached target default.target - Main User Target.
Jul 01 12:56:17 proxmox systemd[14427]: Startup finished in 101ms.
Jul 01 12:56:17 proxmox systemd[1]: Started user@0.service - User Manager for UID 0.
Jul 01 12:56:17 proxmox systemd[1]: Started session-7.scope - Session 7 of User root.
Jul 01 12:56:17 proxmox login[14443]: ROOT LOGIN on '/dev/pts/0'
Jul 01 12:56:19 proxmox systemd[1]: session-7.scope: Deactivated successfully.
Jul 01 12:56:19 proxmox systemd-logind[746]: Session 7 logged out. Waiting for processes to exit.
Jul 01 12:56:19 proxmox systemd-logind[746]: Removed session 7.
Jul 01 12:56:19 proxmox pvedaemon[1095]: <root@pam> end task UPIDroxmox:00003852:00092DAD:6682D1A1:vncshell::root@pam: OK
Jul 01 12:56:23 proxmox pvedaemon[1096]: <root@pam> successful auth for user 'root@pam'