Wondering why Proxmox disconnect every night (almost)

[teokoul]

Hello there,

I am wondering why proxmox disconnect at nights or evenings.

The only clue I have is the log below :

At Sep 13 16:30:00 is the last log.

At Sep 14 09:22:39 (today morning) is the time that I press the button to close the tower and press it again to re-open it.

Sep 13 16:25:00 pve systemd[1]: Started Proxmox VE replication runner.
Sep 13 16:26:00 pve systemd[1]: Starting Proxmox VE replication runner...
Sep 13 16:26:00 pve systemd[1]: pvesr.service: Succeeded.
Sep 13 16:26:00 pve systemd[1]: Started Proxmox VE replication runner.
Sep 13 16:27:00 pve systemd[1]: Starting Proxmox VE replication runner...
Sep 13 16:27:00 pve systemd[1]: pvesr.service: Succeeded.
Sep 13 16:27:00 pve systemd[1]: Started Proxmox VE replication runner.
Sep 13 16:28:00 pve systemd[1]: Starting Proxmox VE replication runner...
Sep 13 16:28:00 pve systemd[1]: pvesr.service: Succeeded.
Sep 13 16:28:00 pve systemd[1]: Started Proxmox VE replication runner.
Sep 13 16:29:00 pve systemd[1]: Starting Proxmox VE replication runner...
Sep 13 16:29:00 pve systemd[1]: pvesr.service: Succeeded.
Sep 13 16:29:00 pve systemd[1]: Started Proxmox VE replication runner.
Sep 13 16:30:00 pve systemd[1]: Starting Proxmox VE replication runner...
Sep 13 16:30:00 pve systemd[1]: pvesr.service: Succeeded.
Sep 13 16:30:00 pve systemd[1]: Started Proxmox VE replication runner.
Sep 14 09:22:39 pve kernel: [    0.000000] Linux version 5.4.73-1-pve (build@pve) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP PVE 5.4.73-1 (Mon, 16 Nov 2020 10:52:16 +01$
Sep 14 09:22:39 pve systemd-modules-load[508]: Inserted module 'vfio'
Sep 14 09:22:39 pve kernel: [    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.73-1-pve root=/dev/mapper/pve-root ro quiet amd_iommu=on pcie_acs_override=downstre$
Sep 14 09:22:39 pve kernel: [    0.000000] KERNEL supported cpus:
Sep 14 09:22:39 pve kernel: [    0.000000]   Intel GenuineIntel
Sep 14 09:22:39 pve kernel: [    0.000000]   AMD AuthenticAMD
Sep 14 09:22:39 pve blkmapd[525]: open pipe file /run/rpc_pipefs/nfs/blocklayout failed: No such file or directory
Sep 14 09:22:39 pve kernel: [    0.000000]   Hygon HygonGenuine
Sep 14 09:22:39 pve kernel: [    0.000000]   Centaur CentaurHauls
Sep 14 09:22:39 pve kernel: [    0.000000]   zhaoxin   Shanghai 
Sep 14 09:22:39 pve dmeventd[526]: dmeventd ready for processing.
Sep 14 09:22:39 pve kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
Sep 14 09:22:39 pve lvm[499]:   6 logical volume(s) in volume group "SSD_VMs" monitored
Sep 14 09:22:39 pve lvm[499]:   6 logical volume(s) in volume group "pve" monitored
Sep 14 09:22:39 pve lvm[526]: Monitoring thin pool pve-data-tpool.

 

[Moayad]

Is your Proxmox VE installed on a laptop?

Could you post the output of the following commands:

cat /etc/systemd/sleep.conf
cat /etc/systemd/logind.conf

 

[teokoul]

Is your Proxmox VE installed on a laptop?

Could you post the output of the following commands:

cat /etc/systemd/sleep.conf
cat /etc/systemd/logind.conf

Hello Moayad,

Thank you for the reply!

----------------------------------------

No, the proxmox VE isn't installed in a laptop.

My Build is :

MOBO : GA-AB350-GAMING-3
CPU: AMD RYZEN 7 1800X
GPU1: GTX 1050 Ti
GPU2: GT 1030
RAM : 64GB
NVME : SAMSUNG 860 EVO NVme (FOR PROXMOX)
SSD : SAMSUNG 870 EVO (FOR VM)
HDD: 2 x 4TB TOSHIBA (FOR FREENAS STORAGE ALREADY SETUP)

----------------------------------------

You can see the outputs of the sleep.conf and logind.conf.

[Sleep]
#AllowSuspend=yes
#AllowHibernation=yes
#AllowSuspendThenHibernate=yes
#AllowHybridSleep=yes
#SuspendMode=
#SuspendState=mem standby freeze
#HibernateMode=platform shutdown
#HibernateState=disk
#HybridSleepMode=suspend platform shutdown
#HybridSleepState=disk
#HibernateDelaySec=180min

[Login]
#NAutoVTs=6
#ReserveVT=6
#KillUserProcesses=no
#KillOnlyUsers=
#KillExcludeUsers=root
#InhibitDelayMaxSec=5
#HandlePowerKey=poweroff
#HandleSuspendKey=suspend
#HandleHibernateKey=hibernate
#HandleLidSwitch=suspend
#HandleLidSwitchExternalPower=suspend
#HandleLidSwitchDocked=ignore
#PowerKeyIgnoreInhibited=no
#SuspendKeyIgnoreInhibited=no
#HibernateKeyIgnoreInhibited=no
#LidSwitchIgnoreInhibited=yes
#HoldoffTimeoutSec=30s
#IdleAction=ignore
#IdleActionSec=30min
#RuntimeDirectorySize=10%
#RemoveIPC=yes
#InhibitorsMax=8192
#SessionsMax=8192

I have a feeling that this is gonna solved. But I don't want to mess around with those files.

 

[Moayad]

Please try to edit the /etc/systemd/logind.conf file and set the HandleLidSwitch=ignore then reboot the host and see if that help

 

[t.lamprecht]

Sep 13 16:30:00 pve systemd[1]: pvesr.service: Succeeded.
Sep 13 16:30:00 pve systemd[1]: Started Proxmox VE replication runner.
Sep 14 09:22:39 pve kernel: [ 0.000000] Linux version 5.4.73-1-pve (build@pve) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP PVE 5.4.73-1 (Mon, 16 Nov 2020 10:52:16 +01$
Sep 14 09:22:39 pve systemd-modules-load[508]: Inserted module 'vfio'

Seems like there's a full reset going on...

I'd first update to latest Proxmox VE 6.4 updates, the kernel you boot is quite a bit outdated. If you see no pending updates then ensure that you got a valid PVE package repository setup that you can access: https://pve.proxmox.com/wiki/Package_repositories

If it still happens it'd be interesting to see if we can get more info out from the logs, as they seem to be truncated due to the reset. You could try to connect via SSH and run journalctl -f there overnight, if it resets again we hopefully have more log send over SSH than the PVE host could save on the machine and can narrow the actual issue better down.

 

[teokoul]

Seems like there's a full reset going on...

I'd first update to latest Proxmox VE 6.4 updates, the kernel you boot is quite a bit outdated. If you see no pending updates then ensure that you got a valid PVE package repository setup that you can access: https://pve.proxmox.com/wiki/Package_repositories

If it still happens it'd be interesting to see if we can get more info out from the logs, as they seem to be truncated due to the reset. You could try to connect via SSH and run journalctl -f there overnight, if it resets again we hopefully have more log send over SSH than the PVE host could save on the machine and can narrow the actual issue better down.

Please try to edit the /etc/systemd/logind.conf file and set the HandleLidSwitch=ignore then reboot the host and see if that help

I decide to reply to you when I collect more information about the crashes of the server.

**I would like to remind you that the server closed at nights and the morning when I want to work on the VMs I don't have connection. (The tower is open but there is no connection)**

The updated situation :

1) Like Moayad advised I try to edit the /etc/systemd/logind.conf and set the HandleLidSwitch=ignore .
2) I made some updates and upgrades on my system.

For 2 days I don't have any disconnection.
At Saturday I lost the connection.

Like t.lamprecht advised I have connected via SSH and run journalctl -f there overnight and over weekend.

The logs is attached here as files because they are many lines and I can't to upload them within the reply-post.

**Today morning I was able to ping or connect via SSH to the server.

--------------------------------------------------------

Thank you for your time.

 

[robocon]

I decide to reply to you when I collect more information about the crashes of the server.

**I would like to remind you that the server closed at nights and the morning when I want to work on the VMs I don't have connection. (The tower is open but there is no connection)**

The updated situation :

1) Like Moayad advised I try to edit the /etc/systemd/logind.conf and set the HandleLidSwitch=ignore .
2) I made some updates and upgrades on my system.

For 2 days I don't have any disconnection.
At Saturday I lost the connection.

Like t.lamprecht advised I have connected via SSH and run journalctl -f there overnight and over weekend.

The logs is attached here as files because they are many lines and I can't to upload them within the reply-post.

**Today morning I was able to ping or connect via SSH to the server.

--------------------------------------------------------

Thank you for your time.

Hello @teokoul
Did you manage to solve the issue?