PVEPROXY - fails

Harlin

Active Member
Dec 19, 2019
4
0
41
39
The WebGui keeps failing every now and again i have tried to restart pveproxy using "systemctl restart pveproxy" and there is no luck.

Previously the browser would state "Performing TLS negotiation" and then states timeout

is there a way to restart PVEPROXY - this seems to be a repeating problem

tserver: /var/lib/vz# systemctl status pveproxy.service
● pveproxy.service - PVE API Proxy Server
Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled; vendor preset: enabled)
Active: deactivating (final-sigkill) (Result: timeout) since Thu 2020-01-23 15:19:42 CAT; 19min ago
Process: 46914 ExecStartPre=/usr/bin/pvecm updatecerts --silent (code=killed, signal=TERM)
Tasks: 1 (limit: 9830)
Memory: 75.1M
CGroup: /system.slice/pveproxy.service
└─6312 /usr/bin/perl -T /usr/bin/pveproxy stop

Jan 23 15:36:15 evp systemd[1]: Starting PVE API Proxy Server...
Jan 23 15:37:45 evp systemd[1]: pveproxy.service: Start-pre operation timed out. Terminating.
Jan 23 15:37:45 evp systemd[1]: pveproxy.service: Control process exited, code=killed, status=15/TERM
Jan 23 15:39:16 evp systemd[1]: pveproxy.service: State 'stop-final-sigterm' timed out. Killing.
Jan 23 15:39:16 evp systemd[1]: pveproxy.service: Killing process 6312 (pveproxy) with signal SIGKILL.
 
some additional information


journalctl -u pveproxy
Jan 22 11:25:03 evp pveproxy[3059]: worker 39695 finished
Jan 22 11:25:03 evp pveproxy[3059]: starting 1 worker(s)
Jan 22 11:25:03 evp pveproxy[3059]: worker 59032 started
Jan 22 11:27:37 evp pveproxy[51311]: worker exit
Jan 22 11:27:37 evp pveproxy[3059]: worker 51311 finished
Jan 22 11:27:37 evp pveproxy[3059]: starting 1 worker(s)
Jan 22 11:27:37 evp pveproxy[3059]: worker 9239 started
Jan 22 11:35:18 evp pveproxy[33448]: worker exit
Jan 22 11:53:50 evp pveproxy[3059]: worker 33449 finished
Jan 22 11:53:50 evp pveproxy[3059]: starting 1 worker(s)
Jan 22 11:53:50 evp pveproxy[3059]: worker 42723 started
Jan 22 11:53:50 evp pveproxy[42722]: got inotify poll request in wrong process - disabling inotify
Jan 22 11:55:46 evp pveproxy[59032]: worker exit
Jan 22 11:55:46 evp pveproxy[3059]: worker 59032 finished
Jan 22 11:55:46 evp pveproxy[3059]: starting 1 worker(s)
Jan 22 11:55:46 evp pveproxy[3059]: worker 49971 started
Jan 22 11:58:52 evp pveproxy[9239]: worker exit
Jan 22 11:58:52 evp pveproxy[3059]: worker 9239 finished
Jan 22 11:58:52 evp pveproxy[3059]: starting 1 worker(s)
Jan 22 11:58:52 evp pveproxy[3059]: worker 32396 started
Jan 22 12:19:23 evp pveproxy[42723]: worker exit
Jan 22 12:19:23 evp pveproxy[3059]: worker 42723 finished
Jan 22 12:19:23 evp pveproxy[3059]: starting 1 worker(s)
Jan 22 12:19:23 evp pveproxy[3059]: worker 49894 started
Jan 22 12:26:51 evp pveproxy[32396]: worker exit
Jan 23 10:44:56 evp pveproxy[3059]: worker 13679 finished
Jan 23 10:44:56 evp pveproxy[3059]: starting 1 worker(s)
Jan 23 10:44:56 evp pveproxy[3059]: worker 54908 started
Jan 23 11:01:52 evp pveproxy[13681]: received unhandled websocket opcode 9
Jan 23 11:01:57 evp pveproxy[13681]: worker exit
Jan 23 11:01:57 evp pveproxy[3059]: worker 13681 finished
Jan 23 11:01:57 evp pveproxy[3059]: starting 1 worker(s)
Jan 23 11:01:57 evp pveproxy[3059]: worker 42480 started
Jan 23 11:02:12 evp pveproxy[13680]: received unhandled websocket opcode 9
Jan 23 11:02:17 evp pveproxy[13680]: worker exit
Jan 23 11:02:17 evp pveproxy[3059]: worker 13680 finished
Jan 23 11:02:17 evp pveproxy[3059]: starting 1 worker(s)
Jan 23 11:02:17 evp pveproxy[3059]: worker 43533 started
Jan 23 15:19:42 evp systemd[1]: Stopping PVE API Proxy Server...
Jan 23 15:21:12 evp systemd[1]: pveproxy.service: Stopping timed out. Terminating.
Jan 23 15:21:12 evp pveproxy[3059]: received signal TERM
Jan 23 15:21:12 evp pveproxy[3059]: server closing
Jan 23 15:21:22 evp pveproxy[3059]: error stopping workers (will kill them now) - timeout
Jan 23 15:21:22 evp pveproxy[3059]: kill worker 60971
Jan 23 15:21:22 evp pveproxy[3059]: kill worker 4353
Jan 23 15:21:22 evp pveproxy[3059]: kill worker 1237
Jan 23 15:21:22 evp pveproxy[3059]: server stopped
Jan 23 15:22:42 evp systemd[1]: pveproxy.service: State 'stop-sigterm' timed out. Killing.
Jan 23 15:22:42 evp systemd[1]: pveproxy.service: Killing process 6312 (pveproxy) with signal SIGKILL.
Jan 23 15:24:13 evp systemd[1]: pveproxy.service: Processes still around after SIGKILL. Ignoring.
Jan 23 15:25:43 evp systemd[1]: pveproxy.service: State 'stop-final-sigterm' timed out. Killing.
Jan 23 15:25:43 evp systemd[1]: pveproxy.service: Killing process 6312 (pveproxy) with signal SIGKILL.
Jan 23 15:27:13 evp systemd[1]: pveproxy.service: Processes still around after final SIGKILL. Entering failed mode.
Jan 23 15:27:13 evp systemd[1]: pveproxy.service: Failed with result 'timeout'.
Jan 23 15:27:13 evp systemd[1]: Stopped PVE API Proxy Server.
Jan 23 15:27:13 evp systemd[1]: pveproxy.service: Found left-over process 6312 (pveproxy) in control group while starting unit. Ignoring.
Jan 23 15:27:13 evp systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
Jan 23 15:27:13 evp systemd[1]: Starting PVE API Proxy Server...
Jan 23 15:28:43 evp systemd[1]: pveproxy.service: Start-pre operation timed out. Terminating.
Jan 23 15:28:43 evp systemd[1]: pveproxy.service: Control process exited, code=killed, status=15/TERM
Jan 23 15:30:14 evp systemd[1]: pveproxy.service: State 'stop-final-sigterm' timed out. Killing.
Jan 23 15:30:14 evp systemd[1]: pveproxy.service: Killing process 6312 (pveproxy) with signal SIGKILL.
Jan 23 15:31:44 evp systemd[1]: pveproxy.service: Processes still around after final SIGKILL. Entering failed mode.
Jan 23 15:31:44 evp systemd[1]: pveproxy.service: Failed with result 'timeout'.
Jan 23 15:31:44 evp systemd[1]: Failed to start PVE API Proxy Server.
Jan 23 15:31:44 evp systemd[1]: pveproxy.service: Service RestartSec=100ms expired, scheduling restart.
Jan 23 15:31:44 evp systemd[1]: pveproxy.service: Scheduled restart job, restart counter is at 1.
Jan 23 15:31:44 evp systemd[1]: Stopped PVE API Proxy Server.
Jan 23 15:31:44 evp systemd[1]: pveproxy.service: Found left-over process 6312 (pveproxy) in control group while starting unit. Ignoring.
Jan 23 15:31:44 evp systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
Jan 23 15:31:44 evp systemd[1]: Starting PVE API Proxy Server...
Jan 23 15:33:14 evp systemd[1]: pveproxy.service: Start-pre operation timed out. Terminating.
Jan 23 15:33:14 evp systemd[1]: pveproxy.service: Control process exited, code=killed, status=15/TERM
Jan 23 15:34:45 evp systemd[1]: pveproxy.service: State 'stop-final-sigterm' timed out. Killing.
Jan 23 15:34:45 evp systemd[1]: pveproxy.service: Killing process 6312 (pveproxy) with signal SIGKILL.
Jan 23 15:36:15 evp systemd[1]: pveproxy.service: Processes still around after final SIGKILL. Entering failed mode.
Jan 23 15:36:15 evp systemd[1]: pveproxy.service: Failed with result 'timeout'.
Jan 23 15:36:15 evp systemd[1]: Failed to start PVE API Proxy Server.
Jan 23 15:36:15 evp systemd[1]: pveproxy.service: Service RestartSec=100ms expired, scheduling restart.
Jan 23 15:36:15 evp systemd[1]: pveproxy.service: Scheduled restart job, restart counter is at 2.
Jan 23 15:36:15 evp systemd[1]: Stopped PVE API Proxy Server.
Jan 23 15:36:15 evp systemd[1]: pveproxy.service: Found left-over process 6312 (pveproxy) in control group while starting unit. Ignoring.
Jan 23 15:36:15 evp systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
Jan 23 15:36:15 evp systemd[1]: Starting PVE API Proxy Server...
Jan 23 15:37:45 evp systemd[1]: pveproxy.service: Start-pre operation timed out. Terminating.
Jan 23 15:37:45 evp systemd[1]: pveproxy.service: Control process exited, code=killed, status=15/TERM
Jan 23 15:39:16 evp systemd[1]: pveproxy.service: State 'stop-final-sigterm' timed out. Killing.
Jan 23 15:39:16 evp systemd[1]: pveproxy.service: Killing process 6312 (pveproxy) with signal SIGKILL.
Jan 23 15:40:46 evp systemd[1]: pveproxy.service: Processes still around after final SIGKILL. Entering failed mode.
Jan 23 15:40:46 evp systemd[1]: pveproxy.service: Failed with result 'timeout'.
Jan 23 21:05:58 evp systemd[1]: Starting PVE API Proxy Server...
Jan 23 21:07:28 evp systemd[1]: pveproxy.service: Start-pre operation timed out. Terminating.
Jan 23 21:07:28 evp systemd[1]: pveproxy.service: Control process exited, code=killed, status=15/TERM
Jan 23 21:08:59 evp systemd[1]: pveproxy.service: State 'stop-final-sigterm' timed out. Killing.
Jan 23 21:08:59 evp systemd[1]: pveproxy.service: Killing process 6312 (pveproxy) with signal SIGKILL.
Jan 23 21:10:29 evp systemd[1]: pveproxy.service: Processes still around after final SIGKILL. Entering failed mode.
Jan 23 21:10:29 evp systemd[1]: pveproxy.service: Failed with result 'timeout'.
Jan 23 21:10:29 evp systemd[1]: Failed to start PVE API Proxy Server.
Jan 23 21:10:29 evp systemd[1]: pveproxy.service: Service RestartSec=100ms expired, scheduling restart.
Jan 23 21:10:29 evp systemd[1]: pveproxy.service: Scheduled restart job, restart counter is at 76.
Jan 23 21:10:29 evp systemd[1]: Stopped PVE API Proxy Server.
Jan 23 21:10:29 evp systemd[1]: pveproxy.service: Found left-over process 6312 (pveproxy) in control group while starting unit. Ignoring.
Jan 23 21:10:29 evp systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
Jan 23 21:10:29 evp systemd[1]: Starting PVE API Proxy Server...