grey question mark

H

hmmatos

Renowned Member
Sep 8, 2015
11
1
68
Hello,

I have been having this recurring problem, grey question mark

Screenshot 2023-03-08 at 15.18.18.png

I tried to restart the services but the problem remains

Code: 
systemctl stop pve-cluster && systemctl stop corosync && systemctl stop pvedaemon && systemctl stop pveproxy && systemctl stop pvestatd
systemctl start pve-cluster && systemctl start corosync && systemctl start pvedaemon && systemctl start pveproxy && systemctl start pvestatd

The only way to fix it in restart the node.

--------------------------

cat /var/log/syslog

Code: 
Mar  8 15:23:43 cofgr-pve-001 pveproxy[1375231]: proxy detected vanished client connection
Mar  8 15:23:43 cofgr-pve-001 pveproxy[1375230]: proxy detected vanished client connection
Mar  8 15:23:46 cofgr-pve-001 pveproxy[1375230]: proxy detected vanished client connection
Mar  8 15:23:46 cofgr-pve-001 pveproxy[1375230]: proxy detected vanished client connection
Mar  8 15:23:47 cofgr-pve-001 pveproxy[1375230]: proxy detected vanished client connection

journalctl -f

Code: 
Mar 08 15:32:29 cofgr-pve-001 pveproxy[1375230]: proxy detected vanished client connection
Mar 08 15:32:30 cofgr-pve-001 pveproxy[1375232]: proxy detected vanished client connection
Mar 08 15:32:33 cofgr-pve-001 pveproxy[1375232]: proxy detected vanished client connection
Mar 08 15:32:43 cofgr-pve-001 pveproxy[1375232]: proxy detected vanished client connection
Mar 08 15:32:53 cofgr-pve-001 pveproxy[1375232]: proxy detected vanished client connection

pveversion -v

Code: 
proxmox-ve: 7.3-1 (running kernel: 5.15.85-1-pve)
pve-manager: 7.3-6 (running version: 7.3-6/723bb6ec)
pve-kernel-helper: 7.3-5
pve-kernel-5.15: 7.3-2
pve-kernel-5.13: 7.1-9
pve-kernel-5.11: 7.0-10
pve-kernel-5.15.85-1-pve: 5.15.85-1
pve-kernel-5.15.83-1-pve: 5.15.83-1
pve-kernel-5.13.19-6-pve: 5.13.19-15
pve-kernel-5.13.19-2-pve: 5.13.19-4
pve-kernel-5.11.22-7-pve: 5.11.22-12
pve-kernel-5.11.22-4-pve: 5.11.22-9
ceph-fuse: 16.2.11-pve1
corosync: 3.1.7-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve2
libproxmox-acme-perl: 1.4.4
libproxmox-backup-qemu0: 1.3.1-1
libpve-access-control: 7.3-1
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.3-2
libpve-guest-common-perl: 4.2-3
libpve-http-server-perl: 4.1-5
libpve-storage-perl: 7.3-2
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.2-2
lxcfs: 5.0.3-pve1
novnc-pve: 1.4.0-1
proxmox-backup-client: 2.3.3-1
proxmox-backup-file-restore: 2.3.3-1
proxmox-mail-forward: 0.1.1-1
proxmox-mini-journalreader: 1.3-1
proxmox-offline-mirror-helper: 0.5.1-1
proxmox-widget-toolkit: 3.5.5
pve-cluster: 7.3-2
pve-container: 4.4-2
pve-docs: 7.3-1
pve-edk2-firmware: 3.20220526-1
pve-firewall: 4.2-7
pve-firmware: 3.6-3
pve-ha-manager: 3.5.1
pve-i18n: 2.8-3
pve-qemu-kvm: 7.2.0-5
pve-xtermjs: 4.16.0-1
qemu-server: 7.3-4
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+3
vncterm: 1.7-1
zfsutils-linux: 2.1.9-pve1

uptime

Code: 
15:48:43 up 28 days, 22:21,  1 user,  load average: 1.75, 1.96, 2.08

any ideas on how to fix this?
 
Hello,

yes, Firefox and Safari is the same.

another thing that happens is in the login to be able to login I first have to restart the services.
 
Can you provide us more verbose from the syslog/journalctl?

From the image you posted, I see that there is no cluster, right? if it's no a cluster, you not need to restart the corosync and pve-cluster services.


Another thing I would check is the (Storage) is all storage are online pvesm status?
 
right there is no cluster.

Code: 
root@cofgr-pve-001:~# pvesm status
Name                      Type     Status           Total            Used       Available        %
Veeam-BackupServer        cifs     active     62507778048     55362366528      7145411520   88.57%
loca-zfs               zfspool     active      2901934080        73251116      2828682964    2.52%
local                      dir     active        57225328        15028124        39257916   26.26%

Code: 
root@cofgr-pve-001:~# journalctl -f
-- Journal begins at Tue 2022-12-20 01:20:42 WET. --
Mar 09 09:20:17 cofgr-pve-001 pveproxy[3429953]: proxy detected vanished client connection
Mar 09 09:20:23 cofgr-pve-001 pveproxy[3429953]: proxy detected vanished client connection
Mar 09 09:20:29 cofgr-pve-001 pveproxy[3429954]: proxy detected vanished client connection
Mar 09 09:20:33 cofgr-pve-001 pveproxy[3429952]: proxy detected vanished client connection
Mar 09 09:20:39 cofgr-pve-001 pveproxy[3429953]: proxy detected vanished client connection
Mar 09 09:20:43 cofgr-pve-001 pveproxy[3429952]: proxy detected vanished client connection
Mar 09 09:20:43 cofgr-pve-001 pveproxy[3429954]: proxy detected vanished client connection
Mar 09 09:20:52 cofgr-pve-001 pveproxy[3429952]: proxy detected vanished client connection
Mar 09 09:20:53 cofgr-pve-001 pveproxy[3429952]: proxy detected vanished client connection
Mar 09 09:21:03 cofgr-pve-001 pveproxy[3429953]: proxy detected vanished client connection
Mar 09 09:21:13 cofgr-pve-001 pveproxy[3429952]: proxy detected vanished client connection
Mar 09 09:21:16 cofgr-pve-001 pveproxy[3429952]: proxy detected vanished client connection
Mar 09 09:21:22 cofgr-pve-001 pveproxy[3429952]: proxy detected vanished client connection
Mar 09 09:21:23 cofgr-pve-001 pveproxy[3429952]: proxy detected vanished client connection

Code: 
root@cofgr-pve-001:~# tail -f /var/log/syslog
Mar  9 09:21:29 cofgr-pve-001 pveproxy[3429953]: proxy detected vanished client connection
Mar  9 09:21:30 cofgr-pve-001 pveproxy[3429953]: proxy detected vanished client connection
Mar  9 09:21:33 cofgr-pve-001 pveproxy[3429953]: proxy detected vanished client connection
Mar  9 09:21:43 cofgr-pve-001 pveproxy[3429954]: proxy detected vanished client connection
Mar  9 09:21:43 cofgr-pve-001 pveproxy[3429954]: proxy detected vanished client connection
Mar  9 09:21:43 cofgr-pve-001 pveproxy[3429952]: proxy detected vanished client connection
Mar  9 09:21:46 cofgr-pve-001 pveproxy[3429952]: proxy detected vanished client connection
Mar  9 09:21:52 cofgr-pve-001 pveproxy[3429954]: proxy detected vanished client connection
Mar  9 09:21:53 cofgr-pve-001 pveproxy[3429952]: proxy detected vanished client connection
Mar  9 09:22:03 cofgr-pve-001 pveproxy[3429954]: proxy detected vanished client connection
Mar  9 09:22:13 cofgr-pve-001 pveproxy[3429954]: proxy detected vanished client connection
Mar  9 09:22:16 cofgr-pve-001 pveproxy[3429954]: proxy detected vanished client connection
Mar  9 09:22:16 cofgr-pve-001 pveproxy[3429952]: proxy detected vanished client connection
Mar  9 09:22:22 cofgr-pve-001 pveproxy[3429954]: proxy detected vanished client connection
Mar  9 09:22:23 cofgr-pve-001 pveproxy[3429952]: proxy detected vanished client connection


Screenshot 2023-03-09 at 09.27.02.png
 
Can you restart only the pveproxy and pvedaemon services and then post the status output here:

Bash: 
systemctl restart pveproxy.service
systemctl status pveproxy.service

systemctl restart pvedaemon.service
systemctl status pvedaemon.service
 
Code: 
root@cofgr-pve-001:~# systemctl restart pveproxy.service
root@cofgr-pve-001:~# systemctl status pveproxy.service
● pveproxy.service - PVE API Proxy Server
     Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2023-03-09 09:45:33 WET; 8s ago
    Process: 3551694 ExecStartPre=/usr/bin/pvecm updatecerts --silent (code=exited, status=0/SUCCESS)
    Process: 3551697 ExecStart=/usr/bin/pveproxy start (code=exited, status=0/SUCCESS)
   Main PID: 3551699 (pveproxy)
      Tasks: 4 (limit: 154305)
     Memory: 206.7M
        CPU: 1.934s
     CGroup: /system.slice/pveproxy.service
             ├─3551699 pveproxy
             ├─3551700 pveproxy worker
             ├─3551701 pveproxy worker
             └─3551702 pveproxy worker

Mar 09 09:45:31 cofgr-pve-001 systemd[1]: Starting PVE API Proxy Server...
Mar 09 09:45:33 cofgr-pve-001 pveproxy[3551699]: starting server
Mar 09 09:45:33 cofgr-pve-001 pveproxy[3551699]: starting 3 worker(s)
Mar 09 09:45:33 cofgr-pve-001 pveproxy[3551699]: worker 3551700 started
Mar 09 09:45:33 cofgr-pve-001 pveproxy[3551699]: worker 3551701 started
Mar 09 09:45:33 cofgr-pve-001 pveproxy[3551699]: worker 3551702 started
Mar 09 09:45:33 cofgr-pve-001 systemd[1]: Started PVE API Proxy Server.

Code: 
root@cofgr-pve-001:~# systemctl restart pvedaemon.service
root@cofgr-pve-001:~# systemctl status pvedaemon.service
● pvedaemon.service - PVE API Daemon
     Loaded: loaded (/lib/systemd/system/pvedaemon.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2023-03-09 09:46:42 WET; 3s ago
    Process: 3553733 ExecStart=/usr/bin/pvedaemon start (code=exited, status=0/SUCCESS)
   Main PID: 3553735 (pvedaemon)
      Tasks: 5 (limit: 154305)
     Memory: 137.4M
        CPU: 1.350s
     CGroup: /system.slice/pvedaemon.service
             ├─3553735 pvedaemon
             ├─3553736 pvedaemon worker
             ├─3553737 pvedaemon worker
             ├─3553738 pvedaemon worker
             └─3553856 lxc-info -n 103 -p

Mar 09 09:46:40 cofgr-pve-001 systemd[1]: Starting PVE API Daemon...
Mar 09 09:46:41 cofgr-pve-001 pvedaemon[3553735]: starting server
Mar 09 09:46:41 cofgr-pve-001 pvedaemon[3553735]: starting 3 worker(s)
Mar 09 09:46:41 cofgr-pve-001 pvedaemon[3553735]: worker 3553736 started
Mar 09 09:46:41 cofgr-pve-001 pvedaemon[3553735]: worker 3553737 started
Mar 09 09:46:41 cofgr-pve-001 pvedaemon[3553735]: worker 3553738 started
Mar 09 09:46:42 cofgr-pve-001 systemd[1]: Started PVE API Daemon.

I have a connection issue on postfix but I thinks is not related

Code: 
Mar 09 09:41:36 cofgr-pve-001 postfix/smtp[3535933]: connect to XXX.com[192.168.XXX.XXX]:25: Connection timed out
Mar 09 09:41:36 cofgr-pve-001 postfix/smtp[3535933]: DF3662E104E: to=<XXX@XXX.com>, relay=none, delay=380495, delays=380345/0.02/150/0, dsn=4.4.1, status=deferred (connect to XXX.com[192.168.XXX.XXX]:25: Connection timed out)
Mar 09 09:41:36 cofgr-pve-001 postfix/smtp[3535934]: connect to XXX.com[192.168.XXX.XXX]:25: Connection timed out
Mar 09 09:41:36 cofgr-pve-001 postfix/smtp[3535934]: 66E1A2E1069: to=<XXX@XXX.com>, relay=none, delay=405692, delays=405542/0.02/150/0, dsn=4.4.1, status=deferred (connect to XXX.com[192.168.XXX.XXX]:25: Connection timed out)
 
2 or 3 minutes after restart proxy service starts showing "proxy detected vanished client connection"

Code: 
root@cofgr-pve-001:~# systemctl status pveproxy.service
● pveproxy.service - PVE API Proxy Server
     Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2023-03-09 09:45:33 WET; 15min ago
    Process: 3551694 ExecStartPre=/usr/bin/pvecm updatecerts --silent (code=exited, status=0/SUCCESS)
    Process: 3551697 ExecStart=/usr/bin/pveproxy start (code=exited, status=0/SUCCESS)
   Main PID: 3551699 (pveproxy)
      Tasks: 4 (limit: 154305)
     Memory: 222.6M
        CPU: 4.832s
     CGroup: /system.slice/pveproxy.service
             ├─3551699 pveproxy
             ├─3551700 pveproxy worker
             ├─3551701 pveproxy worker
             └─3551702 pveproxy worker

Mar 09 10:00:52 cofgr-pve-001 pveproxy[3551701]: proxy detected vanished client connection
Mar 09 10:00:53 cofgr-pve-001 pveproxy[3551702]: proxy detected vanished client connection
Mar 09 10:00:59 cofgr-pve-001 pveproxy[3551701]: proxy detected vanished client connection
Mar 09 10:01:03 cofgr-pve-001 pveproxy[3551700]: proxy detected vanished client connection
Mar 09 10:01:03 cofgr-pve-001 pveproxy[3551701]: proxy detected vanished client connection
Mar 09 10:01:13 cofgr-pve-001 pveproxy[3551702]: proxy detected vanished client connection
Mar 09 10:01:13 cofgr-pve-001 pveproxy[3551701]: proxy detected vanished client connection
Mar 09 10:01:16 cofgr-pve-001 pveproxy[3551700]: proxy detected vanished client connection
Mar 09 10:01:16 cofgr-pve-001 pveproxy[3551700]: proxy detected vanished client connection
Mar 09 10:01:17 cofgr-pve-001 pveproxy[3551700]: proxy detected vanished client connection
 
Can you post the full syslog after you restart the proxy service?

I would also check the network configuration of the server if everything is ok.
 
Code: 
Mar  9 15:23:49 cofgr-pve-001 pvedaemon[24551]: <root@pam> successful auth for user 'root@pam'
Mar  9 15:24:00 cofgr-pve-001 pvedaemon[24552]: <root@pam> successful auth for user 'root@pam'
Mar  9 15:24:13 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:25:13 cofgr-pve-001 pveproxy[24694]: proxy detected vanished client connection
Mar  9 15:26:13 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:26:16 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:26:16 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:26:17 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:26:23 cofgr-pve-001 pveproxy[24694]: proxy detected vanished client connection
Mar  9 15:26:43 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:26:46 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:26:46 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:26:47 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:26:52 cofgr-pve-001 pveproxy[24694]: proxy detected vanished client connection
Mar  9 15:26:53 cofgr-pve-001 pveproxy[24694]: proxy detected vanished client connection
Mar  9 15:27:13 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:27:16 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:27:16 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:27:17 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:27:23 cofgr-pve-001 pveproxy[24694]: proxy detected vanished client connection
Mar  9 15:27:29 cofgr-pve-001 pveproxy[24694]: proxy detected vanished client connection
Mar  9 15:27:33 cofgr-pve-001 pveproxy[24692]: proxy detected vanished client connection
Mar  9 15:27:39 cofgr-pve-001 pveproxy[24694]: proxy detected vanished client connection
Mar  9 15:27:43 cofgr-pve-001 pveproxy[24694]: proxy detected vanished client connection
Mar  9 15:27:43 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:27:46 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:27:46 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:27:52 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:27:53 cofgr-pve-001 pveproxy[24694]: proxy detected vanished client connection
Mar  9 15:28:13 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:28:16 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:28:16 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:28:22 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:28:33 cofgr-pve-001 pveproxy[24692]: proxy detected vanished client connection
Mar  9 15:28:39 cofgr-pve-001 pveproxy[24692]: proxy detected vanished client connection
Mar  9 15:28:43 cofgr-pve-001 pveproxy[24694]: proxy detected vanished client connection
Mar  9 15:28:43 cofgr-pve-001 pveproxy[24694]: proxy detected vanished client connection
Mar  9 15:28:43 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:28:46 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:28:46 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:28:52 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:28:53 cofgr-pve-001 pveproxy[24692]: proxy detected vanished client connection
Mar  9 15:29:08 cofgr-pve-001 postfix/qmgr[2910497]: DF3662E104E: from=<pve.cofgr@XXX.com>, size=1751, nrcpt=1 (queue active)
Mar  9 15:29:08 cofgr-pve-001 postfix/qmgr[2910497]: 66E1A2E1069: from=<pve.cofgr@XXX.com>, size=1793, nrcpt=1 (queue active)
Mar  9 15:29:13 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:29:13 cofgr-pve-001 pveproxy[24694]: proxy detected vanished client connection
Mar  9 15:29:16 cofgr-pve-001 pveproxy[24692]: proxy detected vanished client connection
Mar  9 15:29:16 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:29:16 cofgr-pve-001 pveproxy[24692]: proxy detected vanished client connection
Mar  9 15:29:17 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:29:23 cofgr-pve-001 pveproxy[24692]: proxy detected vanished client connection
Mar  9 15:29:29 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:29:33 cofgr-pve-001 pveproxy[24692]: proxy detected vanished client connection
Mar  9 15:29:38 cofgr-pve-001 postfix/smtp[35702]: connect to XXX.com[192.168.XXX.XXX]:25: Connection timed out
Mar  9 15:29:38 cofgr-pve-001 postfix/smtp[35703]: connect to XXX.com[192.168.XXX.XXX]:25: Connection timed out
Mar  9 15:29:38 cofgr-pve-001 postfix/smtp[35703]: connect to XXX.com[192.168.XXX.XXX]:25: Connection refused
Mar  9 15:29:43 cofgr-pve-001 pveproxy[24693]: proxy detected vanished client connection
Mar  9 15:29:46 cofgr-pve-001 pveproxy[24692]: proxy detected vanished client connection
Mar  9 15:29:46 cofgr-pve-001 pveproxy[24692]: proxy detected vanished client connection
Mar  9 15:29:50 cofgr-pve-001 systemd[1]: Stopping PVE API Proxy Server...
Mar  9 15:29:51 cofgr-pve-001 pveproxy[24690]: received signal TERM
Mar  9 15:29:51 cofgr-pve-001 pveproxy[24690]: server closing
Mar  9 15:29:51 cofgr-pve-001 pveproxy[24694]: worker exit
Mar  9 15:29:51 cofgr-pve-001 pveproxy[24692]: worker exit
Mar  9 15:29:51 cofgr-pve-001 pveproxy[24690]: worker 24694 finished
Mar  9 15:29:51 cofgr-pve-001 pveproxy[24690]: worker 24692 finished
Mar  9 15:29:51 cofgr-pve-001 pveproxy[24690]: worker 24693 finished
Mar  9 15:29:51 cofgr-pve-001 pveproxy[24690]: server stopped
Mar  9 15:29:52 cofgr-pve-001 systemd[1]: pveproxy.service: Succeeded.
Mar  9 15:29:52 cofgr-pve-001 systemd[1]: Stopped PVE API Proxy Server.
Mar  9 15:29:52 cofgr-pve-001 systemd[1]: pveproxy.service: Consumed 4.957s CPU time.
Mar  9 15:29:52 cofgr-pve-001 systemd[1]: Starting PVE API Proxy Server...
Mar  9 15:29:54 cofgr-pve-001 pveproxy[36978]: starting server
Mar  9 15:29:54 cofgr-pve-001 pveproxy[36978]: starting 3 worker(s)
Mar  9 15:29:54 cofgr-pve-001 pveproxy[36978]: worker 36979 started
Mar  9 15:29:54 cofgr-pve-001 pveproxy[36978]: worker 36980 started
Mar  9 15:29:54 cofgr-pve-001 pveproxy[36978]: worker 36981 started
Mar  9 15:29:54 cofgr-pve-001 systemd[1]: Started PVE API Proxy Server.

I don't change my network configuration in a long time and everything was working
My container is working, I can reach them and ssh

all except one - 103
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom