Communication Failure on Selecting Container Storage

[henkyprayoga]

Hi,

Suddenly I'm having this problem. I'm no longer able to select container storage.



I tried to execute pvesm status on the shell. The command was executed but required a very long time.



I executed pvesm with strace, it wait at this point:



What went wrong? Please help.

Thank you very much.

 

[Moayad]

Thank you for the output! Can you also provide us with the output of journalctl during listing PVE storage? To do this open a shell and run journalctl -f command, in the another shell/GUI try to list the storage (pvecm status or by GUI "Create a VM/CT")

 

[shadymint]

Hi,
I think I am having similar problems. I just upgraded from 6 to 7.2-5 and in the storage summary panel I am getting 'connection timed out' . This is a standalone (test) server and has no guests running from boot, they appear to start and run OK but control of them is intermittent so stopping from the web interface is not always possible. Like the previous poster, I tried pvesm status in a shell and this seems to always respond but sometimes after a long wait.

root@chervil:~# time pvesm status
Name Type Status Total Used Available %
home dir active 95595940 63415832 27280988 66.34%
local dir active 9545920 6641256 2400040 69.57%
tank lvm active 314241024 203423744 110817280 64.73%

real 0m31.396s
user 0m0.884s
sys 0m0.361s
root@chervil:~#

I'm afraid journalctl doesnt show very much whilst the above was going on.

Jun 24 17:53:09 chervil systemd-logind[690]: New session 3 of user andy.
Jun 24 17:53:09 chervil systemd[1]: Started Session 3 of user andy.
Jun 24 17:53:15 chervil su[18677]: (to root) andy on pts/1
Jun 24 17:53:15 chervil su[18677]: pam_unix(su-l:session): session opened for user root(uid=0) by andy(uid=1000)
Jun 24 17:53:41 chervil pvestatd[1021]: status update time (90.435 seconds)
Jun 24 17:53:42 chervil rsyslogd[678]: -- MARK --
Jun 24 17:54:10 chervil pveproxy[1090]: proxy detected vanished client connection
Jun 24 17:54:11 chervil pvestatd[1021]: status update time (30.589 seconds)
Jun 24 17:55:42 chervil pvestatd[1021]: status update time (90.439 seconds)
Jun 24 17:56:19 chervil pvestatd[1021]: status update time (36.957 seconds)
Jun 24 17:57:20 chervil pveproxy[1090]: proxy detected vanished client connection

In case its of any use, I am attaching the strace output as well.

I dont seem to have this problem with another test system that I built last week currently at 7.2-4
--
Andy

 

[Moayad]

Hi,

Can you try re-issuing the new certificates then restart the below services:

Update certifcates:

pvecm updatecerts --force

Restart pveproxy pvedaemon services:

systemctl restart pveproxy.service pvedaemon.service

 

[shadymint]

Hi,
Thanks for the reply, unfortunately, your suggestion did not provide the answer.

root@chervil:~# time pvesm status
Name         Type     Status           Total            Used       Available        %
home          dir     active        95595940        75648164        15048656   79.13%
local         dir     active         9545920         6616644         2424652   69.31%
tank          lvm     active       314241024       150994944       163246080   48.05%

real    0m31.347s
user    0m0.841s
sys     0m0.396s
root@chervil:~# pvecm updatecerts --force
(re)generate node files
generate new node certificate
merge authorized SSH keys and known hosts
root@chervil:~# systemctl restart pveproxy.service pvedaemon.service
root@chervil:~# time pvesm status
Name         Type     Status           Total            Used       Available        %
home          dir     active        95595940        75648164        15048656   79.13%
local         dir     active         9545920         6617608         2423688   69.32%
tank          lvm     active       314241024       150994944       163246080   48.05%

real    1m11.034s
user    0m0.970s
sys     0m0.703s
root@chervil:~#

However, over the weekend I built 7.2-5 from scratch on the server using a spare disk and the response is fine. I suppose I cant rule out a fault on the old disk, perhaps provoked by the upgrade, but it seems more likely something went wrong with the upgrade. I can keep the old disk around if you want me to try something else. Whether the IO delay on the summary screen is a symptom or the cause I dont know, but I can live with the rebuild, it just informs me the best way to approach the 'production' servers.

Thanks again
--
Andy