[SOLVED] I can't connect through web gui, SSH work

spetial

Member
Dec 25, 2020
16
1
8
47
Hello,
I have a mini PC running Proxmox some time ago I had two power outages in a row.
Since then I can't log into either the Proxmox web interface or AdGuard.

Proxmox is installed on a mini PC, there I have HommeAssistant as a vitrual machine and AdGuard in a container.

Through SSH I can connect to Proxmox but when I try through the site it grinds and nothing happens at the bottom left it says I am waiting for api.start.me.

I hooked up the monitor to this mini PC and I see something like this (attached photo).

The HA itself works but somehow so slow, that's the impression I get, as if something is slowing it down.

Through SSH I found a couple of errors but I don't know if they have anything to do with this behavior of Proxmox.

Code:
[   76.367210] usb 1-1: unable to read config index 0 descriptor/all
[   76.367235] usb 1-1: can't read configurations, error -110
[   76.495120] usb 1-1: new full-speed USB device number 9 using xhci_hcd
[   81.743512] usb 1-1: unable to read config index 0 descriptor/all
[   81.743535] usb 1-1: can't read configurations, error -110
[   81.743651] usb usb1-port1: attempt power cycle
[   82.399445] usb 1-1: new full-speed USB device number 10 using xhci_hcd
[   86.861041] usb 1-1: unable to read config index 0 descriptor/all
[   86.861064] usb 1-1: can't read configurations, error -71


Maybe someone knows what could be the cause of all this?
Here some additional information:


Code:
systemctl status pveproxy
● pveproxy.service - PVE API Proxy Server
     Loaded: loaded (/lib/systemd/system/pveproxy.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-09-01 10:10:33 CEST; 38min ago
    Process: 1018 ExecStartPre=/usr/bin/pvecm updatecerts --silent (code=exited, status=0/SUCCESS)
    Process: 1020 ExecStart=/usr/bin/pveproxy start (code=exited, status=0/SUCCESS)
   Main PID: 1022 (pveproxy)
      Tasks: 4 (limit: 18987)
     Memory: 247.5M
        CPU: 3.577s
     CGroup: /system.slice/pveproxy.service
             ├─1022 pveproxy
             ├─1023 pveproxy worker
             ├─1024 pveproxy worker
             └─1025 pveproxy worker

Sep 01 10:10:32 proxmox systemd[1]: Starting PVE API Proxy Server...
Sep 01 10:10:33 proxmox pveproxy[1022]: starting server
Sep 01 10:10:33 proxmox pveproxy[1022]: starting 3 worker(s)
Sep 01 10:10:33 proxmox pveproxy[1022]: worker 1023 started
Sep 01 10:10:33 proxmox pveproxy[1022]: worker 1024 started
Sep 01 10:10:33 proxmox pveproxy[1022]: worker 1025 started
Sep 01 10:10:33 proxmox systemd[1]: Started PVE API Proxy Server.

Code:
pveversion -v
proxmox-ve: 7.2-1 (running kernel: 5.15.39-4-pve)
pve-manager: 7.2-7 (running version: 7.2-7/d0dd0e85)
pve-kernel-5.15: 7.2-9
pve-kernel-helper: 7.2-9
pve-kernel-5.13: 7.1-9
pve-kernel-5.4: 6.4-15
pve-kernel-5.15.39-4-pve: 5.15.39-4
pve-kernel-5.15.35-1-pve: 5.15.35-3
pve-kernel-5.13.19-6-pve: 5.13.19-15
pve-kernel-5.4.174-2-pve: 5.4.174-2
pve-kernel-5.4.73-1-pve: 5.4.73-1
ceph-fuse: 14.2.21-1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown: 0.8.36+pve1
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve1
libproxmox-acme-perl: 1.4.2
libproxmox-backup-qemu0: 1.3.1-1
libpve-access-control: 7.2-4
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.2-2
libpve-guest-common-perl: 4.1-2
libpve-http-server-perl: 4.1-3
libpve-storage-perl: 7.2-8
libqb0: 1.0.5-1
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.0-3
lxcfs: 4.0.12-pve1
novnc-pve: 1.3.0-3
proxmox-backup-client: 2.2.5-1
proxmox-backup-file-restore: 2.2.5-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.5.1
pve-cluster: 7.2-2
pve-container: 4.2-2
pve-docs: 7.2-2
pve-edk2-firmware: 3.20220526-1
pve-firewall: 4.2-5
pve-firmware: 3.5-1
pve-ha-manager: 3.4.0
pve-i18n: 2.7-2
pve-qemu-kvm: 7.0.0-2
pve-xtermjs: 4.16.0-1
qemu-server: 7.2-4
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.7.1~bpo11+1
vncterm: 1.7-1
zfsutils-linux: 2.1.5-pve1

Code:
systemctl status pvedaemon
● pvedaemon.service - PVE API Daemon
     Loaded: loaded (/lib/systemd/system/pvedaemon.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-09-01 10:10:31 CEST; 41min ago
    Process: 983 ExecStart=/usr/bin/pvedaemon start (code=exited, status=0/SUCCESS)
   Main PID: 1013 (pvedaemon)
      Tasks: 4 (limit: 18987)
     Memory: 142.3M
        CPU: 1.058s
     CGroup: /system.slice/pvedaemon.service
             ├─1013 pvedaemon
             ├─1014 pvedaemon worker
             ├─1015 pvedaemon worker
             └─1016 pvedaemon worker

Sep 01 10:10:31 proxmox systemd[1]: Starting PVE API Daemon...
Sep 01 10:10:31 proxmox pvedaemon[1013]: starting server
Sep 01 10:10:31 proxmox pvedaemon[1013]: starting 3 worker(s)
Sep 01 10:10:31 proxmox pvedaemon[1013]: worker 1014 started
Sep 01 10:10:31 proxmox pvedaemon[1013]: worker 1015 started
Sep 01 10:10:31 proxmox pvedaemon[1013]: worker 1016 started
Sep 01 10:10:31 proxmox systemd[1]: Started PVE API Daemon.

Code:
lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 004: ID 04d9:1603 Holtek Semiconductor, Inc. Keyboard
Bus 001 Device 008: ID 1cf1:0030 Dresden Elektronik ZigBee gateway [ConBee II]
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
 
Hey,

what's give "lsof -i" , and can you post your interfaces file and the return of "ip a " please ? :)
 
  • Like
Reactions: spetial
Hey,

what's give "lsof -i"

Hi! of course! Thank you for your reply!

Code:
lsof -i
COMMAND      PID     USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
systemd        1     root   36u  IPv4  19617      0t0  TCP *:sunrpc (LISTEN)
systemd        1     root   37u  IPv4  19618      0t0  UDP *:sunrpc
systemd        1     root   38u  IPv6  14550      0t0  TCP *:sunrpc (LISTEN)
systemd        1     root   39u  IPv6  19409      0t0  UDP *:sunrpc
rpcbind      617     _rpc    4u  IPv4  19617      0t0  TCP *:sunrpc (LISTEN)
rpcbind      617     _rpc    5u  IPv4  19618      0t0  UDP *:sunrpc
rpcbind      617     _rpc    6u  IPv6  14550      0t0  TCP *:sunrpc (LISTEN)
rpcbind      617     _rpc    7u  IPv6  19409      0t0  UDP *:sunrpc
sshd         781     root    3u  IPv4  25297      0t0  TCP *:ssh (LISTEN)
sshd         781     root    4u  IPv6  25299      0t0  TCP *:ssh (LISTEN)
master       974     root   13u  IPv4  27021      0t0  TCP localhost.localdomain:smtp (LISTEN)
master       974     root   14u  IPv6  27022      0t0  TCP ip6-localhost:smtp (LISTEN)
pvedaemon   1013     root    6u  IPv4  23295      0t0  TCP localhost.localdomain:85 (LISTEN)
pvedaemon   1014     root    6u  IPv4  23295      0t0  TCP localhost.localdomain:85 (LISTEN)
pvedaemon   1015     root    6u  IPv4  23295      0t0  TCP localhost.localdomain:85 (LISTEN)
pvedaemon   1016     root    6u  IPv4  23295      0t0  TCP localhost.localdomain:85 (LISTEN)
pveproxy    1022 www-data    6u  IPv6  27203      0t0  TCP *:8006 (LISTEN)
pveproxy    1023 www-data    6u  IPv6  27203      0t0  TCP *:8006 (LISTEN)
pveproxy    1024 www-data    6u  IPv6  27203      0t0  TCP *:8006 (LISTEN)
pveproxy    1025 www-data    6u  IPv6  27203      0t0  TCP *:8006 (LISTEN)
spiceprox   1028 www-data    6u  IPv6  26477      0t0  TCP *:3128 (LISTEN)
spiceprox   1029 www-data    6u  IPv6  26477      0t0  TCP *:3128 (LISTEN)
sshd      106836     root    4u  IPv4 593213      0t0  TCP proxmox.local:ssh->ADMIN-I9-9900KS.fritz.box:50035 (ESTABLISHED)


and can you post your interfaces file and the return of "ip a " please ? :)


Code:
auto lo
iface lo inet loopback

iface enp1s0 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.0.9/24
        gateway 192.168.0.1
        bridge_ports enp1s0
        bridge_stp off
        bridge_fd 0


Code:
ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000
    link/ether ac:e2:d3:13:c2:f8 brd ff:ff:ff:ff:ff:ff
3: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether a2:4e:75:3a:30:a2 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.9/24 brd 192.168.0.255 scope global vmbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::a04e:75ff:fe3a:30a2/64 scope link
       valid_lft forever preferred_lft forever
4: tap100i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master fwbr100i0 state UNKNOWN group default qlen 1000
    link/ether 52:57:1d:08:d9:5e brd ff:ff:ff:ff:ff:ff
5: fwbr100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 42:43:ae:c9:af:25 brd ff:ff:ff:ff:ff:ff
6: fwpr100p0@fwln100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether 8e:5b:df:96:b5:f9 brd ff:ff:ff:ff:ff:ff
7: fwln100i0@fwpr100p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr100i0 state UP group default qlen 1000
    link/ether ce:17:05:98:d8:4a brd ff:ff:ff:ff:ff:ff
8: veth101i0@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr101i0 state UP group default qlen 1000
    link/ether fe:29:ab:80:c4:cb brd ff:ff:ff:ff:ff:ff link-netnsid 0
9: fwbr101i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether be:11:dd:74:9d:d3 brd ff:ff:ff:ff:ff:ff
10: fwpr101p0@fwln101i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000
    link/ether ca:aa:ca:88:d9:d4 brd ff:ff:ff:ff:ff:ff
11: fwln101i0@fwpr101p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr101i0 state UP group default qlen 1000
    link/ether 76:7d:40:96:79:85 brd ff:ff:ff:ff:ff:ff
 
Hello,
I haven't solved the problem yet but it is unlikely to be related to proxmox itself.
At a similar time to these power outages I replaced one of my routers with another and this apparently is the reason that I can't connect through the web gui.
The router configuration is the same, the wiring is the same but I can't connect for anything. When I plug in the old router it works! I will try to look into this problem more thoroughly over the weekend.
 

Attachments

  • proxmox1.JPG
    proxmox1.JPG
    73.8 KB · Views: 12
  • proxmox3.JPG
    proxmox3.JPG
    191.4 KB · Views: 12
  • proxmox2.JPG
    proxmox2.JPG
    59.9 KB · Views: 12
I have a similar issue. A week or so ago we lost power and the UPS almost died. I hastily shutdown the server. When the power returned, I restarted the server. Another brief power outage took the server down. After that, I restarted and I believe I logged into the server to check if all the VMs came up.
This morning, I tried to log into to the server and was unable to access the GUI. I could login with my ssh key, but not with the password. I was able to change the password, but still nothing. I only have one user, which is root. I confirmed that Ream is "Linux PAM standard authentication."
In another thread, they talked about clusters. This server is a single server and I could not find any cluster configuration files. The system was put into service earlier this year using proxmox-ve_7.1-2.
pveversion -v proxmox-ve: 7.1-1 (running kernel: 5.13.19-2-pve) pve-manager: 7.1-7 (running version: 7.1-7/df5740ad) pve-kernel-helper: 7.1-6 pve-kernel-5.13: 7.1-5 pve-kernel-5.13.19-2-pve: 5.13.19-4 ceph-fuse: 15.2.15-pve1 corosync: 3.1.5-pve2 criu: 3.15-1+pve-1 glusterfs-client: 9.2-1 ifupdown2: 3.1.0-1+pmx3 ksm-control-daemon: 1.4-1 libjs-extjs: 7.0.0-1 libknet1: 1.22-pve2 libproxmox-acme-perl: 1.4.0 libproxmox-backup-qemu0: 1.2.0-1 libpve-access-control: 7.1-5 libpve-apiclient-perl: 3.2-1 libpve-common-perl: 7.0-14 libpve-guest-common-perl: 4.0-3 libpve-http-server-perl: 4.0-4 libpve-storage-perl: 7.0-15 libspice-server1: 0.14.3-2.1 lvm2: 2.03.11-2.1 lxc-pve: 4.0.11-1 lxcfs: 4.0.11-pve1 novnc-pve: 1.2.0-3 proxmox-backup-client: 2.1.2-1 proxmox-backup-file-restore: 2.1.2-1 proxmox-mini-journalreader: 1.3-1 proxmox-widget-toolkit: 3.4-4 pve-cluster: 7.1-2 pve-container: 4.1-2 pve-docs: 7.1-2 pve-edk2-firmware: 3.20210831-2 pve-firewall: 4.2-5 pve-firmware: 3.3-3 pve-ha-manager: 3.3-1 pve-i18n: 2.6-2 pve-qemu-kvm: 6.1.0-3 pve-xtermjs: 4.12.0-1 qemu-server: 7.1-4 smartmontools: 7.2-1 spiceterm: 3.2-2 swtpm: 0.7.0~rc1+2 vncterm: 1.7-1 zfsutils-linux: 2.1.1-pve3

I greped the /var/log directory for error, refuse deny,denied, login and fail, with no good results:
auth.log:Sep 2 07:55:06 pve1 sshd[75783]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=192.168.x.8 user=root auth.log:Sep 2 07:55:07 pve1 sshd[75783]: Failed password for root from 192.168.x.8 port 45772 ssh2
TIA,
Jens
 

Attachments

  • LoginFailed.jpg
    LoginFailed.jpg
    43.7 KB · Views: 6
Last edited:
After lunch, I tried again and was able to login on both the GUI and ssh with password. I use lastpass and it was acting funny.
Have a great weekend,
Jens
 
  • Like
Reactions: spetial
@spetial
Hey,
What kind of FS are you using on your node ? Zfs ? another ?

ext4

Thanks for your reply,
I struggled a bit with this problem, I wrote on a German forum about it and it turns out that I think it's a NIC problem.

I still need to check one more thing but I think it is actually a hardware problem. I suspect that something broke during the power outage.

Code:
lsblk -f

NAME                         FSTYPE      FSVER    LABEL UUID                                   FSAVAIL FSUSE% MOUNTPOINT
sda
└─sda1                       ext4        1.0            xxxxxxxx-c550-4dd3-bdb7-5ee7f6a2fef1    201.3G     7% /mnt/pve/Backup
nvme0n1
├─nvme0n1p1
├─nvme0n1p2                  vfat        FAT32          xxxx-B774                               510.7M     0% /boot/efi
└─nvme0n1p3                  LVM2_member LVM2 001       xxxxxxx-YKuQ-AyF9-SLpr-VIDG-03Sy-pPCyUM
  ├─pve-swap                 swap        1              xxxxxxac-3696-4384-9744-5a9a317cd8dd                  [SWAP]
  ├─pve-root                 ext4        1.0            xxxxxx03-5696-4535-bd38-c5d1e1108ff9     77.3G    13% /
  ├─pve-data_tmeta
  │ └─pve-data-tpool
  │   ├─pve-data
  │   ├─pve-vm--100--disk--1
  │   ├─pve-vm--100--disk--2
  │   └─pve-vm--101--disk--0 ext4        1.0            xxxxxxxx-c92b-49e8-b8b6-cf6e84ea248a
  └─pve-data_tdata
    └─pve-data-tpool
      ├─pve-data
      ├─pve-vm--100--disk--1
      ├─pve-vm--100--disk--2
      └─pve-vm--101--disk--0 ext4        1.0            xxxxxxxx-c92b-49e8-b8b6-cf6e84ea248a
 
Hey :)

Nothing wrong with "systemctl --failed" ?

Hi,
Thank you for the answer.

Code:
systemctl --failed
  UNIT LOAD ACTIVE SUB DESCRIPTION
0 loaded units listed.

I found out that my built-in network card is broken.
I bought a new one on USB and with proxmox I can already connect, I only still have a problem with virtual machines because they do not get the network card assigned, but this I already wrote on the German forum and I hope that there someone will help me with the configuration.
https://forum.proxmox.com/threads/proxmox-nur-10-mbit-verbindung.114761/