Failed to login on first attempt

Inglebard

Well-Known Member
May 20, 2016
100
5
58
31
Hi,

I notice a strange issue on one of my proxmox server.

When I log in after a certain period of time (several days), the first attempt always fail.
I thought, I mistyped password, but I only need to click again on "login" to be logged-in without error.

Note :
- The server may be overload
- I already encountered several strange behaviour on it

proxmox-ve: 5.1-42 (running kernel: 4.13.16-2-pve)
pve-manager: 5.1-51 (running version: 5.1-51/96be5354)
pve-kernel-4.13: 5.1-44
pve-kernel-4.13.16-2-pve: 4.13.16-47
pve-kernel-4.13.16-1-pve: 4.13.16-46
pve-kernel-4.13.13-6-pve: 4.13.13-42
pve-kernel-4.13.13-5-pve: 4.13.13-38
pve-kernel-4.13.13-4-pve: 4.13.13-35
pve-kernel-4.13.13-3-pve: 4.13.13-34
pve-kernel-4.13.13-2-pve: 4.13.13-33
pve-kernel-4.13.13-1-pve: 4.13.13-31
pve-kernel-4.13.8-3-pve: 4.13.8-30
pve-kernel-4.13.8-2-pve: 4.13.8-28
pve-kernel-4.13.8-1-pve: 4.13.8-27
pve-kernel-4.13.4-1-pve: 4.13.4-26
corosync: 2.4.2-pve4
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.0-8
libpve-apiclient-perl: 2.0-4
libpve-common-perl: 5.0-30
libpve-guest-common-perl: 2.0-14
libpve-http-server-perl: 2.0-8
libpve-storage-perl: 5.0-18
libqb0: 1.0.1-1
lvm2: 2.02.168-pve6
lxc-pve: 3.0.0-2
lxcfs: 3.0.0-1
novnc-pve: 0.6-4
proxmox-widget-toolkit: 1.0-15
pve-cluster: 5.0-25
pve-container: 2.0-21
pve-docs: 5.1-17
pve-firewall: 3.0-8
pve-firmware: 2.0-4
pve-ha-manager: 2.0-5
pve-i18n: 1.0-4
pve-libspice-server1: 0.12.8-3
pve-qemu-kvm: 2.11.1-5
pve-xtermjs: 1.0-2
qemu-server: 5.0-25
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.7-pve1~bpo9
 
Hi,
Here is a short log (I just try this morning)

Code:
192.168.168.86 - - [15/05/2018:09:26:38 +0200] "GET / HTTP/1.1" 200 754
192.168.168.86 - - [15/05/2018:09:26:39 +0200] "GET /pve2/ext6/theme-crisp/resources/theme-crisp-all.css HTTP/1.1" 200 68
192.168.168.86 - - [15/05/2018:09:26:39 +0200] "GET /pve2/fa/css/font-awesome.css HTTP/1.1" 200 7289
192.168.168.86 - - [15/05/2018:09:26:39 +0200] "GET /pve2/ext6/crisp/resources/charts-all.css HTTP/1.1" 200 995
192.168.168.86 - - [15/05/2018:09:26:39 +0200] "GET /pve2/locale/pve-lang-fr.js?ver=5.1-52 HTTP/1.1" 200 6074
192.168.168.86 - - [15/05/2018:09:26:39 +0200] "GET /proxmoxlib.js?ver=1.0-17 HTTP/1.1" 200 41283
192.168.168.86 - - [15/05/2018:09:26:39 +0200] "GET /pve2/css/ext6-pve.css?ver=5.1-52 HTTP/1.1" 200 2751
192.168.168.86 - - [15/05/2018:09:26:39 +0200] "GET /pve2/ext6/charts.js HTTP/1.1" 200 100383
192.168.168.86 - - [15/05/2018:09:26:39 +0200] "GET /pve2/ext6/locale/locale-fr.js HTTP/1.1" 200 2452
192.168.168.86 - - [15/05/2018:09:26:39 +0200] "GET /pve2/ext6/theme-crisp/resources/theme-crisp-all_1.css HTTP/1.1" 200 33744
192.168.168.86 - - [15/05/2018:09:26:39 +0200] "GET /pve2/js/pvemanagerlib.js?ver=5.1-52 HTTP/1.1" 200 146680
192.168.168.86 - - [15/05/2018:09:26:39 +0200] "GET /pve2/ext6/ext-all.js HTTP/1.1" 200 609692
192.168.168.86 - - [15/05/2018:09:26:39 +0200] "GET /pve2/ext6/theme-crisp/resources/theme-crisp-all_2.css HTTP/1.1" 200 5208
192.168.168.86 - - [15/05/2018:09:26:40 +0200] "GET /pve2/ext6/theme-crisp/resources/images/form/checkbox.png HTTP/1.1" 200 17917
192.168.168.86 - - [15/05/2018:09:26:40 +0200] "GET /pve2/ext6/theme-crisp/resources/images/form/trigger.png HTTP/1.1" 200 17627
192.168.168.86 - - [15/05/2018:09:26:40 +0200] "GET /pve2/ext6/theme-crisp/resources/images/util/splitter/mini-bottom.png HTTP/1.1" 200 15274
192.168.168.86 - - [15/05/2018:09:26:41 +0200] "GET /pve2/fa/fonts/fontawesome-webfont.woff2?v=4.7.0 HTTP/1.1" 200 77160
192.168.168.86 - - [15/05/2018:09:26:41 +0200] "GET /pve2/ext6/theme-crisp/resources/images/grid/sort_desc.png HTTP/1.1" 200 18260
192.168.168.86 - - [15/05/2018:09:26:41 +0200] "GET /pve2/ext6/theme-crisp/resources/images/tree/arrows.png HTTP/1.1" 200 3078
192.168.168.86 - - [15/05/2018:09:26:41 +0200] "GET /pve2/images/proxmox_logo.png HTTP/1.1" 200 2809
192.168.168.86 - - [15/05/2018:09:26:41 +0200] "GET /api2/json/access/domains HTTP/1.1" 200 159
192.168.168.86 - - [15/05/2018:09:26:41 +0200] "GET /pve2/images/logo-128.png HTTP/1.1" 200 4977
192.168.168.86 - - [15/05/2018:09:26:43 +0200] "GET /pve2/ext6/theme-crisp/resources/images/form/exclamation.png HTTP/1.1" 200 1351
192.168.168.86 - - [15/05/2018:09:26:44 +0200] "GET /pve2/ext6/theme-crisp/resources/images/loadmask/loading.gif HTTP/1.1" 200 1849
192.168.168.86 - - [15/05/2018:09:26:55 +0200] "POST /api2/extjs/access/ticket HTTP/1.1" 596 -
192.168.168.86 - - [15/05/2018:09:26:55 +0200] "GET /pve2/ext6/theme-crisp/resources/images/tools/tool-sprites.png HTTP/1.1" 200 24404
192.168.168.86 - - [15/05/2018:09:26:55 +0200] "GET /pve2/ext6/theme-crisp/resources/images/shared/icon-error.png HTTP/1.1" 200 18494
192.168.168.86 - - [15/05/2018:09:28:41 +0200] "POST /api2/extjs/access/ticket HTTP/1.1" 200 656
192.168.168.86 - root@pam [15/05/2018:09:28:41 +0200] "GET /api2/extjs/nodes/localhost/subscription?_dc=1526369321547 HTTP/1.1" 200 207
192.168.168.86 - - [15/05/2018:09:28:41 +0200] "GET /pve2/ext6/theme-crisp/resources/images/shared/icon-warning.png HTTP/1.1" 200 17717
192.168.168.86 - root@pam [15/05/2018:09:28:42 +0200] "GET /api2/json/cluster/tasks HTTP/1.1" 200 1020
192.168.168.86 - root@pam [15/05/2018:09:28:42 +0200] "GET /api2/json/cluster/resources HTTP/1.1" 200 576
192.168.168.86 - root@pam [15/05/2018:09:28:42 +0200] "GET /api2/extjs/version?_dc=1526369321546 HTTP/1.1" 200 89
192.168.168.86 - - [15/05/2018:09:28:42 +0200] "GET /pve2/ext6/theme-crisp/resources/images/grid/sort_asc.png HTTP/1.1" 200 18239
192.168.168.86 - root@pam [15/05/2018:09:28:47 +0200] "GET /api2/json/cluster/tasks HTTP/1.1" 200 1020
192.168.168.86 - root@pam [15/05/2018:09:28:48 +0200] "GET /api2/json/cluster/resources HTTP/1.1" 200 575
192.168.168.86 - root@pam [15/05/2018:09:28:51 +0200] "GET /api2/json/cluster/tasks HTTP/1.1" 200 1016
192.168.168.86 - root@pam [15/05/2018:09:28:51 +0200] "GET /api2/json/cluster/resources HTTP/1.1" 200 573
192.168.168.86 - - [15/05/2018:09:28:52 +0200] "GET /pve2/ext6/theme-crisp/resources/images/button/default-toolbar-small-arrow.png HTTP/1.1" 200 17341
192.168.168.86 - - [15/05/2018:09:28:52 +0200] "GET /pve2/ext6/theme-crisp/resources/images/button/default-toolbar-small-s-arrow.png HTTP/1.1" 200 17377
192.168.168.86 - - [15/05/2018:09:28:52 +0200] "GET /pve2/images/logo-ceph.png HTTP/1.1" 200 488
192.168.168.86 - root@pam [15/05/2018:09:28:52 +0200] "GET /api2/json/nodes/bsi-proxmox/status HTTP/1.1" 200 582
192.168.168.86 - root@pam [15/05/2018:09:28:54 +0200] "GET /api2/json/nodes/bsi-proxmox/status HTTP/1.1" 200 582
192.168.168.86 - root@pam [15/05/2018:09:28:54 +0200] "GET /api2/json/cluster/tasks HTTP/1.1" 200 1016
192.168.168.86 - root@pam [15/05/2018:09:28:54 +0200] "GET /api2/json/cluster/resources HTTP/1.1" 200 562
192.168.168.86 - root@pam [15/05/2018:09:28:55 +0200] "GET /api2/json/nodes/bsi-proxmox/status HTTP/1.1" 200 582
192.168.168.86 - root@pam [15/05/2018:09:28:55 +0200] "GET /api2/json/nodes/bsi-proxmox/apt/update HTTP/1.1" 200 11
192.168.168.86 - root@pam [15/05/2018:09:28:56 +0200] "GET /api2/json/nodes/bsi-proxmox/status HTTP/1.1" 200 616
192.168.168.86 - - [15/05/2018:09:28:56 +0200] "GET /pve2/ext6/theme-crisp/resources/images/grid/hd-pop.png HTTP/1.1" 200 17305
192.168.168.86 - root@pam [15/05/2018:09:28:57 +0200] "GET /api2/extjs/nodes/localhost/subscription?_dc=1526369336840 HTTP/1.1" 200 207
192.168.168.86 - root@pam [15/05/2018:09:28:57 +0200] "GET /api2/json/nodes/bsi-proxmox/status HTTP/1.1" 200 616
192.168.168.86 - root@pam [15/05/2018:09:28:57 +0200] "GET /api2/json/cluster/tasks HTTP/1.1" 200 1016
192.168.168.86 - root@pam [15/05/2018:09:28:58 +0200] "GET /api2/json/cluster/resources HTTP/1.1" 200 573
192.168.168.86 - root@pam [15/05/2018:09:28:58 +0200] "GET /api2/json/nodes/bsi-proxmox/status HTTP/1.1" 200 616
192.168.168.86 - root@pam [15/05/2018:09:28:59 +0200] "GET /api2/json/nodes/bsi-proxmox/status HTTP/1.1" 200 616
192.168.168.86 - root@pam [15/05/2018:09:29:00 +0200] "GET /api2/json/nodes/bsi-proxmox/status HTTP/1.1" 200 616
192.168.168.86 - root@pam [15/05/2018:09:29:00 +0200] "GET /api2/json/cluster/tasks HTTP/1.1" 200 1023
192.168.168.86 - root@pam [15/05/2018:09:29:01 +0200] "GET /api2/json/cluster/resources HTTP/1.1" 200 569
192.168.168.86 - root@pam [15/05/2018:09:29:01 +0200] "GET /api2/json/nodes/bsi-proxmox/status HTTP/1.1" 200 615
192.168.168.86 - root@pam [15/05/2018:09:29:02 +0200] "GET /api2/json/nodes/bsi-proxmox/status HTTP/1.1" 200 615
192.168.168.86 - root@pam [15/05/2018:09:29:03 +0200] "GET /api2/json/nodes/bsi-proxmox/status HTTP/1.1" 200 615
192.168.168.86 - root@pam [15/05/2018:09:29:03 +0200] "GET /api2/json/cluster/tasks HTTP/1.1" 200 1023
192.168.168.86 - root@pam [15/05/2018:09:29:04 +0200] "GET /api2/json/cluster/resources HTTP/1.1" 200 566
192.168.168.86 - root@pam [15/05/2018:09:29:04 +0200] "GET /api2/json/nodes/bsi-proxmox/status HTTP/1.1" 200 615

I suppose, here is the problem :
192.168.168.86 - - [15/05/2018:09:26:55 +0200] "POST /api2/extjs/access/ticket HTTP/1.1" 596 -

But I don't know what does this mean.
 
Unfortunately, there is nothing special:

May 15 09:25:00 bsi-proxmox systemd[1]: Starting Proxmox VE replication runner...
May 15 09:25:00 bsi-proxmox systemd[1]: Started Proxmox VE replication runner.
May 15 09:25:23 bsi-proxmox postfix/smtpd[17684]: connect from localhost.localdomain[127.0.0.1]
May 15 09:25:23 bsi-proxmox postfix/smtpd[17684]: disconnect from localhost.localdomain[127.0.0.1] quit=1 commands=1
May 15 09:25:25 bsi-proxmox sshd[21150]: Connection closed by 127.0.0.1 port 56042 [preauth]
May 15 09:26:00 bsi-proxmox systemd[1]: Starting Proxmox VE replication runner...
May 15 09:26:00 bsi-proxmox systemd[1]: Started Proxmox VE replication runner.
May 15 09:26:23 bsi-proxmox postfix/smtpd[17684]: connect from localhost.localdomain[127.0.0.1]
May 15 09:26:23 bsi-proxmox postfix/smtpd[17684]: disconnect from localhost.localdomain[127.0.0.1] quit=1 commands=1
May 15 09:26:25 bsi-proxmox sshd[21334]: Connection closed by 127.0.0.1 port 56060 [preauth]
May 15 09:27:00 bsi-proxmox systemd[1]: Starting Proxmox VE replication runner...
May 15 09:27:01 bsi-proxmox systemd[1]: Started Proxmox VE replication runner.
May 15 09:27:24 bsi-proxmox postfix/smtpd[17684]: connect from localhost.localdomain[127.0.0.1]
May 15 09:27:24 bsi-proxmox postfix/smtpd[17684]: disconnect from localhost.localdomain[127.0.0.1] quit=1 commands=1
May 15 09:27:25 bsi-proxmox sshd[21517]: Connection closed by 127.0.0.1 port 56078 [preauth]

Any other idea how to debug ?

Edit: I use mainly firefox. (up-to-date)
 
Last edited:
Any other idea how to debug ?

The next time, before you try to login (and you know that this will happen) please open the developer console's network Inspector (CTRL+SHIFT+E), then enter user/password and try to login as usual.
If it failed, as described here, right click in the network view and select "Save All As HAR", then - very important - open the file and redact the password, IP/Hostnames (if they're public) and other possible sensible information then send it to me (my username here + @proxmox.com), or attach it to this thread, whichever you prefer. Maybe I find some hint about this problem...

A few other questions:
* do you access the WebUI through LAN or over (multiple) routers?
* do you always have the WebUI open in the background or does this happen even if you open it in a new tab and directly try to login?
 
Hi,

I have .har with the error. I sent an email.

I access the WebUI throught LAN.
I don't have this WebUI open in background. I just open it to check the updates time to time and the error appear.

Note: I use multiple proxmox server/cluster with and without subscription. It only happens to this one (server without subscription).
 
I have .har with the error. I sent an email.

Thanks I've got it, the thing which looks weird is that the CSRFPrevetionToken is null for the POST request...
Could you trigger this also if you load the side twice before logging in?

Note: I use multiple proxmox server/cluster with and without subscription. It only happens to this one (server without subscription).
Hmm, do they differ in package versions?
 
Hmm, do they differ in package versions?
Since some have premium subscription and some haven't, yes but all up to date.

Thanks I've got it, the thing which looks weird is that the CSRFPrevetionToken is null for the POST request...
Could you trigger this also if you load the side twice before logging in?
I just try, the issue doesn't appear but I need to retry tuesday. (It always happens between 2 spaced sessions)
But, I note on a successful connection (Response 200 and logged-in), I have also a header "CSRFPreventionToken" set as null.
 
Hi and good day,

I comment on this thread, because I think it makes more sense than to open a new one.

I have also an issue with the login behind a httpd reverse proxy.

The interesting part why I comment now, is that it was working from scratch when i repload the page before login a second time:
I could offer to post my httpd configuration that is on the CentOS VM inside the Proxmox VM.

This issue only appears when I remotely login via my external URL from the Internet
My Mikrotik router NATs incoming port 443 to the CentOS 7 VM running httpd
httpd reverse proxies the questioned subdomain to the proxmox host (IP:8006) a lot of rules are activated, dont ask me how i succeeded :)
So the last issue is only with the login, It was working already, it is possible that it does not work since the last time when I upgraded proxmox

and now you show me a way around:
- GO TO URL and wait webpage for loading
- PRESS F5 and load Proxmox
Result:
- Login is working only with Linux PAM standard authentication
- Login with Domain Authentication with Samba Domain Controller running 2008R2 Domain is even not working via external URL, when I reload the page, maybe it is working as designed, because my domain controller is not on the Internet
- Login via Two Factor I still try to achieve, I did not succeed yet, I would like to open a seperate thread for this. just first trying to get my reverse proxy settings correct

just retried and i can reproduce the login successful on second reload, I am not allowed to press login on the first load of the webpage
(only Linux PAM standard authentication)

Virtual Environment 5.1-47
If you would like to know more details, just go ahead an ask before I post irrelevant information

Thanks for reading
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!