[SOLVED] TFA [2FA] Two-Factor Authentication Broken in Proxmox 5.4

Apr 14, 2019
8
1
3
53
After upgrading to 5.4-3 TFA [2FA] seems to be completely broken ... Both Yubico and OATH methods now spit out an authentication error in GUI

Screenshot 2019-04-14 at 07.37.58.png

Interestingly the log shows that the user has been authenticated correctly ??

Screenshot 2019-04-14 at 07.58.15.png

It all works fine in 5.3-11

Tested on 4 servers in total - exactly the same symptoms.

Code:
root@Beast-01:~# pveversion -v
proxmox-ve: 5.4-1 (running kernel: 4.15.18-12-pve)
pve-manager: 5.4-3 (running version: 5.4-3/0a6eaa62)
pve-kernel-4.15: 5.3-3
pve-kernel-4.15.18-12-pve: 4.15.18-35
pve-kernel-4.15.18-10-pve: 4.15.18-32
corosync: 2.4.4-pve1
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.1-8
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-50
libpve-guest-common-perl: 2.0-20
libpve-http-server-perl: 2.0-13
libpve-storage-perl: 5.0-41
libqb0: 1.0.3-1~bpo9
lvm2: 2.02.168-pve6
lxc-pve: 3.1.0-3
lxcfs: 3.0.3-pve1
novnc-pve: 1.0.0-3
proxmox-widget-toolkit: 1.0-25
pve-cluster: 5.0-36
pve-container: 2.0-37
pve-docs: 5.4-2
pve-edk2-firmware: 1.20190312-1
pve-firewall: 3.0-19
pve-firmware: 2.0-6
pve-ha-manager: 2.0-9
pve-i18n: 1.1-4
pve-libspice-server1: 0.14.1-2
pve-qemu-kvm: 2.12.1-3
pve-xtermjs: 3.12.0-1
qemu-server: 5.0-50
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.13-pve1~bpo2
 
There have been some changes to the TFA handling and how the configuration is stored in the backend, but both old and new configurations should work with the new code.

Can you describe the login process you see on your UI?
-) Do you see the OTP input box below the password in the login window? (That should be gone with the update, but should still work if it is used (eg. old browser cache) - while this *should* still work, try clearing the browser cache anyway.)
-) Do you get a 2nd factor popup window after clicking login? (the new UI performs the 2nd factor in a 2nd step)

Also, can you try setting up TOTP for a user via the UI (the new 'TFA' button in the user tab) to see if the same happens when using the new configuration method?
 
There have been some changes to the TFA handling and how the configuration is stored in the backend, but both old and new configurations should work with the new code.

Can you describe the login process you see on your UI?
-) Do you see the OTP input box below the password in the login window? (That should be gone with the update, but should still work if it is used (eg. old browser cache) - while this *should* still work, try clearing the browser cache anyway.)
-) Do you get a 2nd factor popup window after clicking login? (the new UI performs the 2nd factor in a 2nd step)

Also, can you try setting up TOTP for a user via the UI (the new 'TFA' button in the user tab) to see if the same happens when using the new configuration method?

Yes, I have cleared the cache, and I can see the new process in the UI (separate box for OTP that pops up after the main credentials were entered)

Yes, as part of the testing I have used the per user TFA button to generate the QR code and it did not work as well.

As an update, I can see in the logs that it seems that it does not parse the entered OTP to the backend - screenshot attached.

Screenshot 2019-04-14 at 09.21.38.png

So it seems that it passes the AD auth and then fails on verifying the OTP.
I have tried that with both TOTP and Yubico, on all of the authentication providers (PAM, PV, AD) with the same results.

Just to make it clear, it is affecting both upgraded installs (from 5.3-11) and fresh installs (5.4-3).
 
Last edited:
Yes, as part of the testing I have used the per user TFA button to generate the QR code and it did not work as well.
Did you also remove the 'yubico' auth type from the authentication realm when you did this? The realm's configured TFA type will conflict otherwise.
 
Did you also remove the 'yubico' auth type from the authentication realm when you did this? The realm's configured TFA type will conflict otherwise.

Hi,
Yepp, I have tested that and you are indeed correct, the TFA = yubico declared in realm stopped per-user (new TFA button) OATH from working correctly. Now I have per-user OATH working, but still it does not respect the old keys that were working in 5.3. Just to shed a bit of light on the usage scenario - I've had 3 different OATH keys for the root user in pam realm, and per user Yubikey in AD realm. Yubikey still does not work - I am getting 'msg=missing one time password for two-factor authentication 'yubico'' after NPS on the domain controller issues access-accept, also how do I get back to being able to use multiple private keys for the single user in OATH ?

Thanks.
 
Thanks @janssensm for confirming, but the U2F is the new 5.4 implementation of TFA, and it was working all along.

Ok, I have found what the issue is, and yes, it is broken in 5.4. What is actually broken is the parsing (as suspected) from the new OTP window introduced in 5.4 and it is affecting just Yubico integration (OATH started working after re-starting the server - thats the OLD style OATH that was there before update, not the new per-user implementation introduced in 5.4 (TFA button)).

Drumroll ... It seems that the problem lies in the fact that Yubikey automatically sends the CR after the OTP when you touch it, and that's what is confusing the parser, so it does not parse the content of the input field, hence
Code:
msg=missing one time password for two-factor authentication 'yubico'
. When you press the button and harvest the OTP in the text editor and then COPY it to the OTP window and manually press return key / click on Login button it works as expected.

Now, can we have a fix please ??

Thanks,
SierraEcho
 
  • Like
Reactions: janssensm
Thanks for digging into this.
When you press the button and harvest the OTP in the text editor and then COPY it to the OTP window and manually press return key / click on Login button it works as expected.
That actually helped pinpointing an issue.
 
Wolfgang indeed posted a patch and as a few other small fixes/improvements were already there we bumped the package version, it is currently available through our pvetest repo

You could manually install it without switching (temporarily) to pvetest via CLI like:
Code:
# wget http://download.proxmox.com/debian/pve/dists/stretch/pvetest/binary-amd64/pve-manager_5.4-4_amd64.deb http://download.proxmox.com/debian/pve/dists/stretch/pvetest/binary-amd64/proxmox-widget-toolkit_1.0-26_all.deb
# apt install ./proxmox-widget-toolkit_1.0-26_all.deb ./pve-manager_5.4-4_amd64.deb

Ensure to force reload the WebUI afterwards, it normally should happen automatically through our index versioning, but still..
 
Yeah, this slipped review, normally it does not even is problematic, just if the \n is already sent by the yubico key... ExtJS (the framework for the WebUI) does a bit of magic here...
Anyway, thanks for reporting and quick testing!
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!