[SOLVED] TFA [2FA] Two-Factor Authentication Broken in Proxmox 5.4

[SierraEcho]

After upgrading to 5.4-3 TFA [2FA] seems to be completely broken ... Both Yubico and OATH methods now spit out an authentication error in GUI



Interestingly the log shows that the user has been authenticated correctly ??



It all works fine in 5.3-11

Tested on 4 servers in total - exactly the same symptoms.

root@Beast-01:~# pveversion -v
proxmox-ve: 5.4-1 (running kernel: 4.15.18-12-pve)
pve-manager: 5.4-3 (running version: 5.4-3/0a6eaa62)
pve-kernel-4.15: 5.3-3
pve-kernel-4.15.18-12-pve: 4.15.18-35
pve-kernel-4.15.18-10-pve: 4.15.18-32
corosync: 2.4.4-pve1
criu: 2.11.1-1~bpo90
glusterfs-client: 3.8.8-1
ksm-control-daemon: 1.2-2
libjs-extjs: 6.0.1-2
libpve-access-control: 5.1-8
libpve-apiclient-perl: 2.0-5
libpve-common-perl: 5.0-50
libpve-guest-common-perl: 2.0-20
libpve-http-server-perl: 2.0-13
libpve-storage-perl: 5.0-41
libqb0: 1.0.3-1~bpo9
lvm2: 2.02.168-pve6
lxc-pve: 3.1.0-3
lxcfs: 3.0.3-pve1
novnc-pve: 1.0.0-3
proxmox-widget-toolkit: 1.0-25
pve-cluster: 5.0-36
pve-container: 2.0-37
pve-docs: 5.4-2
pve-edk2-firmware: 1.20190312-1
pve-firewall: 3.0-19
pve-firmware: 2.0-6
pve-ha-manager: 2.0-9
pve-i18n: 1.1-4
pve-libspice-server1: 0.14.1-2
pve-qemu-kvm: 2.12.1-3
pve-xtermjs: 3.12.0-1
qemu-server: 5.0-50
smartmontools: 6.5+svn4324-1
spiceterm: 3.0-5
vncterm: 1.5-3
zfsutils-linux: 0.7.13-pve1~bpo2

 

[wbumiller]

There have been some changes to the TFA handling and how the configuration is stored in the backend, but both old and new configurations should work with the new code.

Can you describe the login process you see on your UI?
-) Do you see the OTP input box below the password in the login window? (That should be gone with the update, but should still work if it is used (eg. old browser cache) - while this *should* still work, try clearing the browser cache anyway.)
-) Do you get a 2nd factor popup window after clicking login? (the new UI performs the 2nd factor in a 2nd step)

Also, can you try setting up TOTP for a user via the UI (the new 'TFA' button in the user tab) to see if the same happens when using the new configuration method?

 

[SierraEcho]

There have been some changes to the TFA handling and how the configuration is stored in the backend, but both old and new configurations should work with the new code.

Can you describe the login process you see on your UI?
-) Do you see the OTP input box below the password in the login window? (That should be gone with the update, but should still work if it is used (eg. old browser cache) - while this *should* still work, try clearing the browser cache anyway.)
-) Do you get a 2nd factor popup window after clicking login? (the new UI performs the 2nd factor in a 2nd step)

Also, can you try setting up TOTP for a user via the UI (the new 'TFA' button in the user tab) to see if the same happens when using the new configuration method?

Yes, I have cleared the cache, and I can see the new process in the UI (separate box for OTP that pops up after the main credentials were entered)

Yes, as part of the testing I have used the per user TFA button to generate the QR code and it did not work as well.

As an update, I can see in the logs that it seems that it does not parse the entered OTP to the backend - screenshot attached.



So it seems that it passes the AD auth and then fails on verifying the OTP.
I have tried that with both TOTP and Yubico, on all of the authentication providers (PAM, PV, AD) with the same results.

Just to make it clear, it is affecting both upgraded installs (from 5.3-11) and fresh installs (5.4-3).

 

[wbumiller]

Yes, as part of the testing I have used the per user TFA button to generate the QR code and it did not work as well.

Did you also remove the 'yubico' auth type from the authentication realm when you did this? The realm's configured TFA type will conflict otherwise.

 

[SierraEcho]

Did you also remove the 'yubico' auth type from the authentication realm when you did this? The realm's configured TFA type will conflict otherwise.

Hi,
Yepp, I have tested that and you are indeed correct, the TFA = yubico declared in realm stopped per-user (new TFA button) OATH from working correctly. Now I have per-user OATH working, but still it does not respect the old keys that were working in 5.3. Just to shed a bit of light on the usage scenario - I've had 3 different OATH keys for the root user in pam realm, and per user Yubikey in AD realm. Yubikey still does not work - I am getting 'msg=missing one time password for two-factor authentication 'yubico'' after NPS on the domain controller issues access-accept, also how do I get back to being able to use multiple private keys for the single user in OATH ?

Thanks.

 

[SierraEcho]

Sorry to bump that artificially, but can anyone at least confirm if their Yubico/OATH TFA is working after the upgrade to 5.4?

 

[janssensm]

Perhaps this thread is just concerning TFA OATH, but I can confirm that the Yubico Security Key with u2f is working with 5.4.
https://www.yubico.com/products/yubikey-hardware/compare-security-key-series/
This is the cheaper key without OATH and other features.

 

[SierraEcho]

Thanks @janssensm for confirming, but the U2F is the new 5.4 implementation of TFA, and it was working all along.

Ok, I have found what the issue is, and yes, it is broken in 5.4. What is actually broken is the parsing (as suspected) from the new OTP window introduced in 5.4 and it is affecting just Yubico integration (OATH started working after re-starting the server - thats the OLD style OATH that was there before update, not the new per-user implementation introduced in 5.4 (TFA button)).

Drumroll ... It seems that the problem lies in the fact that Yubikey automatically sends the CR after the OTP when you touch it, and that's what is confusing the parser, so it does not parse the content of the input field, hence

msg=missing one time password for two-factor authentication 'yubico'

. When you press the button and harvest the OTP in the text editor and then COPY it to the OTP window and manually press return key / click on Login button it works as expected.

Now, can we have a fix please ??

Thanks,
SierraEcho

 

[wbumiller]

Thanks for digging into this.

When you press the button and harvest the OTP in the text editor and then COPY it to the OTP window and manually press return key / click on Login button it works as expected.

That actually helped pinpointing an issue.

 

[SierraEcho]

Thanks for digging into this.

That actually helped pinpointing an issue.

Thanks Wolfgang, I can see that you have assigned the bug on bugzilla to yourself already. Looking forward to a patch then.

S.E.

 

[t.lamprecht]

Wolfgang indeed posted a patch and as a few other small fixes/improvements were already there we bumped the package version, it is currently available through our pvetest repo

You could manually install it without switching (temporarily) to pvetest via CLI like:

# wget http://download.proxmox.com/debian/pve/dists/stretch/pvetest/binary-amd64/pve-manager_5.4-4_amd64.deb http://download.proxmox.com/debian/pve/dists/stretch/pvetest/binary-amd64/proxmox-widget-toolkit_1.0-26_all.deb
# apt install ./proxmox-widget-toolkit_1.0-26_all.deb ./pve-manager_5.4-4_amd64.deb

Ensure to force reload the WebUI afterwards, it normally should happen automatically through our index versioning, but still..

 

[SierraEcho]

Klasse!

I can see what was causing the problem now


Thanks for sorting that out so quickly, marking as resolved.

 

[t.lamprecht]

Yeah, this slipped review, normally it does not even is problematic, just if the \n is already sent by the yubico key... ExtJS (the framework for the WebUI) does a bit of magic here...
Anyway, thanks for reporting and quick testing!