[SOLVED] TFA [2FA] Two-Factor Authentication Broken in Proxmox 5.4

Discussion in 'Proxmox VE: Installation and configuration' started by SierraEcho, Apr 14, 2019 at 09:07.

Tags:
  1. SierraEcho

    SierraEcho New Member
    Proxmox Subscriber

    Joined:
    Sunday
    Messages:
    8
    Likes Received:
    1
    After upgrading to 5.4-3 TFA [2FA] seems to be completely broken ... Both Yubico and OATH methods now spit out an authentication error in GUI

    Screenshot 2019-04-14 at 07.37.58.png

    Interestingly the log shows that the user has been authenticated correctly ??

    Screenshot 2019-04-14 at 07.58.15.png

    It all works fine in 5.3-11

    Tested on 4 servers in total - exactly the same symptoms.

    Code:
    root@Beast-01:~# pveversion -v
    proxmox-ve: 5.4-1 (running kernel: 4.15.18-12-pve)
    pve-manager: 5.4-3 (running version: 5.4-3/0a6eaa62)
    pve-kernel-4.15: 5.3-3
    pve-kernel-4.15.18-12-pve: 4.15.18-35
    pve-kernel-4.15.18-10-pve: 4.15.18-32
    corosync: 2.4.4-pve1
    criu: 2.11.1-1~bpo90
    glusterfs-client: 3.8.8-1
    ksm-control-daemon: 1.2-2
    libjs-extjs: 6.0.1-2
    libpve-access-control: 5.1-8
    libpve-apiclient-perl: 2.0-5
    libpve-common-perl: 5.0-50
    libpve-guest-common-perl: 2.0-20
    libpve-http-server-perl: 2.0-13
    libpve-storage-perl: 5.0-41
    libqb0: 1.0.3-1~bpo9
    lvm2: 2.02.168-pve6
    lxc-pve: 3.1.0-3
    lxcfs: 3.0.3-pve1
    novnc-pve: 1.0.0-3
    proxmox-widget-toolkit: 1.0-25
    pve-cluster: 5.0-36
    pve-container: 2.0-37
    pve-docs: 5.4-2
    pve-edk2-firmware: 1.20190312-1
    pve-firewall: 3.0-19
    pve-firmware: 2.0-6
    pve-ha-manager: 2.0-9
    pve-i18n: 1.1-4
    pve-libspice-server1: 0.14.1-2
    pve-qemu-kvm: 2.12.1-3
    pve-xtermjs: 3.12.0-1
    qemu-server: 5.0-50
    smartmontools: 6.5+svn4324-1
    spiceterm: 3.0-5
    vncterm: 1.5-3
    zfsutils-linux: 0.7.13-pve1~bpo2
     
  2. wbumiller

    wbumiller Proxmox Staff Member
    Staff Member

    Joined:
    Jun 23, 2015
    Messages:
    639
    Likes Received:
    80
    There have been some changes to the TFA handling and how the configuration is stored in the backend, but both old and new configurations should work with the new code.

    Can you describe the login process you see on your UI?
    -) Do you see the OTP input box below the password in the login window? (That should be gone with the update, but should still work if it is used (eg. old browser cache) - while this *should* still work, try clearing the browser cache anyway.)
    -) Do you get a 2nd factor popup window after clicking login? (the new UI performs the 2nd factor in a 2nd step)

    Also, can you try setting up TOTP for a user via the UI (the new 'TFA' button in the user tab) to see if the same happens when using the new configuration method?
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. SierraEcho

    SierraEcho New Member
    Proxmox Subscriber

    Joined:
    Sunday
    Messages:
    8
    Likes Received:
    1
    Yes, I have cleared the cache, and I can see the new process in the UI (separate box for OTP that pops up after the main credentials were entered)

    Yes, as part of the testing I have used the per user TFA button to generate the QR code and it did not work as well.

    As an update, I can see in the logs that it seems that it does not parse the entered OTP to the backend - screenshot attached.

    Screenshot 2019-04-14 at 09.21.38.png

    So it seems that it passes the AD auth and then fails on verifying the OTP.
    I have tried that with both TOTP and Yubico, on all of the authentication providers (PAM, PV, AD) with the same results.

    Just to make it clear, it is affecting both upgraded installs (from 5.3-11) and fresh installs (5.4-3).
     
    #3 SierraEcho, Apr 15, 2019 at 10:18
    Last edited: Apr 15, 2019 at 11:47
  4. wbumiller

    wbumiller Proxmox Staff Member
    Staff Member

    Joined:
    Jun 23, 2015
    Messages:
    639
    Likes Received:
    80
    Did you also remove the 'yubico' auth type from the authentication realm when you did this? The realm's configured TFA type will conflict otherwise.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  5. SierraEcho

    SierraEcho New Member
    Proxmox Subscriber

    Joined:
    Sunday
    Messages:
    8
    Likes Received:
    1
    Hi,
    Yepp, I have tested that and you are indeed correct, the TFA = yubico declared in realm stopped per-user (new TFA button) OATH from working correctly. Now I have per-user OATH working, but still it does not respect the old keys that were working in 5.3. Just to shed a bit of light on the usage scenario - I've had 3 different OATH keys for the root user in pam realm, and per user Yubikey in AD realm. Yubikey still does not work - I am getting 'msg=missing one time password for two-factor authentication 'yubico'' after NPS on the domain controller issues access-accept, also how do I get back to being able to use multiple private keys for the single user in OATH ?

    Thanks.
     
  6. SierraEcho

    SierraEcho New Member
    Proxmox Subscriber

    Joined:
    Sunday
    Messages:
    8
    Likes Received:
    1
    Sorry to bump that artificially, but can anyone at least confirm if their Yubico/OATH TFA is working after the upgrade to 5.4?
     
  7. janssensm

    janssensm Member
    Proxmox Subscriber

    Joined:
    Dec 18, 2016
    Messages:
    40
    Likes Received:
    13
  8. SierraEcho

    SierraEcho New Member
    Proxmox Subscriber

    Joined:
    Sunday
    Messages:
    8
    Likes Received:
    1
    Thanks @janssensm for confirming, but the U2F is the new 5.4 implementation of TFA, and it was working all along.

    Ok, I have found what the issue is, and yes, it is broken in 5.4. What is actually broken is the parsing (as suspected) from the new OTP window introduced in 5.4 and it is affecting just Yubico integration (OATH started working after re-starting the server - thats the OLD style OATH that was there before update, not the new per-user implementation introduced in 5.4 (TFA button)).

    Drumroll ... It seems that the problem lies in the fact that Yubikey automatically sends the CR after the OTP when you touch it, and that's what is confusing the parser, so it does not parse the content of the input field, hence
    Code:
    msg=missing one time password for two-factor authentication 'yubico'
    . When you press the button and harvest the OTP in the text editor and then COPY it to the OTP window and manually press return key / click on Login button it works as expected.

    Now, can we have a fix please ??

    Thanks,
    SierraEcho
     
    janssensm likes this.
  9. wbumiller

    wbumiller Proxmox Staff Member
    Staff Member

    Joined:
    Jun 23, 2015
    Messages:
    639
    Likes Received:
    80
    Thanks for digging into this.
    That actually helped pinpointing an issue.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  10. SierraEcho

    SierraEcho New Member
    Proxmox Subscriber

    Joined:
    Sunday
    Messages:
    8
    Likes Received:
    1
    Thanks Wolfgang, I can see that you have assigned the bug on bugzilla to yourself already. Looking forward to a patch then.

    S.E.
     
  11. t.lamprecht

    t.lamprecht Proxmox Staff Member
    Staff Member

    Joined:
    Jul 28, 2015
    Messages:
    1,086
    Likes Received:
    131
    Wolfgang indeed posted a patch and as a few other small fixes/improvements were already there we bumped the package version, it is currently available through our pvetest repo

    You could manually install it without switching (temporarily) to pvetest via CLI like:
    Code:
    # wget http://download.proxmox.com/debian/pve/dists/stretch/pvetest/binary-amd64/pve-manager_5.4-4_amd64.deb http://download.proxmox.com/debian/pve/dists/stretch/pvetest/binary-amd64/proxmox-widget-toolkit_1.0-26_all.deb
    # apt install ./proxmox-widget-toolkit_1.0-26_all.deb ./pve-manager_5.4-4_amd64.deb
    
    Ensure to force reload the WebUI afterwards, it normally should happen automatically through our index versioning, but still..
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  12. SierraEcho

    SierraEcho New Member
    Proxmox Subscriber

    Joined:
    Sunday
    Messages:
    8
    Likes Received:
    1
    Klasse!

    I can see what was causing the problem now ;)

    Screenshot 2019-04-16 at 15.54.07.png
    Thanks for sorting that out so quickly, marking as resolved.
     
  13. t.lamprecht

    t.lamprecht Proxmox Staff Member
    Staff Member

    Joined:
    Jul 28, 2015
    Messages:
    1,086
    Likes Received:
    131
    Yeah, this slipped review, normally it does not even is problematic, just if the \n is already sent by the yubico key... ExtJS (the framework for the WebUI) does a bit of magic here...
    Anyway, thanks for reporting and quick testing!
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice