[SOLVED] Failed to verify TOTP challenge

[cglmicro]

Hi.
On one of my PBS, the 2FA (TOTP) stopped working.
I had to SSH and "mv /etc/proxmox-backup/tfa.json /etc/proxmox-backup/tfa.json_backup" to be able to log back in.
Now I'm trying to add 2FA again in PBS > CONFIGURATION ACCESS CONTROL > TWO FACTOR AUTHENTICATION, but I always get "FAILED TO VERIFIEY TOTP CHALLENGE" when I scan the code and I try to verify it.

Any idea where I can start my tests ?

 

[t.lamprecht]

Hi,

I had to SSH and "mv /etc/proxmox-backup/tfa.json /etc/proxmox-backup/tfa.json_backup" to be able to log back in.

FYI, there's pveum user tfa delete <userid> too to avoid deleting all of the users second factors, but just one of them (for multi user PVE setups).

Any idea where I can start my tests ?

Is the time synchronised on the server, e.g., check systemctl status chrony (chrony is our default NTP daemon since a while, but there could be others too for older installations or if got manually switched). For a quick check it may be enough to execute the date command.

If that's not the case then a full pveversion -v output plus details about the TOTP client used and possibly journal/syslog entries around the time of login could be interesting. Also, any recent upgrade done or other change made in the setup or its environment?

 

[cglmicro]

You are probably right.
Chrony is not installed on my server, but NTP is and it's not working:

root@pbs102:~# systemctl status ntp
● ntp.service
     Loaded: masked (Reason: Unit ntp.service is masked.)
     Active: failed (Result: exit-code) since Thu 2022-06-30 16:45:17 EDT; 1min 53s ago
        CPU: 15ms

Jun 30 16:45:17 pbs102 systemd[1]: Starting Network Time Service...
Jun 30 16:45:17 pbs102 systemd[1]: ntp.service: Control process exited, code=exited, status=1/FAILURE
Jun 30 16:45:17 pbs102 systemd[1]: ntp.service: Failed with result 'exit-code'.
Jun 30 16:45:17 pbs102 systemd[1]: Failed to start Network Time Service.

root@pbs102:~# systemctl start ntp
Failed to start ntp.service: Unit ntp.service is masked.

root@pbs102:~# journalctl -xe | grep ntp
Jun 30 16:45:17 pbs102 systemd[1]: ntp.service: Control process exited, code=exited, status=1/FAILURE
░░ An ExecStart= process belonging to unit ntp.service has exited.
Jun 30 16:45:17 pbs102 systemd[1]: ntp.service: Failed with result 'exit-code'.
░░ The unit ntp.service has entered the 'failed' state with result 'exit-code'.
░░ Subject: A start job for unit ntp.service has failed
░░ A start job for unit ntp.service has finished with a failure.
Jun 30 16:45:17 pbs102 kernel: audit: type=1400 audit(1656621917.631:32): apparmor="DENIED" operation="create" profile="/usr/sbin/ntpd" pid=4061380 comm="ntpd" family="unix" sock_type="dgram" protocol=0 requested_mask="create" denied_mask="create" addr=none
Jun 30 16:45:17 pbs102 kernel: audit: type=1400 audit(1656621917.631:33): apparmor="DENIED" operation="create" profile="/usr/sbin/ntpd" pid=4061380 comm="ntpd" family="unix" sock_type="dgram" protocol=0 requested_mask="create" denied_mask="create" addr=none
Jun 30 16:45:17 pbs102 kernel: audit: type=1400 audit(1656621917.631:34): apparmor="DENIED" operation="create" profile="/usr/sbin/ntpd" pid=4061380 comm="ntpd" family="unix" sock_type="dgram" protocol=0 requested_mask="create" denied_mask="create" addr=none
Jun 30 16:45:17 pbs102 kernel: audit: type=1400 audit(1656621917.631:35): apparmor="DENIED" operation="create" profile="/usr/sbin/ntpd" pid=4061380 comm="ntpd" family="unix" sock_type="dgram" protocol=0 requested_mask="create" denied_mask="create" addr=none
Jun 30 16:45:17 pbs102 kernel: audit: type=1400 audit(1656621917.631:36): apparmor="DENIED" operation="create" profile="/usr/sbin/ntpd" p

Thank you.

 

[cglmicro]

SOLVED !
It was a conflict with APPARMOR solved with this:

ln -s /etc/apparmor.d/usr.sbin.ntpd /etc/apparmor.d/disable/ 
apparmor_parser -R /etc/apparmor.d/usr.sbin.ntpd

 

[kuerbis42]

Thx. Same problem. Works!

maybe add:

systemctl restart ntp

 

[t.lamprecht]

Chrony is not installed on my server, but NTP is and it's not working:

FYI, this is tracked under entry #4576 – see especially the third comment for why this happens on PBS (and PMG) but not PVE.

 

[killproc]

I had problems because the Hetzner firewall didn't let through NTP, so it looks like, only Hetzner's NTP works:
ntp1.hetzner.de
ntp2.hetzner.com
ntp3.hetzner.net
Then it stopped working after the time change - for ssh (pam) it was enough to set the time back locally to get the correct TOTP, but on the web the TOTP was blocked.
Thanks to the post I found the correct solution;
pveum user tfa unlock <user-id>
To see the Lock, use;
pveum user list
Thanks