Yes fail2ban this is one of the first things done when starting the server.
You talk about double authentication with otp. On PVE I do it without problem. Off on PMG I do not see the option. The oathkeygen command is not taken into account on PMG.
In debian stretch it's just: # apt-get install libpam-yubico
Create a yubikey mapping file /etc/yubikey_mappings <first user name>:<Yubikey token ID1>:<Yubikey token ID2>:….
<second user name>:<Yubikey token ID3>:<Yubikey token ID4>:….
# vi /etc/pam.d/auth_common
Add first line after comments: auth required pam_yubico.so id=[Your API Client ID] authfile=/etc/yubikey_mappings
Find that line and add try_first_pass auth [success=1 default=ignore] pam_unix.so nullok_secure try_first_pass
done - GUI and all other pam authentication (ssh, su, ....) is protected.
PS: Test with a non root user first. If it does not work for root (or ur key get lost), u can always login via console and recovery mode.
FW settings https/out to Yubico Auth Servers should be allowed.