Proxmox Backup Server 4.0 BETA released!

Has anyone had this issue with the installed kernels after the update to the PBS Beta?

Code:
root@pbs:~# apt update
Hit:1 http://ftp.au.debian.org/debian trixie InRelease
Hit:2 http://ftp.au.debian.org/debian trixie-updates InRelease                     
Hit:3 http://security.debian.org trixie-security InRelease                         
Hit:4 http://download.proxmox.com/debian/pbs trixie InRelease
All packages are up to date.           
root@pbs:~# apt dist-upgrade
Summary:                        
  Upgrading: 0, Installing: 0, Removing: 0, Not Upgrading: 0
  1 not fully installed or removed.
  Space needed: 0 B / 449 GB available

Continue? [Y/n] 
Setting up initramfs-tools (0.148.3) ...
update-initramfs: deferring update (trigger activated)
Processing triggers for initramfs-tools (0.148.3) ...
update-initramfs: Generating /boot/initrd.img-6.14.8-2-pve
I: The initramfs will attempt to resume from /dev/dm-0
I: (/dev/mapper/pbs-swap)
I: Set the RESUME variable to override this.
Running hook script 'zz-proxmox-boot'..
Re-executing '/etc/kernel/postinst.d/zz-proxmox-boot' in new private mount namespace..
No /etc/kernel/cmdline found - falling back to /proc/cmdline
Copying and configuring kernels on /dev/disk/by-uuid/3255-912E
 Copying kernel and creating boot-entry for 6.14.8-2-pve
run-parts: /etc/initramfs/post-update.d//proxmox-boot-sync exited with return code 1
dpkg: error processing package initramfs-tools (--configure):
 installed initramfs-tools package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 initramfs-tools
Error: Sub-process /usr/bin/dpkg returned an error code (1)
root@pbs:~#
 
Last edited:
No, but the error message being incorrect is only cosmetic (as the wrong error state was logged), the underlying checks are correct. Please post the output of proxmox-backup-manager version --verbose --output-format json-pretty, best as file attachment

For what its worth - I'm getting this too.

Code:
Aug 06 16:06:07 pbs unix_chkpwd[23232]: password check failed for user (root)
Aug 06 16:06:07 pbs proxmox-backup-api[612]: pam_unix(proxmox-backup-auth:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost=<ip>  user=root
Aug 06 16:06:07 pbs proxmox-backup-api[612]: authentication failure; rhost=[<ip>]:39034 user=root@pam msg=authentication error - SUCCESS (0)
Aug 06 16:06:09 pbs proxmox-backup-api[612]: authentication failure; rhost=[<ip>]:48168 user=root@pam msg=authentication error - SUCCESS (0)
Aug 06 16:06:10 pbs unix_chkpwd[23234]: password check failed for user (root)
Aug 06 16:06:10 pbs proxmox-backup-api[612]: pam_unix(proxmox-backup-auth:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost=<ip>  user=root
Aug 06 16:06:12 pbs unix_chkpwd[23236]: password check failed for user (root)
Aug 06 16:06:12 pbs proxmox-backup-api[612]: pam_unix(proxmox-backup-auth:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost=<ip>  user=root
Aug 06 16:06:12 pbs proxmox-backup-api[612]: authentication failure; rhost=[<ip>]:48186 user=root@pam msg=authentication error - SUCCESS (0)
Aug 06 16:06:13 pbs proxmox-backup-api[612]: authentication failure; rhost=[<ip>]:48180 user=root@pam msg=authentication error - SUCCESS (0)
Aug 06 16:06:14 pbs unix_chkpwd[23238]: password check failed for user (root)
Aug 06 16:06:14 pbs proxmox-backup-api[612]: pam_unix(proxmox-backup-auth:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost=<ip>  user=root
Aug 06 16:06:16 pbs proxmox-backup-api[612]: authentication failure; rhost=[<ip>]:48198 user=root@pam msg=authentication error - SUCCESS (0)
Aug 06 16:06:19 pbs proxmox-backup-api[612]: POST /api2/json/access/ticket: 401 Unauthorized: [client [<ip>]:48198] permission check failed.
Aug 06 16:08:10 pbs unix_chkpwd[23244]: password check failed for user (root)
Aug 06 16:08:10 pbs proxmox-backup-api[612]: pam_unix(proxmox-backup-auth:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost=<ip>  user=root
Aug 06 16:08:12 pbs proxmox-backup-api[612]: authentication failure; rhost=[<ip>]:48198 user=root@pam msg=authentication error - SUCCESS (0)
Aug 06 16:08:15 pbs proxmox-backup-api[612]: POST /api2/json/access/ticket: 401 Unauthorized: [client [<ip>]:48198] permission check failed.

I keep getting logged out of the web UI.

I reset the password via SSH using `passwd` - as well as via `proxmox-backup-manager user update root@pam --password` - but the password check still fails.

Code:
root@pbs:~# proxmox-backup-manager version --verbose
proxmox-backup                           4.0.0            running kernel: 6.14.8-2-pve
proxmox-backup-server                    4.0.10-1         running version: 4.0.10     
proxmox-kernel-helper                    9.0.3                                        
proxmox-kernel-6.14.8-2-pve-signed       6.14.8-2                                     
proxmox-kernel-6.14                      6.14.8-2                                     
proxmox-kernel-6.14.8-1-bpo12-pve-signed 6.14.8-1~bpo12+1                             
ifupdown2                                3.3.0-1+pmx9                                 
libjs-extjs                              7.0.0-5                                      
proxmox-backup-docs                      4.0.10-1                                     
proxmox-backup-client                    4.0.10-1                                     
proxmox-mail-forward                     1.0.2                                        
proxmox-mini-journalreader               1.6                                          
proxmox-offline-mirror-helper            0.7.0                                        
proxmox-widget-toolkit                   5.0.4                                        
pve-xtermjs                              5.5.0-2                                      
smartmontools                            7.4-pve1                                     
zfsutils-linux                           2.3.3-pve1
 
For what its worth - I'm getting this too.

Code:
Aug 06 16:06:07 pbs unix_chkpwd[23232]: password check failed for user (root)
Aug 06 16:06:07 pbs proxmox-backup-api[612]: pam_unix(proxmox-backup-auth:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost=<ip>  user=root
Aug 06 16:06:07 pbs proxmox-backup-api[612]: authentication failure; rhost=[<ip>]:39034 user=root@pam msg=authentication error - SUCCESS (0)
Aug 06 16:06:09 pbs proxmox-backup-api[612]: authentication failure; rhost=[<ip>]:48168 user=root@pam msg=authentication error - SUCCESS (0)
Aug 06 16:06:10 pbs unix_chkpwd[23234]: password check failed for user (root)
Aug 06 16:06:10 pbs proxmox-backup-api[612]: pam_unix(proxmox-backup-auth:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost=<ip>  user=root
Aug 06 16:06:12 pbs unix_chkpwd[23236]: password check failed for user (root)
Aug 06 16:06:12 pbs proxmox-backup-api[612]: pam_unix(proxmox-backup-auth:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost=<ip>  user=root
Aug 06 16:06:12 pbs proxmox-backup-api[612]: authentication failure; rhost=[<ip>]:48186 user=root@pam msg=authentication error - SUCCESS (0)
Aug 06 16:06:13 pbs proxmox-backup-api[612]: authentication failure; rhost=[<ip>]:48180 user=root@pam msg=authentication error - SUCCESS (0)
Aug 06 16:06:14 pbs unix_chkpwd[23238]: password check failed for user (root)
Aug 06 16:06:14 pbs proxmox-backup-api[612]: pam_unix(proxmox-backup-auth:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost=<ip>  user=root
Aug 06 16:06:16 pbs proxmox-backup-api[612]: authentication failure; rhost=[<ip>]:48198 user=root@pam msg=authentication error - SUCCESS (0)
Aug 06 16:06:19 pbs proxmox-backup-api[612]: POST /api2/json/access/ticket: 401 Unauthorized: [client [<ip>]:48198] permission check failed.
Aug 06 16:08:10 pbs unix_chkpwd[23244]: password check failed for user (root)
Aug 06 16:08:10 pbs proxmox-backup-api[612]: pam_unix(proxmox-backup-auth:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost=<ip>  user=root
Aug 06 16:08:12 pbs proxmox-backup-api[612]: authentication failure; rhost=[<ip>]:48198 user=root@pam msg=authentication error - SUCCESS (0)
Aug 06 16:08:15 pbs proxmox-backup-api[612]: POST /api2/json/access/ticket: 401 Unauthorized: [client [<ip>]:48198] permission check failed.

I keep getting logged out of the web UI.

I reset the password via SSH using `passwd` - as well as via `proxmox-backup-manager user update root@pam --password` - but the password check still fails.

Code:
root@pbs:~# proxmox-backup-manager version --verbose
proxmox-backup                           4.0.0            running kernel: 6.14.8-2-pve
proxmox-backup-server                    4.0.10-1         running version: 4.0.10    
proxmox-kernel-helper                    9.0.3                                       
proxmox-kernel-6.14.8-2-pve-signed       6.14.8-2                                    
proxmox-kernel-6.14                      6.14.8-2                                    
proxmox-kernel-6.14.8-1-bpo12-pve-signed 6.14.8-1~bpo12+1                            
ifupdown2                                3.3.0-1+pmx9                                
libjs-extjs                              7.0.0-5                                     
proxmox-backup-docs                      4.0.10-1                                    
proxmox-backup-client                    4.0.10-1                                    
proxmox-mail-forward                     1.0.2                                       
proxmox-mini-journalreader               1.6                                         
proxmox-offline-mirror-helper            0.7.0                                       
proxmox-widget-toolkit                   5.0.4                                       
pve-xtermjs                              5.5.0-2                                     
smartmontools                            7.4-pve1                                    
zfsutils-linux                           2.3.3-pve1
FYI That's the same issue I reported earlier in the thread.
 
  • Like
Reactions: CRCinAU
FYI That's the same issue I reported earlier in the thread.
What browser do you use? @CRCinAU – as while the error message is confusing due to the earlier result being encoded in there, there still is an error there, which comes from pam. Anything specific installed on the system, especially w.r.t. adapting PAM?

Can you create an account for the PBS realm and try that?

Code:
proxmox-backup-manager user create test-admin@pbs --password "super-secure"
proxmox-backup-manager acl update / Admin --auth-id test-admin@pbs

Then select the PBS realm in the login window and try loggin in with the credentials you used, i.e. in above axample it would be user test-admin and password super-secure
 
What browser do you use?

I'm using Edge on linux (Fedora 42).

Anything specific installed on the system, especially w.r.t. adapting PAM?
No customisation of PAM, and only minimal changes to the underlying OS.


Can you create an account for the PBS realm and try that?
I don't seem to see any errors for /api2/* in the journal when using a PBS realm user.

EDIT: After adding a 2FA WebAuthn device to the PBS realm user, I logged out, then tried to log back in again and hit this still:
 
Last edited:
Here you go, let me know if there is anything else!
Hey again, a couple of questions just to clarify:

- You can't log in at all or do you get logged out again immediately after you initially logged in?
- Do the log lines with authentication error - SUCCESS (0) still happen when you log in after the update? If so did you reboot the machine?
- Same question as for CRCinAU, can you log in with a user from a different realm? For example, the PBS realm?
 
Last edited:
EDIT: After adding a 2FA WebAuthn device to the PBS realm user, I logged out, then tried to log back in again and hit this still:
I think this is probably unrelated to the issue with pam here though. If I remember correctly, we have changed some dependencies there, but I am not too familiar with the TFA part of our authentication code.

Regarding the authentication error - lines, the still occur on PBS 4.0.11 or 4.0.10 for you? Even after a reboot?
 
Regarding the authentication error - lines, the still occur on PBS 4.0.11 or 4.0.10 for you? Even after a reboot?

It occurs in 4.0.10 - even after reboots etc. I can't see a 4.0.11 anywhere at the moment.

Also, I can login ok, but normally get logged out again quickly. The journal seems to show the auth failures, and the login session will be invalidated within a couple of page loads.
 
Last edited:
Hey again, a couple of questions just to clarify:

- You can't log in at all or do you get logged out again immediately after you initially logged in?
- Do the log lines with authentication error - SUCCESS (0) still happen when you log in after the update? If so did you reboot the machine?
- Same question as for CRCinAU, can you log in with a user from a different realm? For example, the PBS realm?
I created a testuser (Admin Role) in the 'Proxmox Backup authentication server' realm and everything seemed to work as expected, no errors no forced logout.

The moment I added WebAuthn to this testuser the same issue occurs (like with the root user in the 'Linux PAM standard authentication' realm and the user is forcefully logged out getting these messages in the error log:

Code:
Aug 06 10:45:53 <redacted> proxmox-backup-api[716]: authentication failure; rhost=[::ffff:<redacted>]:40198 user=testuser@pbs msg=invalid credentials
Aug 06 10:45:56 <redacted> proxmox-backup-api[716]: POST /api2/json/access/ticket: 401 Unauthorized: [client [::ffff:<redacted>]:40198] permission check failed.
Aug 06 10:46:07 <redacted> proxmox-backup-api[716]: authentication failure; rhost=[::ffff:<redacted>]:40198 user=testuser@pbs msg=invalid credentials
Aug 06 10:46:10 <redacted> proxmox-backup-api[716]: POST /api2/json/access/ticket: 401 Unauthorized: [client [::ffff:<redacted>]:40198] permission check failed.

The logout after a successful login happens within <10s or a few page loads.

Current running version (system was rebooted after latest update):

Code:
root@pbs ~ # uptime; proxmox-backup-manager version
 10:40:11 up 1 min,  1 user,  load average: 0.08, 0.03, 0.00
proxmox-backup-server 4.0.10-1 running version: 4.0.10
 
Last edited:
Also, I can login ok, but normally get logged out again quickly. The journal seems to show the auth failures, and the login session will be invalidated within a couple of page loads.
Can you post the logs of that? The log above look like you can't log in at all:
Aug 06 16:06:12 pbs proxmox-backup-api[612]: authentication failure; rhost=[<ip>]:48186 user=root@pam msg=authentication error - SUCCESS (0)
This for example indicates that pam could not authenticate you at all.
 
@Bent when you say WebAuthn, are you using physical tokens or (software) passkeys? could you provide as much detail about your setup as possible? thanks!
 
@Bent when you say WebAuthn, are you using physical tokens or (software) passkeys? could you provide as much detail about your setup as possible? thanks!
WebAuthn TFA via YubiKey (server certificate is trusted (Let's Encrypt) as required)
 
Last edited:
Is there a Backup Client 4.0 somewhere? I have proxmox-backup-client-3.4.2-2 as RPM installed, but using 4.0.10 Beta server
 
http://download.proxmox.com/debian/pbs-client trixie test

if you just want the client repository
 
  • Like
Reactions: rfox
Updated all node to VE 9 and also a working PBS 3.4.3 to 4.0.10 Beta - seem to work, but the datastore graphs not showing any info - had to recreate the datastore PBSBackup due to another issue - but if I look back a week - it shows the previous datastore which was destroyed but with the same name ?

Is this normal??

After recreating PBSBackup:
1754483672446.png

Showing week view - which shows previous PBSBackup before being destroyed and recreated:
1754483924861.png
 
Confirmed mounted fine, no errors..
Well so oddly it was working, and I repeated the setup process and now always get this error when trying to add a new datastore to check the step by step setup. I will say I got this error overnight on the bucket that initially was working with a small backup job running too, so I tried to create a completely new datastore and the error is the same on both. Very odd this is coming from a brand new bucket entirely

Code:
TASK ERROR: failed to get in-use marker from bucket: object is archived and inaccessible until restored

I can open a bug if you like. this is the same B2 account as before I just created a new bucket and new keys.

I have access to other S3 options to test as well to see if this is a B2 specific, but let me know how I can help
 
Last edited:
Well so oddly it was working, and I repeated the setup process and now always get this error

Code:
TASK ERROR: failed to get in-use marker from bucket: object is archived and inaccessible until restored

I can open a bug if you like. this is the same B2 account as before I just created a new bucket and new keys.

I have access to other S3 options to test as well to see if this is a B2 specific, but let me know how I can help
Do you have versioning enabled for this bucket? or did move the contents to a different storage class?

Also, note that it is not required to recreate the datastores while upgrading from the beta to the stable release, just to mention this for eveybody upgrading to the latest version.
 
Last edited:
Do you have versioning enabled for this bucket? or did move the contents to a different storage class?

Also, note that it is not required to recreate the datastores while upgrading from the beta to the stable release, just to mention this for eveybody upgrading to the latest version.
I do not I did not touch versioning, but I am not sure if B2 auto moves anything either I am looking into that, but the error on a brand new bucket as a test is very odd...

As best I can tell B2 has no other storage classes it's just the one with versioning options.

1754489598861.png
 
Last edited:
I do not I did not touch versioning, but I am not sure if B2 auto moves anything either I am looking into that, but the error on a brand new bucket as a test is very odd...

View attachment 88964
just double checked with the just released PBS stable. Datastore creation works for both cases here without issue, when deleting the datastore contents during datastore destruction as well as if checking the reuse datastore and overwrite in-use marker if reusing the previous datastore contents.

Settings for my backblaze bucket look the same as for you. Keep all versions and no object locking.

What version are you running at the moment? Please post the output of proxmox-backup-manager versions --verbose