Proxmox Backup Server 4.0 released!

Is it possible for us to update the package ourselves in the meantime for testing purposes? If so, which one would that be?
 
proxmox-backup-server and proxmox-network-interface-pinning
 
  • Like
Reactions: Franky779
Updated my Prox cluster to 9 version and PBS to 4 version.
Backup restoring speed increased x3 due to multithreading, I suppose.
Everything work nice and smooth.
GREAT THANKS FOR PROXMOX TEAM!
 
I upgraded to PBS 4 recently and now Gotify notification are not working any longer:
Could not test target: could not notify via endpoint(s): Gotify: io: Permission denied (os error 13) (500)

Using curl to send a Gotify message to my Gotify-Server works. So not a network issue. On PVE 9 the Gotify notifications are working after upgrade. Any idea?
 
Last edited:
SnartLessing said:
I upgraded to PBS 9 recently and now Gotify notification are not working any longer:
Could not test target: could not notify via endpoint(s): Gotify: io: Permission denied (os error 13) (500)

Using curl to send a Gotify message to my Gotify-Server works. So not a network issue. On PVE 9 the Gotify notifications are working after upgrade. Any idea?
Click to expand...

Just to double check, are you experiencing the problem under PBS 4 or PVE 9? (since you wrote PBS 9...)
Anything noteworthy about your setup? E.g. if it is PBS, did you install it straight from the ISO installer or was it installed it on top of Debian? Anything non-standard with regards to your disk partitions/mount-points? Do you maybe run PBS in a container or something?

The code throwing the error runs as the root user, so I'm wondering why it might throw a 'Permission denied' at all...
 
Lukas Wagner said:
Just to double check, are you experiencing the problem under PBS 4 or PVE 9? (since you wrote PBS 9...)
Anything noteworthy about your setup? E.g. if it is PBS, did you install it straight from the ISO installer or was it installed it on top of Debian? Anything non-standard with regards to your disk partitions/mount-points? Do you maybe run PBS in a container or something?

The code throwing the error runs as the root user, so I'm wondering why it might throw a 'Permission denied' at all...
Click to expand...
Sorry Lukas for my mistake: I upgraded PBS from 3 to 4 and with this Gotify notifications are not working any more. PBS 3 was installed via ISO on bare metal hardware.
Menwhile I also figured out that on PVE 9 (upgraded from 8 to 9) the Gotify Test notification is working, but the notifications from backup jobs are not working also:

Code: 
ERROR: could not notify via target `Gotify`: could not notify via endpoint(s): Gotify: http status: 500

Looks like Gotify notifications in PVE 9 and PBS 4 are not working properly.
Happy to provide more details if required. Have not found more log information on this so far.
 
SnartLessing said:
Looks like Gotify notifications in PVE 9 and PBS 4 are not working properly.
Click to expand...

Could you provide me with the outputs of
  • pveversion -v (for the PVE node)
  • proxmox-backup-manager versions --verbose (for the PBS node)
Also, which version of Gotify do you use? Do you see anything useful in Gotify's logs? There should be reqests to the '/message' endpoint whenever PBS/PVE tries to send a notification, but at this point I'm not sure if it ever goes that far. The PVE logs seem to indicate that there's something going on the Gotify side, while the PBS error appears like it does not even get to send the HTTP request - odd.
 
Lukas Wagner said:
Could you provide me with the outputs of
  • pveversion -v (for the PVE node)
  • proxmox-backup-manager versions --verbose (for the PBS node)
Also, which version of Gotify do you use? Do you see anything useful in Gotify's logs? There should be reqests to the '/message' endpoint whenever PBS/PVE tries to send a notification, but at this point I'm not sure if it ever goes that far. The PVE logs seem to indicate that there's something going on the Gotify side, while the PBS error appears like it does not even get to send the HTTP request - odd.
Click to expand...

Using Gotify 2.6.3. PBS 3 or PVE 8 working as expected. Only Gotify Test in PVE 9 reaching Gotify Server.
Here are the requested outputs:

Code: 
root@pve:~# pveversion -v
proxmox-ve: 9.0.0 (running kernel: 6.14.8-2-pve)
pve-manager: 9.0.5 (running version: 9.0.5/9c5600b249dbfd2f)
proxmox-kernel-helper: 9.0.3
proxmox-kernel-6.14.8-2-pve-signed: 6.14.8-2
proxmox-kernel-6.14: 6.14.8-2
proxmox-kernel-6.8.12-13-pve-signed: 6.8.12-13
proxmox-kernel-6.8: 6.8.12-13
ceph-fuse: 19.2.3-pve1
corosync: 3.1.9-pve2
criu: 4.1.1-1
frr-pythontools: 10.3.1-1+pve4
ifupdown2: 3.3.0-1+pmx9
intel-microcode: 3.20250512.1
ksm-control-daemon: 1.5-1
libjs-extjs: 7.0.0-5
libproxmox-acme-perl: 1.7.0
libproxmox-backup-qemu0: 2.0.1
libproxmox-rs-perl: 0.4.1
libpve-access-control: 9.0.3
libpve-apiclient-perl: 3.4.0
libpve-cluster-api-perl: 9.0.6
libpve-cluster-perl: 9.0.6
libpve-common-perl: 9.0.9
libpve-guest-common-perl: 6.0.2
libpve-http-server-perl: 6.0.4
libpve-network-perl: 1.1.6
libpve-rs-perl: 0.10.10
libpve-storage-perl: 9.0.13
libspice-server1: 0.15.2-1+b1
lvm2: 2.03.31-2+pmx1
lxc-pve: 6.0.4-2
lxcfs: 6.0.4-pve1
novnc-pve: 1.6.0-3
proxmox-backup-client: 4.0.14-1
proxmox-backup-file-restore: 4.0.14-1
proxmox-backup-restore-image: 1.0.0
proxmox-firewall: 1.1.1
proxmox-kernel-helper: 9.0.3
proxmox-mail-forward: 1.0.2
proxmox-mini-journalreader: 1.6
proxmox-offline-mirror-helper: 0.7.0
proxmox-widget-toolkit: 5.0.5
pve-cluster: 9.0.6
pve-container: 6.0.9
pve-docs: 9.0.8
pve-edk2-firmware: 4.2025.02-4
pve-esxi-import-tools: 1.0.1
pve-firewall: 6.0.3
pve-firmware: 3.16-3
pve-ha-manager: 5.0.4
pve-i18n: 3.5.2
pve-qemu-kvm: 10.0.2-4
pve-xtermjs: 5.5.0-2
qemu-server: 9.0.18
smartmontools: 7.4-pve1
spiceterm: 3.4.0
swtpm: 0.8.0+pve2
vncterm: 1.9.0
zfsutils-linux: 2.3.3-pve1

Code: 
root@pbs:~# proxmox-backup-manager versions --verbose
proxmox-backup                      4.0.0        running kernel: 6.14.8-2-pve
proxmox-backup-server               4.0.14-1     running version: 4.0.14     
proxmox-kernel-helper               9.0.3                                   
proxmox-kernel-6.14.8-2-pve-signed  6.14.8-2                                 
proxmox-kernel-6.14                 6.14.8-2                                 
proxmox-kernel-6.8.12-13-pve-signed 6.8.12-13                               
proxmox-kernel-6.8                  6.8.12-13                               
proxmox-kernel-6.8.12-11-pve-signed 6.8.12-11                               
ifupdown2                           3.3.0-1+pmx9                             
libjs-extjs                         7.0.0-5                                 
proxmox-backup-docs                 4.0.14-1                                 
proxmox-backup-client               4.0.14-1                                 
proxmox-mail-forward                1.0.2                                   
proxmox-mini-journalreader          1.6                                     
proxmox-offline-mirror-helper       0.7.0                                   
proxmox-widget-toolkit              5.0.5                                   
pve-xtermjs                         5.5.0-2                                 
smartmontools                       7.4-pve1                                 
zfsutils-linux                      2.3.3-pve1
 
Ernst T. said:
You are probably missing http:// before the server address.
Click to expand...
No, that would give them a different kind of error. Also then it also would not have worked with PVE8/PBS3.

SnartLessing said:
Using curl to send a Gotify message to my Gotify-Server works.
Click to expand...
Did you try curl from the PBS host as root?

Did you by any chance change anything with regards to the notification templates? [1].

I'm a bit confused about this "Permission denied" error here. I've traced the origin of this error to two possible locations, either it happens when trying to read the notification template file from /usr/share/proxmox-backup/templates/... (with possible override which is checked first at /etc/proxmox-backup/notification-templates) or when doing the HTTP POST (TCP connection).

Do you have firewall rules set up for this PBS host? According to man connect one can also get a Permission Denied error when the connection is blocked by a local firewall rule:

Code: 
EPERM  The  user  tried  to  connect to a broadcast address without having the socket broadcast flag enabled or the connection request failed because of a local firewall
       rule.

Check with 'iptables-save' or 'nft list ruleset' and see if you have any rules set up here. By default, PBS does not have any rules set up whatsoever.

[1] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_overriding_notification_templates
 
  • Like
Reactions: Johannes S
  • Like
Reactions: EllerholdAG
Hey there, I am not sure if someone already asked this, but I do have a little bit of performance issues using S3 endpoints. I use the latest updates of PBS4 and have an Synology C2 connected.

While performing the sync job to offload my backups with PVE 9 (latest updates), I have longer than usual upload times. The throughput is very low, but my connection is 300Mbit/s in upload. Usually it takes under an hour for my files to get uploaded, but PBS4 takes around 15 hours.

Is it something known or is it maybe normal? I have encrpyted backups and use C2 immutable.

Thank you guys in advance.
 
Hi,
DFlorin said:
While performing the sync job to offload my backups with PVE 9 (latest updates), I have longer than usual upload times. The throughput is very low, but my connection is 300Mbit/s in upload. Usually it takes under an hour for my files to get uploaded, but PBS4 takes around 15 hours.

Is it something known or is it maybe normal? I have encrpyted backups and use C2 immutable.
Click to expand...
hard to tell since it is not clear what you are comparing... Pre PBS4 there was no S3 backend for datastores, so when you state that usually the files are uploaded in under an hour, what are you referring to?

There is still room for improvements of course...

Note however, there are users which report good throughput, e.g. see https://forum.proxmox.com/threads/s3-buckets-constantly-failing-verification-jobs.169875/post-793894
 
Chris said:
Hi,

hard to tell since it is not clear what you are comparing... Pre PBS4 there was no S3 backend for datastores, so when you state that usually the files are uploaded in under an hour, what are you referring to?

There is still room for improvements of course...

Note however, there are users which report good throughput, e.g. see https://forum.proxmox.com/threads/s3-buckets-constantly-failing-verification-jobs.169875/post-793894
Click to expand...
Hi Chris,

thanks for getting back to me.

I compare throughput based on Veeam Backup with VMware ESXi to the same C2 backend and on my connection. The raw throughput is around 130GBytes/h in upload.
 
DFlorin said:
Hi Chris,

thanks for getting back to me.

I compare throughput based on Veeam Backup with VMware ESXi to the same C2 backend and on my connection. The raw throughput is around 130GBytes/h in upload.
Click to expand...
What disk are you using as backing device for the local cache?

Edit: You can use proxmox-backup-client benchmark --no-cache --repository <your-repository> on the PVE host to check performance when bypassing the local cache, and again without the flag set to compare.
 
Last edited:
You must log in or register to reply here.
Top Bottom