Hi!
First of all, I want to thank the Proxmox team as well as the various contributors for their work.
I am currently conducting some tests of a backup infrastructure using PBS with automatic client deployment on about ten NFS servers, only the exported partition is backed up.
I find the solution very interesting and I am trying to adapt it to a fairly specific workflow.
The backup server I am currently using have a volume of around 300TB (3*vdev.raidz2.8*18TB_HDD+1*spec_vdev.mirror.2*4TB_SAS; 1TB RAM; dual Xeon). In the future i am planning, depending on the solution performance and reliabilty to get 6 server with 3xmirors synced.
Where I’m seeking your help is in the areas of restoration capabilities and flexibility in the creation and management of backups, which fall into several points:
What i want ( for the moment )( every project has its own group of users, right are managed with LDAP groups) :
Server configuration:
- PBS is integrated with LDAP
Backup configuration:
- Backups are triggered manually, wrapped in a bash script to handle errors; they will soon be triggered via systemd timers, and the backups will be daily.
- The --change-detection-mode option is set to metadata to allow faster backups on volumes with few differences between each state.
- Encryption will soon be activated.
Restoration configuration:
- Restorations are done by looking onto the NFS mount points from the client workstation, not directly on the relevant servers.
- Currently, restoration can be done directly through the web interface, and it’s possible to browse through the backups despite the change-detection-mode being set to metadata, which is handy.
I am looking for advice on how to meet my needs.
Thanks for reading !
First of all, I want to thank the Proxmox team as well as the various contributors for their work.
I am currently conducting some tests of a backup infrastructure using PBS with automatic client deployment on about ten NFS servers, only the exported partition is backed up.
I find the solution very interesting and I am trying to adapt it to a fairly specific workflow.
The backup server I am currently using have a volume of around 300TB (3*vdev.raidz2.8*18TB_HDD+1*spec_vdev.mirror.2*4TB_SAS; 1TB RAM; dual Xeon). In the future i am planning, depending on the solution performance and reliabilty to get 6 server with 3xmirors synced.
Where I’m seeking your help is in the areas of restoration capabilities and flexibility in the creation and management of backups, which fall into several points:
What i want ( for the moment )( every project has its own group of users, right are managed with LDAP groups) :
- daily backup of my different NFS server ( from the NFS host with pbs client ) one backup per project in my $NFS_SERVER:/export/$project; i can have a same project on multiple server with different data
- each project backup on a dedicated NS or repository
- each LDAP users on a project group could restore a last version of a file or if not exists of a folder
Server configuration:
- PBS is integrated with LDAP
?- Is it possible to manage permissions on repositories and/or namespaces directly via LDAP groups, rather than with users ? there is as many backups to do as there are projects per server, and the same project may exist on different servers with different data.
-> I can write a script to directly insert users belonging to an LDAP group in Proxmox and assign them the necessary permissions, but it would be very convenient if this were directly integrated into PBS.
-> I can manage this outside of PBS by checking the LDAP group permissions for each user locally and then triggering the restoration with a user per project user on the server, but I find that less clean.
- Creation of multiple datastores and namespaces?- Should I favor a single datastore with namespaces for my different projects, or create a datastore per project? I am mainly thinking in terms of performances, especially for chunk evaluation, but also in term of right management granularity.
Backup configuration:
- Backups are triggered manually, wrapped in a bash script to handle errors; they will soon be triggered via systemd timers, and the backups will be daily.
- The --change-detection-mode option is set to metadata to allow faster backups on volumes with few differences between each state.
- Encryption will soon be activated.
Restoration configuration:
- Restorations are done by looking onto the NFS mount points from the client workstation, not directly on the relevant servers.
- Currently, restoration can be done directly through the web interface, and it’s possible to browse through the backups despite the change-detection-mode being set to metadata, which is handy.
?- Is it possible to export outside of my download path in Proxmox Backup Server? Perhaps a "save as" feature?
-Otherwise, I would prefer to provide my users with a restoration tool like "pbs-restore [-f $myFile] [-d $myFolder]", but this raises several issues:- I haven't found a simple way to have my index in metadata. I have two solutions:
- Solution 1 (slow and requires effort to be properly maintained and usable):
- Mount my mpxar archive
Code:
proxmox-backup-client mount host/my-server/2024-09-23T08:17:30Z my_path.mpxar /mnt/TEST_MOUNT- getting my index with time
Code:
find /mnt/TEST_MOUNT -type f -exec stat --format="%y %n" {} ; > mount_index_time.log- Then, we grep for $myFile or myFolder in the index to provide the correct options for proxmox-backup-client restore when launched on a client workstation .
- Solution 2 (a bit shaky as it only takes modifications into account):
- Enable
to get the file paths saved.
Code:
PBS_LOG=debug- Then grep for $myFile or myFolder in the index to provide the correct options for proxmox-backup-client restore.
?- Is it possible to export logs differently at runtime?
?- Does enabling these logs impact backup performance?
?- Are there more elegant/effective solutions?
?- Should i already take my pbs licence to get some help I am looking for advice on how to meet my needs.
Thanks for reading !
Last edited:
, which would be the ideal solution for me ( compared to the web interface restoration )