PBS appears not to write to disk

[sublimebutbroken]

Hello friends.

New user of Proxmox and PBS here. I'm evaluating it for use in a tech company that is starting to need virtualization of a few services. Right now I have a basic working config on a local server:

PVE - running 4 x VMs
PBS - also running as a VM inside PVE. Datastore is configured to point at a section of thin storage within PVE, not a dedicated disk (yet).

My short term aim is to sync the backups on PBS with a cloud storage provider using Rclone. I am having trouble reading the backup files off the disk inside PBS. The backup appears to go fine from PVE-PBS. I can restore, and I can download all the files - everything appears intact, no errors or problems.

However - if I examine the files on the disk inside PBS the file size is way too small to be complete. a ~20GB backup of a disk appears as only 620kB. If this is sync'd with Rclone, then only 620kB is transferred.

I cannot figure this out. It's almost as if the OS isn't reporting the correct file size or something weird. Can anyone point me in the right direction to resolve this?

 

[Johannes S]

My short term aim is to sync the backups on PBS with a cloud storage provider using Rclone.

Bad idea: https://forum.proxmox.com/threads/datastore-synced-with-rclone-broken.154709/

You are propably better off with a cheap vserver running an offsite pbs or a pbs cloud provider like Tuxis.
The remote pbs would then pull the Backups from your local PBS. Another Option would be to create additional full Backups with PVEs native function aka vzdump and upload these dumps with rclone

However - if I examine the files on the disk inside PBS the file size is way too small to be complete. a ~20GB backup of a disk appears as only 620kB. If this is sync'd with Rclone, then only 620kB is transferred.

The datastore should have a hidden .chunks folder which contains most of the actual data.

How did you do the examination?

 

[Chris]

Hello friends.

New user of Proxmox and PBS here. I'm evaluating it for use in a tech company that is starting to need virtualization of a few services. Right now I have a basic working config on a local server:

PVE - running 4 x VMs
PBS - also running as a VM inside PVE. Datastore is configured to point at a section of thin storage within PVE, not a dedicated disk (yet).

My short term aim is to sync the backups on PBS with a cloud storage provider using Rclone. I am having trouble reading the backup files off the disk inside PBS. The backup appears to go fine from PVE-PBS. I can restore, and I can download all the files - everything appears intact, no errors or problems.

However - if I examine the files on the disk inside PBS the file size is way too small to be complete. a ~20GB backup of a disk appears as only 620kB. If this is sync'd with Rclone, then only 620kB is transferred.

I cannot figure this out. It's almost as if the OS isn't reporting the correct file size or something weird. Can anyone point me in the right direction to resolve this?

Hi,
this is because the content addressable datastore does not store the data where you expect it to be. Besides the other folders on the datastores location, you will find a .chunks folder, that's where your data is stored. The other folders only contain files with logical information how to access the data chunks. Please have a look at https://pbs.proxmox.com/docs/technical-overview.html

Also, as pointed out by @Johannes S , such syncs will most likely lead to missing chunks if not performed correctly. Using a sync job to a remote PBS instance is therefore your safest option, see https://pbs.proxmox.com/docs/managing-remotes.html#sync-jobs

 

[sublimebutbroken]

This is exactly the information I was looking for, thanks.

Understood that this is not the best option - I'm still in evaluation mode at the moment. If we stick with this longer term then a standalone offsite PBS will be the way forward.