New Proxmox as a replacement for ESXi

[Johannes S]

Im starting to see how powerful it is. Im really liking it. I might have missed it but is there a guide to mirror the VMs to say Synology or offsite storage like backblaze?

S3 is not supported at the moment, allthough it might work with a third-party-hack. I wouldn't do this tbh.
I described the options in an earlier post:

J

Post in thread 'Backup PBS to cloud provider with incremental chunks'

Nov 25, 2024

Hello,

this most likely won't work or in other words: You can do this, but then your backups will get broken which explains why nobody uses this:
https://forum.proxmox.com/threads/datastore-synced-with-rclone-broken.154709/
https://forum.proxmox.com/threads/pbs-appears-not-to-write-to-disk.157751/

You have following options:

• Using a PBS cloud provider like this one: https://www.tuxis.nl/de/proxmox-backup-server/ Tuxis don't sell to private customers outside the Netherlands or in general to customers outside the EU but they might have resellers (I know of at...

• Johannes S

• 

Basically the best option is to use a pbs as a service provider ( like cloud-pbs, tuxis.nl or inett ) or rent a vserver to run pbs.
Using Rclone for sycing to s3 targets is known to break datastores.

 

[LerryV2]

if I can just push it to Synology from PBS then I can move it to backblaze after that. Ill check those links out and see if I can get that working, thanks.

I was able to connect to Synology via a NFS share and a scheduled backup via PVE.

 

[Johannes S]

if I can just push it to Synology from PBS then I can move it to backblaze after that. Ill check those links out and see if I can get that working, thanks.

if I can just push it to Synology from PBS then I can move it to backblaze after that. Ill check those links out and see if I can get that working, thanks.

I was able to connect to Synology via a NFS share and a scheduled backup via PVE.

This is possible yes, but not recommended either. At least it doesn't break data. Network shares are not known for great performance though with larger data sets. One guy did some benchmarking on it:
https://forum.proxmox.com/threads/datastore-performance-tester-for-pbs.148694/

Please note the discussion on the validity of his approach (part of Proxmox developers had some critical points in this thread: https://forum.proxmox.com/threads/developer-question-about-chunk-folder.148167/ ), but imho the basic picture is clear: Network shares are not a good idea for saving large amount of data with PBS, even on a local network. Over a WAN (like some people do) it's a no for enterprise usage.

 

[LerryV2]

This is possible yes, but not recommended either. At least it doesn't break data. Network shares are not known for great performance though with larger data sets. One guy did some benchmarking on it:
https://forum.proxmox.com/threads/datastore-performance-tester-for-pbs.148694/

Please note the discussion on the validity of his approach (part of Proxmox developers had some critical points), but imho the basic picture is clear: Network shares are not a good idea for saving large amount of data with PBS, even on a local network. Over a WAN (like some people do) it's a no for enterprise usage.

Even if your just using it to backup an VM? Seems like if it takes a snapshot and it backs it up, as long as Im not taking a snap faster then it takes to transfer over it shouldnt really matter how long it takes as long as it gets there in one piece. Ill read what the tests say, Im curious.

 

[Johannes S]

Even if your just using it to backup an VM? Seems like if it takes a snapshot and it backs it up, as long as Im not taking a snap faster then it takes to transfer over it shouldnt really matter how long it takes as long as it gets there in one piece. Ill read what the tests say, Im curious.

With only one VM it's most likely not a problem the trouble starts with large data sets since PBS works by splitting it's data in a lot of small files (called chunks) for deduplication, so after an initial backup of the chunks contents only new or changed data is uploaded:
https://pbs.proxmox.com/docs/technical-overview.html

Just you get an idea this is from a log of my latest garbage collection job:

2025-01-11T02:00:25+01:00: Original data usage: 19.087 TiB
2025-01-11T02:00:25+01:00: On-Disk usage: 199.521 GiB (1.02%)
2025-01-11T02:00:25+01:00: On-Disk chunks: 157025
2025-01-11T02:00:25+01:00: Deduplication factor: 97.96
2025-01-11T02:00:25+01:00: Average chunk size: 1.301 MiB

I have around 390 snapshots from the last three months after setting up the PBS. Together they amount to around 19TB if restored. Thanks to deduplication they are only around 200 GB on disk and compromised of 157025 chunks. Each garbage collection will read and alter the metadata of each chunk to decide which chunks can be removed because no snapshot reference it anymore. Now my environment isn't even big (homelab), but I would expect that even more data and thus chunks would be present in a corporate or enterprise environment (way more lxcs and vms, thus more snapshots over a longer time , thus more chunks to work with).

PBS is phantastic but it's really not made for network file systems like CIFS or NFS. The mentioned benchmark tool is quite enlightening since it's developer used it for benchmarks on the local machine: He set up several storages in different filesystems and NFS/CIFS shares, all on the same machine, so his tool works with localhost. With other words the network performance wouldn't be an issue. The only differences would be due to the nature of the used way to access the storage.

 

[LerryV2]

Then what is a good way to push the backups off the server onto backblaze? We have 20 or so VMs.

 

[Johannes S]

Then what is a good way to push the backups off the server onto backblaze? We have 20 or so VMs.

At the moment there isn't really a good option sadly for this specific goal. S3 is on the Roadmap for some years but afik there is no work on it right now. Your best bet would be to run an offsite PBS on a virtual or dedicated server or going with a PBS hosting Provider like tuxis.nl or Inett. Then you could use PBS integrated sync functions for syncing the backups between the PBS instances. Maybe office@proxmox.com knows of such a company in your country.

Another Option might be to use zfs send/receive to replicate the datastore to an offsite location e.g. a TrueNAS/FreeBSD server in a data center . Obviouvsly this would need ZFS on the datastore and the remote server. rsync.net provides zfs as a service: https://www.rsync.net/products/zfsintro.html