External Storage -> VM -> PBS

[tokukarin]

Hello,

I would like to ask I my plan will work.

I would like to install PBS inside a VM attach a CIFS onto the VM create a Datastore on the mounted CIFS directory and then backup my host bare metal Proxmox Server onto the external CIFS.

As far as I googled the main reason not to install PBS inside a VM is:

So I am not concerned that my host dies, I am of course running RAID, but you know:"One does not simply have no backups".

And If I myself delete system32 (it happens folks)
I should be able reinstall Proxmox then install PBS inside a VM again attach the CIFS again and restore the Backup, right?
Worst case If the host dies I would boot up a second server with PBS and attach the CIFS to that one, but theoretical and practically, this should work right?

Thanks in advance

 

[Dunuin]

So I am not concerned that my host dies, I am of course running RAID, but you know:"One does not simply have no backups".

Stuff that might still kill your guests + all backups in case the SMB server is a second host:
- fire
- water damage
- thunderstrike
- theft
- ransomware
- user error
- failing RAM

Stuff that might still kill your guests + all backups in case the SMB server is on the same server as PVE:
- dying PSU
- short circuit
- failing disk controller
- power outage (in case backups are stored on the same disks/array as the guests)
- kernel crash (in case backups are stored on the same disks/array as the guests)

I should be able reinstall Proxmox then install PBS inside a VM again attach the CIFS again and restore the Backup, right?

Jep. But don't forget to backup your encryption key or all backups will be lost.

And keep in mind that PBS was developed for local SSDs. It will run on HDDs or SMB/NFS but performance is terrible. Good enough for a homelab where you just want to store some hundred GBs of backups but not great for production or TBs of data.

 

[tokukarin]

There's my answer

 

[tokukarin]

Stuff that might still kill your guests + all backups in case the SMB server is a second host:
- fire
- water damage
- thunderstrike
- theft
- ransomware
- user error
- failing RAM

Stuff that might still kill your guests + all backups in case the SMB server is on the same server as PVE:
- dying PSU
- short circuit
- failing disk controller
- power outage (in case backups are stored on the same disks/array as the guests)
- kernel crash (in case backups are stored on the same disks/array as the guests)


Jep. But don't forget to backup your encryption key or all backups will be lost.

And keep in mind that PBS was developed for local SSDs. It will run on HDDs or SMB/NFS but performance is terrible. Good enough for a homelab where you just want to store some hundred GBs of backups but not great for production or TBs of data.

Thank you for your time, as you can see in my post between yours and this one, it doesn't work :/

But my server is in a colocation so, fire, water, thunderstrike, theft, dying psu, short circuit, power outage are all very unlikely.

 

[Dunuin]

SMB share is mounted as UID 34 or with 777 rights so the "backup" user can read/write it?

 

[tokukarin]

SMB share is mounted as UID 34 or with 777 rights so the "backup" user can read/write it?

Damn, thank you.
I didn't think about that, and because PBS created the .chunks folder I thought that it had r/w access.

P.S. Can you tell me what the .chunks folder is for? I don't know how much folder there are within but I saw that PBS started iterating folders from 0000 until my windows explorer crashed.

thank you again <3

 

[Dunuin]

All your backups will be chopped into chunk files of max 4MB in size. Those chunk files will then be stored in the ".chunks" folder. So that are your backups. And it is normal that this folder contains ten-thousands of sub directories and millions over millions of chunk files. And each time you want to free up space, you will have to run a GC task and that task has to read+write the metadata of every single chunk file (= millions of IO). That is one of the points why local SSDs should be used and no HDDs with crappy IOPS performance or network protocols that add additional latency and cripple the IOPS performance.

 

[tokukarin]

All your backups will be chopped into chunk files of max 4MB in size. Those chunk files will then be stored in the ".chunks" folder. So that are your backups. And it is normal that this folder contains ten-thousands of sub directories and millions over millions of chunk files. And each time you want to free up space, you will have to run a GC task and that task has to read+write the metadata of every single chunk file (= millions of IO). That is one of the points why local SSDs should be used and no HDDs with crappy IOPS performance or network protocols that add additional latency and cripple the IOPS performance.

Mhhh I already thought that it will be like that, I was asking in hopes of getting a different answer....
But yes that explains the high IO, but I don't mind.
I'm gonna use snapshot backups, so the whole host and vm's won't be affected, it will just take long, and I ain't gonna use the HDD for anything else so it should be fine, right?

 

[Dunuin]

I'm gonna use snapshot backups, so the whole host and vm's won't be affected, it will just take long, and I ain't gonna use the HDD for anything else so it should be fine, right?

If that is no problem for you, then sure.

 

[tokukarin]

So, I am finished setting everything up, and:
@Dunuin

INFO: root.pxar: had to backup 5.316 GiB of 5.783 GiB (compressed 2.283 GiB) in 132.93s

It is indeed not fast, but I mean it's fine. I did some testing and during the whole process of backing up this mailserver, the mailserver only was unreachable for like 5 seconds during the whole process.

I honestly have no problem If it would take one hour, as long as the service is not down for longer than a few seconds.
My automatic backups run every night, I expect at max 20-30 minutes for everything to be backed up.

And again, I don't see an issue here, all services are backed up at the end, all services are no longer down than a few seconds, I can store as many backups as I want because it's a HHD. I pay like 3€ for 1TB, and the performance is good.


Danke dir nochmal <3

Update:
Full backup time (host + vms):
ps3.pxar: had to backup 61.18 GiB of 270.278 GiB (compressed 30.956 GiB) in 2127.60s (35 Minutes)

 

[tokukarin]

For everyone else stumbling across and trying to do the same, here's my setup:

Privileged Debian 11 Container with CIFS and Nesting enabled.
I mounted the CIFS inside the container as a service.
Inside PBS I added the location under Datastore. (Administraton/"Storage / Disks"/Directory is the wrong location to try and add it, you have to add it under Datastore and type in the local location)
Host and PBS are on the same internal network.
And then on the Host I added the PBS as a PBS Backup device.

 

[tokukarin]

Those backups work incrementally, this shit is crazy.
The backups will be much faster after the first one.

HAHA

INFO: Finished Backup of VM 105 (00:00:51)

One minute for the backup of my mailserver (incrementally)

I love proxmox

Edit:
@Dunuin
Die Backups sind jetzt tatsächlich viel schneller wie davor, da sie nun inkrementell funktionieren.
Danke, danke, danke <3

 

[Dunuin]

Problem are not the backups/restores. Its the maintaince tasks like GC and Re-Verify. Have some TBs of Backups and they might run for days.

 

[tokukarin]

Problem are not the backups/restores. Its the maintaince tasks like GC and Re-Verify. Have some TBs of Backups and they might run for days.

True, but I did not have the Re-Verify option with normal backups in proxmox anyway.
It's just a bonus to check If the data got corruption.

GC on the other hand, yes true. But I think It's fine If you run it on a "If needed" basis

 

[Dunuin]

True, but I did not have the Re-Verify option with normal backups in proxmox anyway.
It's just a bonus to check If the data got corruption.

Sure, thats optional. But on the other hand backups you can't trust aren't that useful. If you ever need to restore a guest, it would be really bad if that backup is then corrupted. Also keep in mind that all backups share the same chunk files. So with a corrupted chunk file you will not lose one backup, you will probably lose all backups of a VM.

GC on the other hand, yes true. But I think It's fine If you run it on a "If needed" basis

You will need it sooner or later, because without it PBS won't free up any space of pruned backups

 

[tokukarin]

Sure, thats optional. But on the other hand backups you can't trust aren't that useful. If you ever need to restore a guest, it would be really bad if that backup is then corrupted. Also keep in mind that all backups share the same chunk files. So with a corrupted chunk file you will not lose one backup, you will probably lose all backups of a VM.

Forgot that damn chunk system again. Too used to the normal Proxmox backups. :/ Suboptimal...

You will need it sooner or later, because without it PBS won't free up any space of pruned backups

NO I JUST ADD MORE SPACE

 

[tokukarin]

What would you suggest for Prune and Verify schedules?

I have verify now on daily with re-verify after 10 days.
Prune every month
And keep last 30

Is that okay? @Dunuin

 

[Dunuin]

I personally prune every day, GC every week, verify every 30 days and re-verify every 90 days.
And pruning keeps 12 daily, 8 weekly, 12 monthly, 3 annualy + unlimited protected manual backups.

Deduplication is great. Got 36596 GBs of guest backups compressed and deduplicated down to 928 GB.

 

[tokukarin]

I personally prune every day, GC every week, verify every 30 days and re-verify every 90 days.
And pruning keeps 12 daily, 8 weekly, 12 monthly, 3 annualy + unlimited protected manual backups.

Deduplication is great. Got 36596 GBs of guest backups compressed and deduplicated down to 928 GB.

I fell from the chair when I read 40TB, thought you got 40TB Server SSD's but then I read further, haha.

But makes sense, you have a lot of backups so you prune everyday but don't verify often.
I only started to do backups (on pbs) so I have high verify and low prune settings.

I guess when I have more backups I will have to adjust to pruning more often and verifying less, like you.