[SOLVED] Advice on PBS datastore setup and what impact PBS has on running VM's

[greavette]

Hello,

I've setup one datastore for my PBS and testing is so far going well. After seeding my subsequent backups are running very fast and storage used is reduced a lot.

I have various types of VM's in my office:

Database servers that change often due to data being input
Destkops that are used daily with some change to files and folders.
File Servers that have many changes made daily to them.
Workstations that are only used to input data into our database and have almost no changes made directly to the OS

So some of my VM's change daily and some do not. I'm considering creating different datastores with different prune options. My database servers and file servers change a lot so I'd like to be able to keep versions of these throughout the day perhaps. Will running PBS during the day when data is being input into a database slow down the database during the backup?

Any advice others can provide on how they setup their datastore for various types of VM's would be greatly appreciated.

Thank you.

 

[Stefan_R]

So some of my VM's change daily and some do not. I'm considering creating different datastores with different prune options. My database servers and file servers change a lot so I'd like to be able to keep versions of these throughout the day perhaps.

I wouldn't really recommend setting up multiple datastores for this purpose, although it is an option of course. The issue is that you lose deduplication between VMs in different stores.

If your VM does not change often however, it also doesn't matter how often you back it up. If nothing changes, doing a backup will not use more storage. So I'd just back everything up as often as you'd like and use one pruning schedule. Doesn't hurt to have more backups lying around, especially if nothing changed between them (as then they don't use more space).

Will running PBS during the day when data is being input into a database slow down the database during the backup?

In a perfect world, no, it will not. There is a bit of overhead of course, and although QEMU tries to hide it with some clever caching, there's only so much it can do. The faster your PBS storage and network however, the less your VM will be impacted. I'd recommend just testing for yourself if it works in your scenario. If you're doing dirty-bitmap incrementals the time in which a slowdown can occur is shorted even further of course.

 

[oversite]

Hello,

I've setup one datastore for my PBS and testing is so far going well. After seeding my subsequent backups are running very fast and storage used is reduced a lot.

I have various types of VM's in my office:

Database servers that change often due to data being input
Destkops that are used daily with some change to files and folders.
File Servers that have many changes made daily to them.
Workstations that are only used to input data into our database and have almost no changes made directly to the OS

So some of my VM's change daily and some do not. I'm considering creating different datastores with different prune options. My database servers and file servers change a lot so I'd like to be able to keep versions of these throughout the day perhaps. Will running PBS during the day when data is being input into a database slow down the database during the backup?

Any advice others can provide on how they setup their datastore for various types of VM's would be greatly appreciated.

Thank you.

I use one datastore for all VMs regardless of type and changes.
If you have some VMs you want to prune more, you can use the backup-client to do so. I don't do specific pruning but you could use it if you need to prune some specific VMs and keep the default prune for the datastore:
proxmox-backup-client prune vm/350 --dry-run --keep-daily 1 --keep-weekly 2 --keep-monthly 2 --repository backupserver1

 

[greavette]

I wouldn't really recommend setting up multiple datastores for this purpose, although it is an option of course. The issue is that you lose deduplication between VMs in different stores.

If your VM does not change often however, it also doesn't matter how often you back it up. If nothing changes, doing a backup will not use more storage. So I'd just back everything up as often as you'd like and use one pruning schedule. Doesn't hurt to have more backups lying around, especially if nothing changed between them (as then they don't use more space).


In a perfect world, no, it will not. There is a bit of overhead of course, and although QEMU tries to hide it with some clever caching, there's only so much it can do. The faster your PBS storage and network however, the less your VM will be impacted. I'd recommend just testing for yourself if it works in your scenario. If you're doing dirty-bitmap incrementals the time in which a slowdown can occur is shorted even further of course.

Hello @Stefan_R and @oversite ,

Very much appreciate your replies. I agree with both of your assessments of PBS and datastores. I hadn't thought about the size of my "static" vm's backups not increasing due to minimal changes and dedupe. And you are correct in that one datastore could see efficiencies of space savings between VM backups. I will stick to one datastore and as oversite stated I can reduce the number of backups saved in my backup jobs.

Cheers!