Disable dirty-bitmap

debian10

Member
Jul 30, 2019
21
0
6
37
Hello
It seems to be very slow when doing dirty-map backups because it does the comparison.
I know that after rebooting the server, a new dirty-map is needed, but how can this feature be disabled forever?
Thanks
 

spirit

Famous Member
Apr 2, 2010
5,768
673
133
www.odiso.com
The dirty bitmap, it's just an in memory map of changes (0/1) of each disk bloc changes after the last backup. It doesn't compare nothing. Proxmox backup will simply backup only flagged blocks in the dirty bitmap.

After restart the qemu process, the dirty bitmap is lost. At next backup, proxmox will read all block on disk, and compare them to the pbs, to only send incremental.

You can't disable bitmap feature, or you couldn't do any incremental backup, only full backup.
 

leesteken

Famous Member
May 31, 2020
1,938
409
88
It seems to be very slow when doing dirty-map backups because it does the comparison.
It saves the backup from having to read unchanged disk blocks, compressing them and checking with the server if they are there, by only looking at some bits of memory.
Please try the following: make a backup of a running VM to PBS and then make another backup of that (still running) VM to PBS (this will use the bitmap so it does not have to read most of the virtual disk). Then Fully shutdown the VM, restart the VM (to clear the bitmap) and make another backup of that VM to PBS. If the third backup is not slower than the second, please provide timing information and logging.
You can't disable bitmap feature, or you couldn't do any incremental backup, only full backup.
By the way PBS works, all backups are full but only information not already on the PBS is transferred (as the same data is never stored twice). This has nothing to do with the bitmap. The dirty bitmap is only an (optional) optimization that prevents reading disk blocks (and the follow up steps of compressing and checking) that are guaranteed to already be on the PBS.
 
Please try the following: make a backup of a running VM to PBS and then make another backup of that (still running) VM to PBS (this will use the bitmap so it does not have to read most of the virtual disk). Then Fully shutdown the VM, restart the VM (to clear the bitmap) and make another backup of that VM to PBS. If the third backup is not slower than the second, please provide timing information and logging.
My experience:

Have tested this in my actual scenario and the dirty-bitmap, appears to be slow BUT is very quickly compared with full backup.
We have a Zimbra Mailserver VM with 1.4 TB of data, (PBS located in separate server, 1gbps dedicated network and no-specialized hardware)

-- Full backup needs aprox 20hs
-- PBS + Dirty Bitmap needs aprox 5hs
-- PBS + Live incremental backups ... aprox 5 minutes.

We are happy with these times.
Regards!
 

leesteken

Famous Member
May 31, 2020
1,938
409
88
My experience:

Have tested this in my actual scenario and the dirty-bitmap, appears to be slow BUT is very quickly compared with full backup.
We have a Zimbra Mailserver VM with 1.4 TB of data, (PBS located in separate server, 1gbps dedicated network and no-specialized hardware)

-- Full backup needs aprox 20hs
-- PBS + Dirty Bitmap needs aprox 5hs
-- PBS + Live incremental backups ... aprox 5 minutes.

We are happy with these times.
Regards!
Can you please explain to me what a "Live incremental backup" is? I don't understand the difference with "Dirty Bitmap".
 
Can you please explain to me what a "Live incremental backup" is? I don't understand the difference with "Dirty Bitmap".
Hi!
Not sure if the correct name is "Live incremental backup"...

But, in PBS FAQ you can read "With Proxmox Backup Server, backups are sent incrementally to the server, and data is then deduplicated on the server. This minimizes both the storage consumed and the impact on the network"
https://pbs.proxmox.com/docs/faq.html

You can backup the VM in online state, in this case , backup are sent incrementally to PBS.
 
  • Like
Reactions: leesteken

debian10

Member
Jul 30, 2019
21
0
6
37
In short, what solutions do you suggest to increase the backup speed?
PBS backup speed is really slow.
Servers use NVMe but Storage Server have a Enterprise HDD Disk.
 

debian10

Member
Jul 30, 2019
21
0
6
37
Well, it is not really economical to use SSD and NVMe disks for backing up this amount of data!
Do not know a way to use the Single HDD with NVME cache?
 

Dunuin

Famous Member
Jun 30, 2020
7,206
1,715
149
Germany
You can also increase the recordsize of the ZFS pool to 1M (or 4M if you enable the big records feature for your pool) so there is less metadata per chunk and therefore less IO and enable the relatime option for your pool. But using a "special device" will primarily help with prune and GC maintaince tasks. For backups, restores or verify tasks it won't help that much (only seen performance increase up to factor 3 for really small reads/writes...not sure how much it would help with bigger 1-4MB chunks).

PBS stores everything as small chunk files with a max 4MB size. Lets say you got an average of 2MB per chunk. Then a 2 TB HDD would need to store 1 million chunk files spread across the whole disk. HDDs are neigher good at handling that much small files nor at accessing files that are placed randomly across the disk. So if you want a really fast PBS datastore there is no way around a full SSD storage with magnitudes of better IOPS and random access performance.
 
Last edited:
  • Like
Reactions: leesteken

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!