Does changing "CT Change Detection Mode" from default to data speed up GC?
- Thread starter windowsdesxtop
- Start date
I'm asking because backups on zfs hdd are so slow (compared to ext4). zfs datastore gets like 1iops(while disk is like 100iops) while ext is closer to 1:1
I don't think this affects GC times. If you want faster backups try the metadata mode. If you want faster GCs we need to look at your pool and your settings.
Let's start with
Let's start with
Bash:
lsblk -o+FSTYPE,LABEL,MODEL
zpool status -vtLP
zfs get all pool/datasetofpbsShare the data and how you retrieved it.
@Impact thank you.
I have two drastically different examples but I'm going with easiest one:
Raid1 x2 disks used only for pbs datastore (not for main pool). Attached to pbs lxc as mountpoint
"Share the data and how you retrieved it."
Example I'm running GC job notice how slow first part
Inside datastore graph below we can see like ~1IOPS
However in iostat -mx 1
Disk is doing about 100iops
I have two drastically different examples but I'm going with easiest one:
Raid1 x2 disks used only for pbs datastore (not for main pool). Attached to pbs lxc as mountpoint
"Share the data and how you retrieved it."
Example I'm running GC job notice how slow first part
Inside datastore graph below we can see like ~1IOPS
However in iostat -mx 1
Disk is doing about 100iops
Last edited:
Is there a specific reason you disabled compression? AFAIK it will only help due to metadata compression, not hurt. Especially with HDDs. I'd try disabling
Is the HDD pool used for anything else? Maybe look at it more closely: https://gist.github.com/Impact123/3dbd7e0ddaf47c5539708a9cbcaab9e3#io-debugging
I'd also play around with the datastore's tuning options. GC Cache for example.
atime and enabling compression again. Ideally you set it at the pool root and let the CT's disk inherit it. Did you modify the PBS tuning options? What's zarcsummary -s arc on the node and cat /etc/proxmox-backup/datastore.cfg in the CT look like?Is the HDD pool used for anything else? Maybe look at it more closely: https://gist.github.com/Impact123/3dbd7e0ddaf47c5539708a9cbcaab9e3#io-debugging
I'd also play around with the datastore's tuning options. GC Cache for example.
Last edited:
@Impact thank you
> Is there a specific reason you disabled compression?
No reason at all. I will try enabling and rewriting data.
> I'd try disabling atime
Isn't that dangerous?
> Did you modify the PBS tuning options?
> What's zarcsummary -s arc on the node
P.S.
Here we can see that ZFS is reclaiming that cache too early. IMO it could keep it for longer. and maybe next day GC would have that data cached. as looks like second run is slightly faster (cached)
datastore: d10
comment
gc-schedule 00:00
notification-mode notification-system
path /mnt/d10
tuning gc-atime-safety-check=1,gc-cache-capacity=8048576
> Is the HDD pool used for anything else?
No. Only for PBS
> Is there a specific reason you disabled compression?
No reason at all. I will try enabling and rewriting data.
> I'd try disabling atime
Isn't that dangerous?
> Did you modify the PBS tuning options?
> What's zarcsummary -s arc on the node
P.S.
Here we can see that ZFS is reclaiming that cache too early. IMO it could keep it for longer. and maybe next day GC would have that data cached. as looks like second run is slightly faster (cached)
datastore: d10
comment
gc-schedule 00:00
notification-mode notification-system
path /mnt/d10
tuning gc-atime-safety-check=1,gc-cache-capacity=8048576
> Is the HDD pool used for anything else?
No. Only for PBS
You probably don't need to.
Unlikely. It disables implicit
atime but tools can still explicitly set it on files. PBS should do this and it also has safety checks.You could try to set a higher minimum ARC size and also play around with metadata balance and other related properties like it. Try adding SWAP/ZRAM/ZSWAP and push "unused" memory to SWAP so that you have more for cache. That might backfire if not configured "properly" though. In my experience PBS itself doesn't need much memory. Do you know what causes the memory to increase and ARC to drop?
I'd take a look at the various
zpool iostat commands from my link. Maybe some inefficiency sticks out.
Last edited:
> You could try to set a higher minimum ARC size
Don't want to speak too early but looks like this helped so much
..... Unbelievable
> Do you know what causes the memory to increase and ARC to drop?
Sounds like separate ZFS bug/tuning issue. Let's close this ticket.
Don't want to speak too early but looks like this helped so much
..... Unbelievable> Do you know what causes the memory to increase and ARC to drop?
Sounds like separate ZFS bug/tuning issue. Let's close this ticket.