[SOLVED] setup a new ssd and move mega data to that ssd

[Ting]

Hi,

I had a pbs running with 4 sas disk drives, and GC takes 7 days to complete.

After some reading, I think to setup a new ssd storge and use it for all mega data, that would speed up GC process. move backup mega data to ssd, so GC will be faster. Anybody could suggest how to move mega data to the new ssd? thanks in advance.

 

[mariol]

In any case, a very good decision to rely on faster storage ! Since a lot of I/O naturally takes place on the drives, please use enterprise SSDs/NVMEs only here.

how to move mega data to the new ssd?

It depends on which file system you are using. Basically you can use “mv <backupdata> <new place backupdata>”. As this copies at file level, it can take a very long time.
However, if you use ZFS [1], for example, you can snapshot the whole thing and transfer it using “zfs send / receive”. At this point I must also note that ZFS requires a little more performance than other file systems.

Here an example: First stop the services on your PBS

systemctl stop proxmox-backup-proxy.service proxmox-backup.service

Method1: Move your data e.g. ext4 -> if you use SSH, use screen or similar to keep the session active

mv <backupdata> <new place backupdata>

Method2: Move your data ZFS -> if you use SSH, use screen or similar to keep the session active

zfs snapshot <pool>/<dataset>@snapshotname
zfs send -v <pool>/<dataset>@snapshotname | zfs receive <newpool>/newdataset

Please have also a look at [2], [3] and [4].

After everything is ready, adjust the paths under /etc/proxmox-backup/* and restart the PBS services.

systemctl start proxmox-backup-proxy.service proxmox-backup.service

[1] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_zfs
[2] https://pve.proxmox.com/wiki/ZFS:_Tips_and_Tricks
[3] https://pve.proxmox.com/wiki/ZFS_on_Linux
[4] https://deepdoc.at/dokuwiki/doku.ph...:linux_zfs#zfs_snapshots_und_deren_verwaltung

 

[Ting]

Woo, very detailed. Thank you.

Just want to clarify one thing, I still want to keep my sas disk drives for all actual data backup, and these ssd only for necessary indexing data (I called mega data), so I can speed up GC process. I have about 60Gb stuff to backup, I was planning to use 1tb ssd.

is that sound good?

 

[mariol]

ssd only for necessary indexing data

What exactly do you mean by that?

 

[Ting]

Hi,

I want to speed up GC process by adding some ssd to backup server, I read some of threads here, my understanding is, I could use ssd for some files that GC process would need to access.

 

[mariol]

If you use ZFS, you can use a special device to speed up GC [1]. But this only applies to newly written backup data. It is best to move the entire backup to SSDs [2].


[1] https://pbs.proxmox.com/docs/sysadmin.html#zfs-special-device
[2] https://pbs.proxmox.com/docs/installation.html#recommended-server-system-requirements

 

[Johannes S]

To get the speed gains for the old data you could rewrite it with zfs send/receive after adding the special device

 

[Ting]

Great stuff, thank you very much.

 

[Ting]

Also, after I add a special device, exactly how to use zfs send/receive to gain the speed of my old backup? originally I thought to delete the old backup, and start a new one. Now I am wondering whether there is a quicker way to handle all my old backup at once.