Best method for transferring files between two physical mounted drives within Proxmox?

Whoops4685

New Member
Oct 27, 2024
1
0
1
I am currently new and experimenting with the proxmox environment, and I am interested in understanding the optimal way to transfer files between two physical hdd media disks in my server. I have mounted both drives to separate /mnt/ points within proxmox itself. My purpose, is to increase the amount of storage fro 4TB to 8TB and transferring photos, tv shows and old dvd movies to a larger capacity drive. Then purchase another 8TB storage drive to act as a backup. The only things I am running up to now is a OMV, HAOS, and Plex.

At the moment I have logged in, connected at the physical server and I am using rsync to transfer but it seems quite slow, as in three hours or more to transfer a 350GB music collection within the same computer. Is that normal speed between two spinning platter drives? Or is there a better way that is obvious that I am missing?

I thought about mounting them into my OMV VM and share via SMB but it didn't seem to make a difference (as I'm always getting @30MB/s). I tried drag and drop in a sftp app like Forklift, from my laptop but would get an error.

My pc is an optiplex sff i7 10 gen with 16GB ram, so it shouldn't be bottlenecked, I would think. Interested in anyones thoughts of where I am going wrong. Thanks. Hopefully I have provided enough info.
 
Do "cp -a /mnt/hdd1/* /mnt/hdd1/.??* /mnt/hdd2/." (a=archive mode for timestamps etc).
That's even good with nfs disks when possible only otherwise for remote or broken copies (to not start from beginning) take rsync.
With df look for your /dev/sd<x> 4TB device which has the data to copy
and do "echo 16384 >/sys/block/sd<put_here_your_device_letter>/queue/read_ahead_kb".
 
You can use rsync with

rsync -av /mnt/src/ /mnt/target/

to delete files on target that are not on src add --delete, to use numeric ids use --numeric-ids,
use -X for extended attributes and -A for acls.

You can do a test run without writing anything with --dry-run
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!