[SOLVED] Connecting storage from nodes over wan (no cluster,no HA)

[otoman]

Hello,

I've just set up my second Proxmox VE node at my home in my hometown. Since there are some files on Node 1 I'd like to transfer to Node 2, I want to somehow connect their fileservers (SMB) so that I don't have to use a VPN and a client computer/VM since the network is quite slow on Node 2. I've considered clustering, but that doesn't seem plausible given the differences in network speeds and a large physical distance, and sharing Samba via wan is, as I've been informed, quite dangerous. Still, I'd like to be able to input commands (or hopefully use a UI) to transfer files between the two nodes. I don't want HA or any sort of automated mirroring, I just need an easy way to access both at once. Both nodes have DDNS set up and I can open any necessary ports on their routers. Would perhaps an FTP connection work or something? I'd like something secure and reliable.

Thanks in advance!

 

[bbgeek17]

If you need to transfer files from time to time, and you are ok with them being potentially out of sync if you are making changes, then either FTP or SFTP will do.

If you need something more advanced, then the requirements need to be better defined. Based on the information you provided the HA/clusters is out of the question, not that sharing files is the purpose of PVE HA. You could use NFS or SMB, encapsulate it in VPN or TLS tunnel, Neither is PVE specific and many articles online are available for your research.
If you want something "secure" then SFTP will be preferred over FTP.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[otoman]

Thanks for the reply. I tried searching for a solution but I must've not specified my issue correctly as I was unable to find anything outside of clustering. I thought about setting up an SFTP server using Turnkey Fileserver on each node, mounting relevant folders, setting up a single user that would be used by the nodes and then opening the ports on each router. That seems like the easiest solution given my knowledge, though I'm not sure if that would actually work as I expect it to.

 

[bbgeek17]

unless you disable sftp specifically, its already "running" on your PVE host. You dont need any extra VMs/servers:

root@proxmox7-nvme1:~# sftp localhost
Connected to localhost.
sftp>

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[alexskysilk]

Since there are some files on Node 1 I'd like to transfer to Node 2, I want to somehow connect their fileservers (SMB) so that I don't have to use a VPN and a client computer/VM since the network is quite slow on Node 2.

the answer is rsync. you will still need to facilitate connectivity- if not vpn, at least port redirection with a ssl key. I was gonna describe the whole process but then I remembered I have google.

https://fedingo.com/how-to-use-rsync-with-ssh-key/

 

[otoman]

Thanks for the replies. The notification for the last one must've gotten lost in the invox. Anyway, with regards to SFTP, I was thinking it more secure to have containers running fileservers which have only specific folders mounted, rather than accessing directly host-to-host, and, I can always destroy and rebuild them if something messes up.

Why would rsync be preferable to SFTP for instance?

Best regards,

otoman

 

[alexskysilk]

Why would rsync be preferable to SFTP for instance?

rsync is the means. sftp (well, ssh at any rate) is the transport.

edit thats really an oversimplification. rsync is a tool designed for this type of file transfer- with inherent job controls, checksum, and resumability.

 

[pvps1]

you dont define the usecase very well but maybe an asynchronous solution would fit. i'd go for nextcloud and tailscale (why expose pve unnecessarily)

 

[otoman]

rsync is the means. sftp (well, ssh at any rate) is the transport.

edit thats really an oversimplification. rsync is a tool designed for this type of file transfer- with inherent job controls, checksum, and resumability.

Thanks for the clarification. I'll mark the thread as solved since I got several useful suggestions.

you dont define the usecase very well but maybe an asynchronous solution would fit. i'd go for nextcloud and tailscale (why expose pve unnecessarily)

So the 2nd node is more of a media server for my family back home. The 1st node has some media I brought from back home while building the 1st node so now I want to transfer it while avoiding OpenVPN's network overhead as the network for the 2nd node is about 50/20 at best. I don't need synchronicity as I just want a method of copying files between nodes. I agree about the host exposure, hence the VMs/Cts acting as fileservers with particular mountpoints and heavy restrictions.