About ceph over vpn performance question.

A

akong

Well-Known Member
Oct 15, 2018
87
1
48
47
Proxmox ceph node(A) <----> firewall <---------------internet--------------> proxmox ve client(B) (use vpn connect to ceph node and mount rbd)
My question is (A) create vm and storage set to ceph rbd.The VM very fast.
But (B) create vm and set to remote storage rbd is very slow.I has see iops almost not not working.How to test it?
 
I doubt ceph over VPN will work well, ceph needs a good network connection, and routing is not what you want for that.
 
Pretty much, see also the requirements (https://pve.proxmox.com/pve-docs/chapter-pveceph.html):

We recommend a network bandwidth of at least 10 GbE or more, which is used exclusively for Ceph. A meshed network setup [4] is also an option if there are no 10 GbE switches available.

Since Ceph replicates syncrounously, a write command will only be confirmed once at least one copy has been written on the other side. Small bandwidth and high network latency makes it so that you will wait for a long time for your iops to be completed.

You could try using ZFS instead with (asynchronous) replication, but I dont have much experience with that. Also iirc that still requires the hosts to be in the same cluster and Proxmox also requires low latencies for the corosync cluster network.

Generally, making a stretched cluster is not feasible with Proxmox right now.
 
You must log in or register to reply here.
Top Bottom