Ceph storage and VM in the same proxmox hosts?

[openaspace]

I understand. But do you really think that despite all the CEPH traffic, only a virtual machine will contributes to the degradation of performance?

Having a cluster of 3 hosts and not having a live replica of a single VM adopting another backup strategy would be almost paradoxical or simple more expensive.

What will happen? a simple slow performance of the general throughput? Can be also acceptable..

 

[sg90]

I understand. But do you really think that despite all the CEPH traffic, only a virtual machine will contributes to the degradation of performance?

Having a cluster of 3 hosts and not having a live replica of a single VM adopting another backup strategy would be almost paradoxical or simple more expensive.

What will happen? a simple slow performance of the general throughput? Can be also acceptable..

Sorry I don't get what your question is there.

 

[openaspace]

I mean that run a cluster of 3 host without replicating the VM it's expensive.
You really think that's it's not a good idea to use zfs raid 1 instead of mdadm ?

 

[sg90]

I mean that run a cluster of 3 host without replicating the VM it's expensive.
You really think that's it's not a good idea to use zfs raid 1 instead of mdadm ?

Yes, as ZFS is quite memory hungry. Fact you will already be running the VM's and the two OSD proccesses on the RAM of each 3 servers id sugest trying to keep as much memory available for them two and not add any extra resource requirements.

And I don't see how ZFS would add much extra useful functionality to your setup, and if anything just add extra complication.

 

[openaspace]

And I don't see how ZFS would add much extra useful functionality to your setup, and if anything just add extra complication.

Only for file system replication over lan and wan.

 

[sg90]

Only for file system replication over lan and wan.

I thought you wanted to run separate VM's on each host? and not one VM and move across the 3 hosts?

 

[openaspace]

I thought you wanted to run separate VM's on each host? and not one VM and move across the 3 hosts?

at the moment only one VM with failover possibilities of the Ceph,
but also replicate with zfs over the wan the VM to my office with 270mbps fiber upload bandwidth for emergency purposes only.

 

[sg90]

at the moment only one VM with failover possibilities of the Ceph,
but also replicate with zfs over the wan the VM to my office with 270mbps fiber upload bandwidth for emergency purposes only.

Then it really makes sense to host all the images onto the CEPH Storage layer then, your just be limited by the 1Gbps network, but then it seems you don't have huge I/O requirements unless these big files are being changed constantly?

Maybe a good idea to list what kinda I/O you expect? Like are you writing a new 270GB file every hour.. or is it write a file once and just read it a few times every week.

 

[openaspace]

idea

No no there was a misunderstanding.
I will store big files from 200 gb and more, that will be changed rarely (only 4-5 new files at month) but they will be read all the time.

 

[sg90]

No no there was a misunderstanding.
I will store big files from 200 gb and more, that will be changed rarely (only 4-5 new files at month) but they will be read all the time.

Okie, Well for example a 200GB file will take 30 minutes to fully read from your CEPH cluster if the whole 1Gbps NIC was used for nothing else. If 50% of it was used for other network traffic either WAN or LAN it would take a min of 60 minutes to read.

If you then go and add ZFS traffic for the OS disk onto that aswel then your increase the time even more. So its really up to you and what performance you expect / need.

 

[openaspace]

If you then go and add ZFS traffic for the OS disk onto that aswel then your increase the time even more. So its really up to you and what performance you expect / need.

Thank you..
It's clear that with only 1gbps vswitch and wan it's all pier to saving money as well performance.

"No money, no parties"

 

[sg90]

Thank you..
It's clear that with only 1gbps vswitch and wan it's all pier to saving money as well performance.

"No money, no parties"

Also you can't easily change an IP between hetzner servers unless you pay for a failover IP, but then each change has to be done via their API or controlpanel. So if you have a HA VM that moved from one server to another it would have no network connectivity till you also completed the IP move with them.

 

[openaspace]

Thank you
I have already tested this with cloudflare dns, avoiding ip failover.
With cloudflare you can change ip in the NS of the hostname with live propagation. It works great and it's really fast.

https://blog.cloudflare.com/introducing-load-balancing-intelligent-failover-with-cloudflare/