2.5 node setup with JUST corosync on the .5?

[syadnom]

I'm looking for a nice solution to a 2 node cluster with a proper 3rd node for quorum.

How about a small box that just runs corosync to provide quorum? Would also be used for the 3rd node for ceph, but without local storage.

My target box would be a PC Engines APU unit.

Thoughts?

 

[LnxBil]

There exist a working quorum node on a raspberry pi (please use the search box), so it technically works.

APUs are awesome. We're running two of them in our big cluster (one cluster with servers and apu boxes)

 

[syadnom]

There exist a working quorum node on a raspberry pi (please use the search box), so it technically works.

APUs are awesome. We're running two of them in our big cluster (one cluster with servers and apu boxes)

I like Pi's, I have many of them deployed for service kiosks...but I'm a bit wary of running them as a key part of a virtualization cluster...

How do you use the APUs? Are they just for quorum or are you actually hosting VM's/containers on them? Also, how did you install? I just did a debian8 netinstall and ran the install scripts, which is a bit time consuming.

 

[mir]

For a pi: https://forum.proxmox.com/threads/rb-pi-for-third-node.28095/

 

[LnxBil]

How do you use the APUs? Are they just for quorum or are you actually hosting VM's/containers on them? Also, how did you install? I just did a debian8 netinstall and ran the install scripts, which is a bit time consuming.

I run my HA firewall and secondary DNS/DHCP on the APUs. Runs very smooth over DRBD and I also have TRIM for the internal SSDs enabled. Of course they are a bit slow but I only notice this when I do backups. The machines only gets approx. 40 MB/sec with default LZOP backup. Because they're integrated in my main cluster, this setup works as a real HA with quorum and everything. If my whole rack cluster breaks with the attached SAN (of course everything at least twice redundant, so very, very low probability), I'm able to have DNS/DHCP and Internet, so at least some things will still work.

I installed via serial console netboot with stock debian and upgraded to Proxmox VE. Works very smooth but of course not as fast as to plug in a CD-emulator on a usb disk and press enter a couple of times.

Yet I really like serial console. I have even automated some serial consoles via ESP8266 over WiFi. That's really fun!

 

[syadnom]

got a good guide for a 3 node DRBD (2 mirror, 1 quorum) ?

 

[LnxBil]

No, but it's a simple 3 node cluster with two nodes having an additional DRBD. Just build it in that order and it should run fine.

 

[syadnom]

so, ceph w/ 3 monitors and 2 OSD hosting nodes (with 6 OSD per) or DRBD on the 2 main nodes?

Maybe my logic needs a sanity check here, but DRBD w/ LVM on top *doesn't* need a quorum because proxmox will have quorum yes? There is no chance that data same-blocks will be written to by both prox hosts...

DRBD should be faster too right? I have 10G ethernet between the two proxmox boxes that could be dedicated storage.

 

[LnxBil]

Two nodes cannot have quorum, you need an odd number of nodes for that. If you have a split-brain situtation in DRBD, each node thinks its the master and write its own data to disk.