Storage Replication Or Drbd

Deepak Singh · Oct 11, 2018

Dear Members,

I am new to proxmox and I have few doubts.

I am using two nodes cluster with proxmox 5.2 and want to configure the HA without any external storage.

As I can see may topics on DRBD, I am familiar with DRBD and its a good tool, but when I see the Storage replication option in proxmox I get confusion.

1. Is it possible to use storage replication feature and configure the HA with two nodes.

2. If not then will it still achievable through DRBD.

3. If Point number 1 is not possible then what is the exact use of storage replication feature?

Your inputs on these points will be appreciated.

Regards

tom · Oct 11, 2018

Deepak Singh said:
Dear Members,

I am new to proxmox and I have few doubts.

I am using two nodes cluster with proxmox 5.2 and want to configure the HA without any external storage.

As I can see may topics on DRBD, I am familiar with DRBD and its a good tool, but when I see the Storage replication option in proxmox I get confusion.

1. Is it possible to use storage replication feature and configure the HA with two nodes.

2. If not then will it still achievable through DRBD.

3. If Point number 1 is not possible then what is the exact use of storage replication feature?

Your inputs on these points will be appreciated.

Regards

You cannot get HA with 2 nodes. See documentation for Proxmox VE HA.

I suggest configure Proxmox VE replication (based on ZFS). And do not use HA.

Deepak Singh · Oct 17, 2018

tom said:
You cannot get HA with 2 nodes. See documentation for Proxmox VE HA.

I suggest configure Proxmox VE replication (based on ZFS). And do not use HA.

Thanks for the suggestion tom, but with replication can I achieve the automated failover if one node gets down?

Regards,

udo · Oct 17, 2018

Deepak Singh said:
Thanks for the suggestion tom, but with replication can I achieve the automated failover if one node gets down?

Regards,

Hi,
no - how should it works?

If you have only two nodes and one node die, or better the network connection between the nodes go down, none of the node have quorum.

Otherwise it's be fatal, if one node start the same VM in case of an network issue, the VM will run on both nodes on the same time...

This is the reason, why you need three nodes for HA (the third node can be very small - only for quorum).

Udo

adamb · Oct 17, 2018

Worth also adding, the third node can be quite a few linux OS's which have the corosync-qdevice package available. Even BSD has some ports.

guletz · Oct 17, 2018

Hi,

As @udo say, you need 3 nodes for a safe cluster. But you can use a raspberry 3.141 device as a 3rd node. After this step you can use zfs replication and HA .

Deepak Singh · Oct 19, 2018

I got the point behind the HA, I need to test on Raspberry as @guletz has a good suggestion mean while @udo your comment on raspberry is appreciated

guletz · Oct 19, 2018

Take a look here:

https://pve.proxmox.com/pipermail/pve-devel/2017-July/027741.html

https://forum.proxmox.com/threads/pve5-and-quorum-device.37183/

Deepak Singh · Nov 3, 2018

I have installed the corosync in raspberry pi and below is the output

pi@pve3:~ $ sudo corosync-quorumtool | grep Quorate:
Quorate: Yes

Which seems to be working .

I used this link https://pve.proxmox.com/wiki/Raspberry_Pi_as_third_node

Now I am not using shared storage and all nodes are in cluster, so how should I check that the HA is working fine?

Regards

guletz · Nov 3, 2018

Deepak Singh said:
Quorate: Yes

This show you that your cluster is ok.

Deepak Singh · Nov 3, 2018

Thanks for the reply @guletz

But I want to check weather failover is happening or not, so should I have to manually down the one server and check, that the vm's are moving on second server?

I am not using any shared storage the vm disks are on lvm-thin so technically the failover will happen or not?

Regards

guletz · Nov 4, 2018

Deepak Singh said:
But I want to check weather failover is happening or not, so should I have to manually down the one server and check, that the vm's are moving on second server?

Yes. It is highly recommended to test this situation. I do this kind of tests for myself from time to time.

You could also test the case when you lose the network, without power off the node. Only unplug your network cable from server(I also test this). Then after 10 min I plug on the network. And the last scenario that I test is to unplug the power outlet on one node (as a unexpected power loss case)

This are my own test that I do at 3 month.

Good luck !

udo · Nov 6, 2018

Deepak Singh said:
Now I am not using shared storage and all nodes are in cluster, so how should I check that the HA is working fine?

Hi,
how should HA work without shared or distributed storage?
If one node goes down, the VM-data are on this node only...(and not accessible) therefor pve can't start the VM on another node.

Udo

guletz · Nov 6, 2018

Udo, he can use some kind of HA without any shared/distributed storage:

- he can use zfs on both compute nodes
- he can define a replication from node A to B for vmx1...
- he must define ha for vmx1 restrict the nodes (A and B only)

The downside is the fact that if node A is broken, then the vmx will start with the data from the last successful replication of vmx1. Yes is not perfect but is more then nothing(in my terms I calling this as poor man HA)
But you can define a short period for replication (2 min for example), and in some cases where data is changing not so often is ok (a web server, a dns server)

udo · Nov 8, 2018

guletz said:
Udo, he can use some kind of HA without any shared/distributed storage:

- he can use zfs on both compute nodes
- he can define a replication from node A to B for vmx1...
- he must define ha for vmx1 restrict the nodes (A and B only)

The downside is the fact that if node A is broken, then the vmx will start with the data from the last successful replication of vmx1. Yes is not perfect but is more then nothing(in my terms I calling this as poor man HA)
But you can define a short period for replication (2 min for example), and in some cases where data is changing not so often is ok (a web server, a dns server)

Hi Guletz,
good to know.

Udo

Deepak Singh · Nov 8, 2018

guletz said:
Udo, he can use some kind of HA without any shared/distributed storage:

- he can use zfs on both compute nodes
- he can define a replication from node A to B for vmx1...
- he must define ha for vmx1 restrict the nodes (A and B only)

The downside is the fact that if node A is broken, then the vmx will start with the data from the last successful replication of vmx1. Yes is not perfect but is more then nothing(in my terms I calling this as poor man HA)
But you can define a short period for replication (2 min for example), and in some cases where data is changing not so often is ok (a web server, a dns server)

@guletz

Thanks for the help, now I am stuck in the problem of LVM to ZFS, While installing the lvm-local partition is created and for replication I have to use ZFS, so is there any way I can covert that to in ZFS.

If I am not able to convert it then how I can reinstall the Proxmox with ZFS ?

Regards.

guletz · Nov 8, 2018

Deepak Singh said:
Thanks for the help, now I am stuck in the problem of LVM to ZFS, While installing the lvm-local partition is created and for replication I have to use ZFS, so is there any way I can covert that to in ZFS.

If I am not able to convert it then how I can reinstall the Proxmox with ZFS ?

Hi,

zfs can be installed on any block device, including lvm. But is not a optimal solution (you will use a unnecessary layout ). The best you can do is to backup any VM that you have on each node. Move your backup files on the other node, and after that you can make a reinstall of PMX on this node using zfs.
Then you could repeat this tasks for any other node that you have. Of course after your node is have zfs then you can move back the backup files on the zfs node and restore on zfs.
Sorry for my bad english, but I hope you will get the point.

Also I can suggest to take your time and play with zfs. You can use plain files and test zfs on instead of real hdd. zfs it is a very good tool if you take some time to read about and to learn some basics.
As I learn myself, if you use a not so good tool (aka lvm) but you know how to manage at a decent level, then it is far better then use a better tool (aka zfs) but you do not know the basics.

Good luck!

Deepak Singh · Nov 11, 2018

guletz said:
and after that you can make a reinstall of PMX on this node using zfs

Hi @guletz

Even I am planning for the same I have One node which is completely free, where I want to install the PMX with ZFS, the problem is that I am not able to find any option at the installation where ZFS is to be selected. Please let me know how to use ZFS as a file system at the time of installation.

Your response is highly appreciated.

Regards,

guletz · Nov 12, 2018

Hi,

I would recomended you to start a new post/thred with your problem(like: new install without zfs option)

Deepak Singh · Nov 12, 2018

Hi

Thanks for the suggestion.

I have created a new thread please see https://forum.proxmox.com/threads/proxmox-installation-with-zfs.48732/

Regards

Storage Replication Or Drbd

New Member

Proxmox Staff Member

New Member

Distinguished Member

Famous Member

Famous Member

New Member

Famous Member

New Member

Famous Member

New Member

Famous Member

Distinguished Member

Famous Member

Distinguished Member

New Member

Famous Member

New Member

Famous Member

New Member