PVE6 and Mikrotik EoIP

[guletz]

Yes. With any tunnel you can connect 2 hosts/networks.

 

[Drummond Korras]

I was referring to this post
https://forum.proxmox.com/threads/pve6-and-mikrotik-eoip.59536/post-274785

The std dev was too great to accommodate a cluster over WAN

 

[Drummond Korras]

The std dev is now 0.140ms

 

[guletz]

Hi,

So it is better now or not?

 

[Drummond Korras]

Yeah alot better
EoIP = 4.608ms
L2TP,MPLS/VPLS =0.140ms

 

[guletz]

Ok. Now if you want to be kindly, so other forums members can use your experience, maybe you can detail your setup (anonymise ip or other sensible info). Also a mikrotik export (only vpls, l2tp part). I promise that I will explain your exports so anybody will understend!

Thx. very much for your feedback!

Good luck / Bafta!

 

[Drummond Korras]

I most certainly will share - I just want to know from @Alwin if that would suit the Promox VE6 cluster to join over WAN as he suggested.
I will try connect the cluster during the week and post my results

Thank you

 

[guletz]

... anybody make mistakes. Or we can not see the landscape of a person who has a problem. I like a Chinese sentece ... as I rember. Is not important what colur have the cat , but is important if the cat is able to catch the mouse ... (sorry my bad english ).

But you must judge in the end with your own head despite anybody ...whatever will say(me included here).

Enjoy with wonderful Proxmox, linux, mikrotik and mpls.

Goog luck / Bafta.

 

[Drummond Korras]

Ok So these were my findings:
MPLS/VPLS&VLAN - Tries to connect, establishes connection but throughput too slow, eventually collpases [1.96MBps]
L2TP/EoIP&VLAN - Connects and joins, waiting for quorum, all nodes reboot and the entire cluster dies, have to forcefully remove remote node and wait for quorum on the other 2 nodes [80MBps which is to be expected on a 100MBps link]

omping -c 10000 -i 0.001 -F -q 192.168.2.4 192.168.2.2
192.168.2.4 : waiting for response msg
192.168.2.4 : joined (S,G) = (*, 232.43.211.234), pinging
192.168.2.4 : given amount of query messages was sent

192.168.2.4 :   unicast, xmt/rcv/%loss = 10000/9990/0%, min/avg/max/std-dev = 1.382/1.899/33.215/0.804
192.168.2.4 : multicast, xmt/rcv/%loss = 10000/9989/0%, min/avg/max/std-dev = 1.449/1.975/33.243/0.802

Error as seen on remote node3: something about /etc/nodes/node3/pve-ssl.pem no such file for directory
Please note, the EoIP setup worked perfect with PVE5

Best option on PVE6 - setup QDevice

 

[Drummond Korras]

Here is the syslog of the node3 trying to join the cluster

https://pastebin.com/PV7n1WBY

Futher to this - my /etc/hosts is correct with the correct IP's and hostnames

 

[Alwin]

I just want to know from @Alwin if that would suit the Promox VE6 cluster to join over WAN as he suggested.

No. Please see my previous posts. I strongly advise against running corosync over WAN. The cluster needs low and stable latency.
https://pve.proxmox.com/pve-docs/chapter-pvecm.html#_cluster_network

 

[Drummond Korras]

OK perfect, thank you - I WIll have to use QDevice and later purchase another server

 

[guletz]

MPLS/VPLS&VLAN - Tries to connect, establishes connection but throughput too slow, eventually collpases [1.96MBps]

Hi again,

Sorry for my late reply. I think that yor slow throughput is so small because you have not take in account l2mtu. Like I said I do not use vlans but I have had use eoip and l2tp on mikrotik(using cheaper devices compare to you). And I was able to see at least 20-30 Mbits using eoip and 40% min cpu usage. Then I start to use vmpls with the same bandwith, with 10-20 % cpu usage.

As I seen if you replace any tunnel (unencrypted) with vpls with the wright settings, you can get the same speed, better latency and less resurces.

You can find a lot of tutorials about l2mtu and mikrotik.

Good luck/ Bafta