Proxmox on OVH

D

Deleted member 33567

Guest
I have to say that this tonights experience, I will leave OVH hosting for sure, and thinking for replacements for Proxmox too.

Few points about these 2 dark sides of OVH and Proxmox, both are ^ oriented, OVH does not really build a platform around users, they build a platform around whats cheap and sell it expensive.

Proxmox uses at best open source technologies to get back a product to fight million dollars companies that invested into developing some of the very own pieces used in todays virtualization softwares.

Point simple, this whole hybridization and the custom Vrack networks from OVH, has nothing to do or less with whole Virtualization idea behind Proxmox. So in other words, no support natively between proxmox and OVH Vrack, in fact if one wants a secure network, that must be in a vrack, and there one looses ipv6 access to machines, since OVH does not route ipv6 in any ways there, and much more.

For those strugling with OVH, avoid it at any cost, their support has went from doing something, to doing nothing, they will follow a routine, and thats it.

From 21:00 until now 02:00 time of the post, the issue with the network unreaching anything is still not solved, I have tried complete network settings reset on OVH side via rescue mode. Nothing helped.

Over IPMI one cannot work anything:
1615165840590.png
Because these errors/warnings are non stop, no matter what one does, nano, editor, what ever...

Please note that the other 2 servers that were not rebooted, are working fine, to be honest I do not dare to reboot them. I am working now hardly to record any data impact and how this services and kvms moved outside of this machine, and trying out best to clear out the OVH servers and forget I ever had to use such services.

They might have some attractive offers, the 500 GB backup with every server, yet what use features, when the critical need of a support when a issue occurs, is not there, unless one pays all extra layers of mega hyper super smartass support. Only then you can work your way out of this jungle of something that should had been simple.

This was the network setup at the reboot:

1615165926547.png

The only change in above image is the IP ending in .99 . Since this above is from a working server.

Networking made complex for no reasson, without a proper documentation on either sides, since these werent updated in some time now, same for OVH and promox.

They speak about partnerships, who benefits from them? If users join the OVH servers, and users decide to pay Proxmox support, the beneficiary is Proxmox, if the user pays support to OVH, the one benefiting is OVH, while these installs should work out of the box, if in century 21 a critical machine made to handle critical operations, cannot go from a normal state after a single reboot, to destroy the whole networking to a point where every efforts to recover it are in vain.

If anyone has experience in Europe with better providers then OVH I am open to suggestions.

Thank you, and watch out for OVH+Proxmox combination!
 
Last edited by a moderator:
Hello!

without a proper documentation on either sides, since these werent updated in some time now, same for OVH and promox.
The Proxmox VE reference documentation contains a chapter about host network configuration. The reference documentation is updated from our git repositories and should always be up to date, as a consequence. If you could explain what information you are missing or what is wrong, we'd be happy to update it!

The screenshot indicates that the network connection between the members of your Ceph cluster is broken. To help you troubleshoot this, we need more information about how you have set up your Ceph cluster.
 
f anyone has experience in Europe with better providers then OVH I am open to suggestions.
If you know what you are doing Hetzner is a good provider. Good hardware for cheap prices but you really need to know what you are doing and you are in business. Hetzner has vswitches which are similar to the vracks. One can also connect cloud instances and dedicated servers via vswitch vlan.
Hope that helps.
 
Can you SSH into your n01
  • from your computer
  • from n02 or n03
?
 
I don't know what's happening at OVH, I thought my problem was maybe local to my server.

In my case I had a node of a PVE cluster go down with a mainboard failure, the mainboard was replaced and the server was brought back online but I couldn't get any kind of access to my private network over my vRack. Only other variable besides replacing the mainboard is I had upgraded the Proxmox on the box about 6 weeks ago to the then current release, but I had never rebooted it so I was not running the installed kernel. I've since canceled the server since OVH tech support was useless and I've been unable to do anything to pass packets on the vRack network for 3 weeks now. I used the public network and the command line to migrate all my VMs to other nodes and have since done a complete OS reinstall just to see if that would shake anything out with no change. Also the OVH control panel network graph shows I'm passing a total of 0 packets on the vRack network. My vRack is working fine between my other existing boxes with heavy traffic flowing between multiple data centers for my ZFS offsite replication.

My theory is someone didn't program the new MAC address for the private NIC into the local network switch, but after leading this thread I'm thinking it might be something more wide spread and not local to this specific box.

I've since ordered a new server to replace my dead node from their new HG line with the hope of it actually working and getting a hardware refresh for my troubles.

To follow on to Dominic's question, can you ping any of the IP addresses on your private NICs from the working nodes?
 
I've been using OVH since ~2014, and the vRack is the reason why (Hetzner's vswitches came quite late after I've already invested time in understanding and getting ProxMox working on OVH's network)

Yes, OVH's vRack has limits like no IPv6 routed into it.
Yes, their public/outside interface routing is "strange" especially w.r.t. fail over IPs (but I recall it's similar as Hetzners but I might be mistaken)

That said the issues I've encountered was mostly me doing L2 trickery that back fired for "expected" SDN reasons.

Also you have to understand, you are deploying it in a networks that is VERY cost effective (ie. cheap) so you need to make sure you have Hendrik's rules on computing in place:

1) make a backup (in OVH I use the HDDs while the VMs runs on the NVMes)
2) Make *another* backup (Using the OVH backup NFS/CIFS services.. for *backups*)
3) *CHECK* those backups (restoe testing to the other datacentres...)
4) Make sure you have a off-provider copy (I'm using rsync.net, tuxis.nl's PBS and tarsnap.com for various parts and clients)

If you want beter, do check the list on https://www.proxmox.com/en/partners/hosting-partner or best: deploy your own hardware, using proper switches, network interfaces, in your own datacenters with redundant generators, UPSes and aircons each in mulitple countries and suddenly your costs are 10 orders of magnitude more expensive.

Else, just go get an GCP/Azure/RedShift/AWS accounts and deploy there.

in short: OVH *works*, and it work for *me*, and I'll happily assist people in running a ProxMox clusters in OVH, as that's my work, but before you complain about OVH, understand why and how you might've been negligent yourself before blaming OVH for providing a real cheap service (which still has impressive uptimes IMHO! excluding a fire in a DC that had been unfortunate... people keep forgetting AWS's failures)
 
  • Like
Reactions: noko
If you know what you are doing Hetzner is a good provider. Good hardware for cheap prices but you really need to know what you are doing and you are in business. Hetzner has vswitches which are similar to the vracks. One can also connect cloud instances and dedicated servers via vswitch vlan.
Hope that helps.

:)) Hetzner 3MB UDP and they close your server connection, cmon :) i was expected more then this because they advertise. This happened to me after one customer was keep requesting server from them but us to make the management.

DDoS protection​

Hetzner Online will safeguard your Hetzner cloud servers using the latest hardware appliances and sophisticated perimeter security technologies, providing you with first-rate protection against large-scale DDoS attacks. And all that free of charge.
 
OVH works for me also, but I'd like them to at least respond within 2 weeks to my support request with something other then to go read the knowledge base articles or someone is looking into it now. I understand they are a bit over worked with the fire and all but since my dead box was in Canada I assumed someone in Canada could have gotten around to looking at it within 2 weeks.

Anyways it made for great motivation to cancel the box and order one of their new High Grade series to replace it, the value for these boxes is incredible(well maybe I'll feel different once they deliver it, but the specs are great).
 
Can you SSH into your n01
  • from your computer
  • from n02 or n03
?
I wished we could, yet it is not working.

These are the order of the things how it happend.

- OVH toke the server offline for decomissioning, without a valid reason
- server was part of a HA cluster running also ceph
- After few days they said our server is online, yet we never managed since to get access even to working "arp -a" inside the OVH IPMI .

- Now are over 17 days since they disrupted over 20+ infrastructure apps/testing apps, and they did not even bothered to get us a resolution to our support ticket when we opened one, they simply closed the ticket, thats not even legal...


So we decided we need to take action. Ignorance is not a solution in this case anymore. this is a time when it shows how well both proxmox and ovh go hand in hand. :)
 
Last edited by a moderator:
@proxmox admins, we demand complete removal of our accounts on this forums and or on your systems due to GDPR laws.
 
That said the issues I've encountered was mostly me doing L2 trickery that back fired for "expected" SDN reasons.
@hvisage

Reviving an old thread...

Can you kindly please explain more about this? and is it still "expected" for PVE8.3?

I have been bashing my had in the walls for 2 days with my new OVH cluster using the Advanced server line and vRack.
VM to VM via vlan through vRack are not working in any configuration ( vlan, vlan-bridge, SDN )
althou VM to Host is working!
everything is already accounted for, firewall/ebtables

I have the same setup using Rise based servers, working perfectly
Reached out to OVH for support, but you know how slow they are...

I suspect the issue is on the vRack side at OVH, however i am interested knowing more about the L2 trickery that back fired at your end.

Thanks
 
the only way i was able to make VLAN's properly work across the vRack is by using SDN VXLAN, which was a life saver in my situation.

however, this does i think introduce additional latency/overhead to the network traffic and defeat the purpose of having a vRack or a low latency private network.
 
Hi @mikos , I've been runngin VLANs using OpenVswitch over OVH's vRacks without issues (other than when I tried to "map" an IP/vMAC from a public interface to another host via the Vrack interface using VLAN trickety). Currently 2x separate ProxMox clusters deployed with OpenVSwitch that trunk/vlan over the vRack

Their vRacks are 802.1q-in-q type, so 1q vlans "just worked" (The vRack interface is set to be a trunk interface)

The only other "gotcha" is when you have them bonded/multiple linkz (like the OLA and the Scale servers) where you should not setup a Linux BOND interface, but do the bond in OVS to bind to the OpenVswitch.
 
Hi @hvisage, thank you for your feedback much appreciated.

I migrated from an old cluster using the Rise servers line, using SDN Simple Vlan and the cluster worked without any issues, i have started experiencing problems when i added Advanced servers to the cluster, and based on OVH Support feedback it's that they use a different network topology for the vRack but it should not introduce such issues, but in my case it did.

I narrowed it down yesterday to one of the Advanced servers, and used a Public Cloud instance attached to a VLAN on the vRack to track tagged traffic, seems this server is not properly passing traffic over the vRack ( although tcpdump shows ARP requests ) .

The second Advanced server deployed in the cluster works perfectly.

Currently i am not using OLA feature, just routing all traffic ( Public and Private ) via the vRack ( VMBR1 ).
Also never used OpenVSwitch, as SDN features currently in Proxmox worked for me on all deployment scenarios, but i guess i will have to give it a try on the current cluster.

Thanks again
 
I narrowed it down yesterday to one of the Advanced servers,
YEs, the RISE/kimsufi/soyu(?) are on different networks vs. the OVH Advance/Scale/Infra servers....

Might need to double the linux bridge and network settings on this server. But yes, there could've been a network config issue to address
 
Okay, so i activated OLA on the Advanced server that was having issues did fix it.
VLANs between Proxmox nodes is properly working now.

Strange OVH vRack configuration though, this should be mentioned somewhere in their Docs.
 
The only other "gotcha" is when you have them bonded/multiple linkz (like the OLA and the Scale servers) where you should not setup a Linux BOND interface, but do the bond in OVS to bind to the OpenVswitch.
I tried with the Linux Bond interface and it worked
i was able to assign the Public IP from the IP block routed through the vRack and also enable the vlan-aware to tag traffic
 
  • Like
Reactions: hvisage