Automatic Migration of a failed VM

[proxmox_larry]

Hey guys,
I'm currently using a HA cluster with Ceph as shared storage.
Is it possible to initiate a automatic Migration of a VM if it loses the connection to the NIC on one node???
Thanks!

 

[t.lamprecht]

At the moment this is not possible easily. Whats the scenario when this happens, but the host were the VM is currently located still has network? As else, if the host also looses network, it will be already fenced and recovered.

 

[proxmox_larry]

The problem is that when you unplug the ethernet cable dedicated to the ethernet port which the VM is using, nothing happens!
The VM will lose the ethernet connection, but the host is still running in the cluster.

As you said there is still a chance to fix this?? How?
I think that Qemu should recognize the missing ethernet connection and migrate the VM to a node that has a active link...

 

[czechsys]

The problem is that when you unplug the ethernet cable dedicated to the ethernet port which the VM is using, nothing happens!
The VM will lose the ethernet connection, but the host is still running in the cluster.

As you said there is still a chance to fix this?? How?
I think that Qemu should recognize the missing ethernet connection and migrate the VM to a node that has a active link...

That's problematic grey zone. What if link is up, but vlan is down? What if vlan is up, but some fw will block all connections?

Ask Qemu about this. This is not Proxmox problem.

Or use your monitoring and Proxmox api for migration such VM.

 

[proxmox_larry]

That's not really helpful...

 

[czechsys]

That's same as when hdd fail. That's why disk raids, network lacps etc exists. Redundancy.

 

[fabian]

I have to agree with @czechsys - this is what monitoring and having our REST API is for.. there are way too many failure conditions, some of which are only detectable outside of the PVE node. setup your monitoring to detect that the service in the VM has failed, and if the node it is running on is not fenced, attempt a migration/reboot/hard-reset/.. of the guest (or load-balancer fail-over to another instance, or ...). if the node has been fenced already, our HA stack should start the VM on another node anyway, your monitoring will see the service coming up again (hopefully - depending on what's caused the failure )

 

[proxmox_larry]

Thank you for the detailled answer!
Since I'm not a pro in using the API, what would be the best way a to get familiar with it and use it for migration in case of NIC failure?

 

[fabian]

https://pve.proxmox.com/pve-docs/api-viewer/index.html too get an overview over how the API is structured, which parameters/return values there are, etc.pp. (also shipped with your copy of PVE, see the "Documentation" button in the GUI)
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_user_management might also be interesting to create a user with limited permissions
https://pve.proxmox.com/wiki/Proxmox_VE_API for developer-oriented information regarding the API, and various clients/bindings.