Guest default gw after migrating

[thesquiffy]

Hi, I'm currently testing pve features on a 3x nodes cluster.
I have some test vm which I'm able to migrate from host to host easily.

My problem is: I'm using host's ip as current default gateway for the guest os to outgoing to internet, how do I inform/switch the gw in case of a migration?
Do I have to manually log into the guest and change it manually?

At the moment, if I migrate a vm and guest still reference the old host it's working, but I'm guessing if there is an easy and correct way to simplify the post-migration process.

EDIT: I made a diagram to better explain:



Thank you for any help!

 

[Sycoriorz]

If i understand you right you have your hosts behind router/dhcp. And you are using 3 nodes wich has own ip in the network behind the router. You want to use the same nic for guest traffic in the same network as the host connect to the router? If yes you create on all 3 nodes a linux bridge with the same name. As reference nic you choose the nic that you want without any further settings. Then you choose on guest config the created bridge.
You will get automatically an ip adress by the router from the dhcp. Now you can migrate from node to node without any problems. To be sure to stay on the same adress you can say the dhcp server give all the time same ip to this machine based on mac adress by virtual-nic.

 

[thesquiffy]

If i understand you right you have your hosts behind router/dhcp. And you are using 3 nodes wich has own ip in the network behind the router. You want to use the same nic for guest traffic in the same network as the host connect to the router? If yes you create on all 3 nodes a linux bridge with the same name. As reference nic you choose the nic that you want without any further settings. Then you choose on guest config the created bridge.
You will get automatically an ip adress by the router from the dhcp. Now you can migrate from node to node without any problems. To be sure to stay on the same adress you can say the dhcp server give all the time same ip to this machine based on mac adress by virtual-nic.

Hi, thank you for your reply.

Please have a look at the diagram I added to the main thread. Basically, when I need to migrate a VM from an host to another, I also need to change its (guest) default gateway manually. I don't have a common router.

 

[Sycoriorz]

I think but not sure it would be the best to place a reverse proxy between internet and gateways. This would route the vms. But this scenario is new for me and have no experience in this scenario. Here i can not help sorry.

 

[jlebherz]

have you tired to assign an additional ip to lo on alle three hosts and use this address as default gateway?

example
Host1
IP: 192.168.15.11
lo IP: 192.168.15.15

Host2
IP: 192.168.15.12
lo IP: 192.168.15.15

and than use the 192.168.15.15 as default gateway for your vms

 

[thesquiffy]

have you tired to assign an additional ip to lo on alle three hosts and use this address as default gateway?

example
Host1
IP: 192.168.15.11
lo IP: 192.168.15.15

Host2
IP: 192.168.15.12
lo IP: 192.168.15.15

and than use the 192.168.15.15 as default gateway for your vms

hi, I like your idea, could you please just explain a bit more? by "lo" do you mean the loopback device? How do the vms could reach such ip if local only to the hosts?

this is my current interfaces file from one my hosts:

auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual

auto eno2.99
iface eno2.99 inet static
        address 10.240.99.1/24

auto vmbr0
iface vmbr0 inet static
        address XXX.XXX.XXX.XXX/24
        gateway XXX.XXX.XXX.XXX
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0

auto vmbr1
iface vmbr1 inet static
        address 10.240.10.1/24
        bridge-ports eno2.10
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

thank you!

 

[jlebherz]

yes, lo is loopback

please tell me some more details..

IP Configuration of all three hosts?
How did you configure the routing?
how are the vms configured?

 

[jlebherz]

we are doing it like this in our datacenter

auto lo
iface lo inet loopback

auto eno1
iface eno1
    address xxx.xxx.xxx.xxx/30

auto vmbr0
iface vmbr0
        address 192.0.2.1/31
        bridge-stp off
        bridge-fd 0

each host has a running bird instance and we have a script watching which vms are running on this host
and then adds routes to the bird for example

protocol static {
    xxx.xxx.xxx.xxx/32 via 192.0.0.1 via "vmbr0";
}

bird distributes this routes via iBGP to the core Routers


How is your Configuration, do you have a Router in front of your Hosts, are you using HA IPs from the datacenter or are you natting the VMs?

 

[thesquiffy]

yes, lo is loopback

please tell me some more details..

IP Configuration of all three hosts?
How did you configure the routing?
how are the vms configured?

thank you soooo much!

it worked with:

ip addr add 10.240.10.1/32 dev lo

unfortunately I'm unable to set it correctly on network/interfaces file but isn't a great issue, I'm going to set it at startup.

 

[thesquiffy]

we are doing it like this in our datacenter

auto lo
iface lo inet loopback

auto eno1
iface eno1
    address xxx.xxx.xxx.xxx/30

auto vmbr0
iface vmbr0
        address 192.0.2.1/31
        bridge-stp off
        bridge-fd 0

each host has a running bird instance and we have a script watching which vms are running on this host
and then adds routes to the bird for example

protocol static {
    xxx.xxx.xxx.xxx/32 via 192.0.0.1 via "vmbr0";
}

bird distributes this routes via iBGP to the core Routers


How is your Configuration, do you have a Router in front of your Hosts, are you using HA IPs from the datacenter or are you natting the VMs?

These are 3x ovh dedicated servers directly connected to internet with first nic and to their vrack/vlan with second nic.
The second bridge it's used to let the VMs to communicate and to reach other pysical servers I run into the vrack/vlan (via masquerade)

I never used BIRD tbh...