External IPs now working after Restore or Migration

[TimonTNT]

Hey.
I have 3 servers with Proxmox 4.3 with 5-10 VMs each.
Each server have vmbr0 with eth0 for internal network and vmbr1 with eth1 for public IPs
Here the problem:
- when I backup VM and restore on the same server it's working
- when I backup VM and restore on another server my public IP are not working (the same happens with Live Migration)
--- I cannot ping VM, VM cannot access any network

My solution:
- remove 'Network interface' from VM after migration
- remove NIC from /etc/udev/rules.d/70-persistent-rules.net
- reboot
- add new 'Network Interface' (with random MAC)
- since /etc/sysconfig/network-scripts/ifcfg-eth0 already exists within VM nothing else is needed

And when I tried to configure 3 nodes in cluster (fresh installs) with public IP's on eth0, whole network went down during 'pvecm add IP-ADDRESS-CLUSTER'.
I couldn't even ping my firewall (FortiGate).
After about 3 minutes it comes back online.

When I did the same with internal IP addresses everything's ok.

Am I doing anything wrong? I want to do restores and migrations with public IPs.

 

[manu]

Hi Timon,

>> My solution:
- remove 'Network interface' from VM after migration
- remove NIC from /etc/udev/rules.d/70-persistent-rules.net
- reboot
- add new 'Network Interface' (with random MAC)
>>>> - since /etc/sysconfig/network-scripts/ifcfg-eth0 already exists within VM nothing else is needed

I guess the problem here is that the upstream gateway that your VMs are trying to reach is only accessible on the host where they were configured. VMs need to be in the same subnetwork and have the same upstream gateway if you want this to work. Ask your hoster abour this, for instance according to this post https://forum.proxmox.com/threads/cluster-gateway-2-nodes.29598 OVH has a feature called vrack to do that.

>>> And when I tried to configure 3 nodes in cluster (fresh installs) with public IP's on eth0, whole network went down during 'pvecm add IP-ADDRESS-CLUSTER'.
I couldn't even ping my firewall (FortiGate).
After about 3 minutes it comes back online.

NB: PVE clusters are meant to be run in LANs, where the network latency between the servers as measured by ping should be under 2ms.

 

[TimonTNT]

Thanks for the reply.
I designed whole network and public & internal network is <1 ms.
I found a "solution".
The problem is that I am using FortiGate firewall.
When I connect servers directly to Forti the problem exists.
When I connect servers to simple switch and switch to Forti there is no problem.
So now I have two spare ports in Fortigate but I have to use additional switch.
Why is it working ..... I have to idea.