Split network using 2 failover IPs

K

kamzata

Renowned Member
Proxmox Subscriber
Jan 21, 2011
219
9
83
Italy
At the moment, I'm using Proxmox with a single public IP which routes all traffic among my containers. Now I'd like to "split" the network in 2 in order to separate some containers. I thought it could be achieved by using 2 failover IPs and using 2 differents nat rules in the similar way I'm already doing. Something like:

Code: 
-A POSTROUTING -s 192.168.1.0/24 -o vmbr0 -j SNAT --to-source my_first_failover_ip
-A POSTROUTING -s 192.168.2.0/24 -o vmbr1 -j SNAT --to-source my_second_failover_ip

How can I set these 2 failover IPs on Proxmox?
 
How I did it. In proxmox I registered the necessary ip and masks for eth0 (public) and eth1 (private). After that, between the containers of the nodes, the traffic itself ran through eth0. I didn’t do veth containers, I did venet, I assigned it the necessary IP (10.0.0.x), after which the container could normally communicate with other containers and with nodes. For the fact that the container could roam in public through eth1, configured NAT on the nodes. After that, it was a complete breakthrough :) Access from public to containers or by overrunning IP, or forwarding.
 
Thanks but that is not what I would like to do. I'd like to do something similar but I cannot create the second bridge to enp1s0f0 on vmbr2. Is there a way to bypass this issue? enp1s0f0 is the public network interface while enp1s0f1 is the private.
proxmox_failover.png
 
I would recommend to configure both IPs (which are in the same network) on the same interface (vmbr0).
Having 2 Interfaces with IPs from the same network usually leads to problems confusions (since the host will only take the first route to the gateway (unless you configure 2 routing-tables).

You should be able to have both SNAT rules going out of the same interface

Hope this helps
 
Stoiko Ivanov said:
I would recommend to configure both IPs (which are in the same network) on the same interface (vmbr0).
Having 2 Interfaces with IPs from the same network usually leads to problems confusions (since the host will only take the first route to the gateway (unless you configure 2 routing-tables).

You should be able to have both SNAT rules going out of the same interface

Hope this helps
Click to expand...

How can I configure both IPs on the same interface?
 
Here's my /etc/network/interface file. I'm trying to set up it in this way:

Code: 
auto lo
iface lo inet loopback

auto enp1s0f1
iface enp1s0f1 inet dhcp

auto enp1s0f0
iface enp1s0f0 inet static
        address  163.172.20.100 #My_real_IP
        netmask  255.255.255.0
        gateway  163.172.20.1

auto enp1s0f0:0
iface enp1s0f0:0 inet static
        address  62.212.150.20 #My_1_failover_IP
        netmask  255.255.255.0
        gateway  62.212.150.1

auto enp1s0f0:1
iface enp1s0f0:1 inet static
        address  62.212.150.21 #My_2_failover_IP
        netmask  255.255.255.0
        gateway  62.212.150.1

auto vmbr0
iface vmbr0 inet static
        address  192.168.1.1
        netmask  255.255.255.0
        bridge-ports enp1s0f0:0
        bridge-stp off
        bridge-fd 0

auto vmbr1
iface vmbr1 inet static
        address  192.168.2.1
        netmask  255.255.255.0
        bridge-ports enp1s0f0:1
        bridge-stp off
        bridge-fd 0

    post-up echo 1 > /proc/sys/net/ipv4/ip_forward

But it doesn't boot. It boots only if I remove vmbr*. I guess I can't bridge alias like enp1s0f0:0 or enp1s0f0:1 in vmbr0 and vmbr1, can I? Any ideas?
 
Don't configure IPs on ethernet-interfaces which are bridge-ports.
configure the 3 aliases on enp1s0f0, enp1s0f0:0, enp1s0f0:1, and set bridge-ports to none for vmbr0 vmbr1

Afterwards you should be able to do the SNAT via iptables rules (you can also add them in post-up commands for vmbr0, vmbr1

hope this helps!
 
Stoiko Ivanov said:
Don't configure IPs on ethernet-interfaces which are bridge-ports.
configure the 3 aliases on enp1s0f0, enp1s0f0:0, enp1s0f0:1, and set bridge-ports to none for vmbr0 vmbr1

Afterwards you should be able to do the SNAT via iptables rules (you can also add them in post-up commands for vmbr0, vmbr1

hope this helps!
Click to expand...

Thanks for your reply, but I didn't get you. How can it work if there's no ip on ethernet-interface configured and bridge-ports set to none for vmbr*? What do you mean exactly?
 
Sorry - the 2 statements were independent - basically:
configure the 3 aliases on enp1s0f0, enp1s0f0:0, enp1s0f0:1, and set bridge-ports to none for vmbr0 vmbr1

was what I meant for your case

(the first statement was a general tip)
 
Stoiko Ivanov said:
Sorry - the 2 statements were independent - basically:
configure the 3 aliases on enp1s0f0, enp1s0f0:0, enp1s0f0:1, and set bridge-ports to none for vmbr0 vmbr1

was what I meant for your case

(the first statement was a general tip)
Click to expand...

Sorry but I still don't get you. How can I route all packets to vmbr0 or vmbr1 if no bridge-port is set?

Do you mean something like this?

Code: 
auto lo
iface lo inet loopback

auto enp1s0f1
iface enp1s0f1 inet manual

auto enp1s0f0
iface enp1s0f0 inet manual

auto enp1s0f0:0
iface enp1s0f0:0 inet manual

auto enp1s0f0:1
iface enp1s0f0:1 inet manual

auto vmbr0
iface vmbr0 inet static
        address  62.212.150.20 #My_1_failover_IP
        netmask  255.255.255.0
        gateway  62.212.150.1
        bridge-ports none
        bridge-stp off
        bridge-fd 0

auto vmbr1
iface vmbr1 inet static
        address  62.212.150.21 #My_2_failover_IP
        netmask  255.255.255.0
        gateway  62.212.150.1
        bridge-ports none
        bridge-stp off
        bridge-fd 0

auto vmbr2
iface vmbr2 inet static
        address  192.168.1.1
        netmask  255.255.255.0
        bridge-ports vmbr0
        bridge-stp off
        bridge-fd 0

auto vmbr3
iface vmbr3 inet static
        address  192.168.2.1
        netmask  255.255.255.0
        bridge-ports vmbr1
        bridge-stp off
        bridge-fd 0

    post-up echo 1 > /proc/sys/net/ipv4/ip_forward
 
hi,
sorry - meant something like:
Code: 
auto enp1s0f1
iface enp1s0f1 inet manual

auto enp1s0f0
iface enp1s0f0 inet static

auto enp1s0f0:0
iface enp1s0f0:0 inet manual
       address  62.212.150.20 #My_1_failover_IP
       netmask  255.255.255.0
       gateway  62.212.150.1

auto enp1s0f0:1
iface enp1s0f0:1 inet manual
       address  62.212.150.21 #My_2_failover_IP
       netmask  255.255.255.0 #NOT SURE but maybe the netmask should be 255.255.255.255

auto vmbr0
iface vmbr0 inet static
       address 192.168.1.1
       netmask  255.255.255.0
       bridge-ports none
       bridge-stp off
       bridge-fd 0

auto vmbr1
iface vmbr1 inet static
       address 192.168.2.1
       netmask  255.255.255.0
       bridge-ports none
       bridge-stp off
       bridge-fd 0

and then the fitting SNAT rules

Note - I haven't tested this config - but I hope it provides a starting point!
hope this helps!
 
Stoiko Ivanov said:
hi,
sorry - meant something like:
Code: 
auto enp1s0f1
iface enp1s0f1 inet manual

auto enp1s0f0
iface enp1s0f0 inet static

auto enp1s0f0:0
iface enp1s0f0:0 inet manual
       address  62.212.150.20 #My_1_failover_IP
       netmask  255.255.255.0
       gateway  62.212.150.1

auto enp1s0f0:1
iface enp1s0f0:1 inet manual
       address  62.212.150.21 #My_2_failover_IP
       netmask  255.255.255.0 #NOT SURE but maybe the netmask should be 255.255.255.255

auto vmbr0
iface vmbr0 inet static
       address 192.168.1.1
       netmask  255.255.255.0
       bridge-ports none
       bridge-stp off
       bridge-fd 0

auto vmbr1
iface vmbr1 inet static
       address 192.168.2.1
       netmask  255.255.255.0
       bridge-ports none
       bridge-stp off
       bridge-fd 0

and then the fitting SNAT rules

Note - I haven't tested this config - but I hope it provides a starting point!
hope this helps!
Click to expand...

Now I got you! Yeah, that helped a lot! Thanks! Now it works! Could I ask you why it shouldn't be set IP address directly on the enp1s0f0 interface?
 
kamzata said:
Could I ask you why it shouldn't be set IP address directly on the enp1s0f0 interface?
Click to expand...
You definitely can - the example interfaces I posted was focused on getting the misunderstanding of aliases and bridge_ports out of the way.

From the initial interfaces you posted it seems like your 3 IPs (real IP, failover IP 1, failover IP 2) are from 2 ranges:
* 163.x.y.z/24
* 61.u.v.w/24

Since a system can (in general, not considering setting up multiple routing tables) only have 1 default gatway the setup in that case depends on the configuration in the network - In the simples case you setup all 3 IPs and set the gatway on only one IP
e.g.:
Code: 
iface enp1s0f0 inet static
       address  163.172.20.100 #My_real_IP
       netmask  255.255.255.0
       gateway  163.172.20.1
iface enp1s0f0:0 inet static
      address  62.212.150.20 #My_1_failover_IP
      netmask  255.255.255.0
iface enp1s0f0:1 inet static
      address  62.212.150.21 #My_2_failover_IP
      netmask  255.255.255.0

then the packet should get send with the appropriate IP as source address, but to the gateway 163.172.20.1 (works in most datacenters, however certain providers (e.g. hetzner, ovh) do have a slightly different setup, and there it depends on their settings and recommendations

hope this helps!
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom