Add several public IP addresses to a working MASQUERADING setup?

ah sorry, of course this cannot work since everything is going via the gateway which is sent via eth0. I'm afraid you will have to use a VRF, at least I don't see any other way.

Could you maybe explain why this is impossible? I don't understand the underlying cause. It's no problem if different hosts from the same subnet have the same gateway. Why does this cause problems if the interfaces reside on the same machine?
 
With one routing table you can only route the traffic to the gateway via one interface - either eth0 or eth1. If you configure the IPs on the same interface, then that would be no issue, but then you share the same MAC address (which isn't possible in your case). If you wanna route one part of destination IPs via one interface and another part via the other that also wouldn't be a problem (but you want to route traffic to all destinations). The crucial part here is with the different MAC addresses.

I thought maybe macvlan [1] interfaces might be a solution here, but they require the usage of network namespace. So I think the only way to achieve what you want to do is by using multiple routing tables.

[1] https://developers.redhat.com/blog/...nux-interfaces-for-virtual-networking#macvlan
 
I think something like this could work, but I haven't had the time to try it - i might have missed something while thinking about it. I've used 10.0.0.0/24 as an example but of course you would have to substitute your subnet / gateway.

10.0.0.1 is eth1
10.0.0.2 is eth2
10.0.0.100 is gw

Code:
ip link add rt1 type vrf table 10
ip link add rt2 type vrf table 20

ip link set dev eth1 master rt1
ip link set dev eth2 master rt2

ip link set up rt1
ip link set up rt2

ip route add 10.0.0.100/32 dev eth1 src 10.00.0.1 table rt1
ip route add default via 10.0.0.100 dev eth1 table rt1

ip route add 10.0.0.100/32 dev eth2 src 10.0.0.2 table rt2
ip route add default via 10.0.0.100 dev eth2 table rt2

ip rule add from 10.0.0.1/32 table rt1
ip rule add to 10.0.0.1/32 table rt1

ip rule add from 10.0.0.2/32 table rt2
ip rule add to 10.0.0.2/32 table rt2
 
  • Like
Reactions: wowo
Ohh no, now I feel bad about planting my problems into your head on a friday night!

Thank you very much for the draft, I will try it out.
 
Okay, it works (but new problems showed up).

I did this:

Code:
#!/bin/bash

ip link add rt0 type vrf table 10
ip link add rt1 type vrf table 20

ip link set dev eth0 master rt0
ip link set dev eth1 master rt1

ip link set up rt0
ip link set up rt1

ip route add 37.x.x.65/32 dev eth0 src 37.x.x.84 table 10
ip route add default via 37.x.x.65 dev eth0 table 10
ip rule add from 37.x.x.84/32 table 10
ip rule add to 37.x.x.84/32 table 10

ip route add 37.x.x.65/32 dev eth1 src 37.x.x.85 table 20
ip route add default via 37.x.x.65 dev eth1 table 20
ip rule add from 37.x.x.85/32 table 20
ip rule add to 37.x.x.85/32 table 20

Now I have devs rt0 and rt1.
The problem is the system does not use them.

I can use tools where I can specify the interface.
For example

Code:
ping -I rt1 8.8.8.8

Sniffing on the host now shows the right MAC for the floating IP.

But the whole system is unsable because all standard tools without the possibility to specify an interface to bind to don't use rt0/rt1.

When I do a

Code:
ip route add default dev rt0

ICMP starts working (TCP doesn't), but I also fear I might skip the virtual routing tables for rt1 when I do this?

I found this post that explains how to start daemons using the virtual routing tables:
https://access.redhat.com/documenta...f-network_configuring-and-managing-networking

I tried with sshd, but it did not work.

Code:
/usr/sbin/ip vrf exec rt0 /usr/sbin/sshd -D
Failed to load BPF prog: 'Operation not permitted'

So status is now:

- floating IPs are available with the correct MACs. I can ping them from the outside.
- system tools have no internet connectivity, because they don't seem to use devices rt0/rt1
- running daemons with /usr/sbin/ip vrf exec fails with error

Getting closer...
 
Last edited:
- system tools have no internet connectivity, because they don't seem to use devices rt0/rt1

Yes, you would still have to configure the default routing table and make it use whatever you want it to be using as default. Afaik you only need the second IP for the STUN server and hopefully you should be able to configure it to properly use the other rt - I have no experience with that so I cannot help you there.

You could even drop one routing table and just configure it as default and then only configure one extra that uses eth1 and use that specifically where you want to use the other public IP.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!