VMs with public IP for outgoging traffic and local network on routed configuration

[Buschdieb]

Good evening, I have been trying to create a specific network configuration for several days but keep encountering difficulties. Therefore, I would like to take this opportunity to ask the community for some input and assistance. I have a Hetzner server running Proxmox version 8.2.7. My previous use case worked with a single public IP and an internal network (192.168.1.0/24). Several VMs provided services through an Nginx proxy. This worked wonderfully for years with the routed configuration and masquerading. See also this wiki entry on the topic. https://pve.proxmox.com/wiki/Network_Configuration

auto lo
iface lo inet loopback

iface lo inet6 loopback

iface enp0s31f6 inet manual
        up route add -net 122.150.130.0 netmask 255.255.255.192 gw 122.150.130.1 dev enp0s31f6

auto vmbr0
iface vmbr0 inet static
        address  122.150.130.22
        netmask  255.255.255.192
        gateway  122.150.130.1
        bridge-ports enp0s31f6
        bridge-stp off
        bridge-fd 0
# route 122.150.130.0/26 via 122.150.130.1

iface vmbr0 inet6 static
        address  2a02:5a8:231:25c2::2
        netmask  64
        gateway  pe90::1

auto vmbr1
iface vmbr1 inet static
        address  192.168.1.1
        netmask  255.255.255.0
        bridge-ports none
        bridge-stp off
        bridge-fd 0
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up iptables -t nat -A POSTROUTING -s '192.168.1.0/24' -o vmbr0 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '192.168.1.0/24' -o vmbr0 -j MASQUERADE

Now new requirements are being added. I need several VMs, each making outgoing connections with their own public IP. Incoming traffic does not need to be handled. However, these VMs must also be able to access the internal local network (192.168.1.0/24).

So far, I haven't been able to allow the new VMs to access the internet via an additional public IP. Either there is no internet traffic at all, or it goes through the main IP of the server to the internet.

What do you think? Should the approach be to add the additional IPs to vmbr0, or should I create a separate bridge along with an additional public IP? But then, how would I access my local network?

Thank you in advance for the assistance.

 

[ScottZ.]

I THINK (don't hold me to it) this is your problem

up route add -net 122.150.130.0 netmask 255.255.255.192 gw 122.150.130.1 dev enp0s31f6

Your default router is NOT going to the interface, it's going to the VMBR0 interface. Try moving the route command over to the static config of vmbr0 and see if that works.

 

[Buschdieb]

Hello, thank you for your comment. Here is the working configuration: several VMs are using a shared public IP, while others have their own individual public IP. Each VM is accessible from the internet and has its own local IP within the network.

Proxmox Host

auto lo
iface lo inet loopback

iface lo inet6 loopback

iface enp0s31f6 inet manual

auto vmbr0
iface vmbr0 inet static
        address  122.150.130.22
        netmask  255.255.255.192
        gateway  122.150.130.1
        bridge-ports enp0s31f6
        bridge-stp off
        bridge-fd 0
        up route add -net 122.150.130.0 netmask 255.255.255.192 gw 122.150.130.1
# route 122.150.130.0/26 via 122.150.130.1

iface vmbr0 inet6 static
        address  2a02:5a8:231:25c2::2
        netmask  64
        gateway  pe90::1

auto vmbr1
iface vmbr1 inet static
        address  192.168.1.1
        netmask  255.255.255.0
        bridge-ports none
        bridge-stp off
        bridge-fd 0
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up iptables -t nat -A POSTROUTING -s '192.168.1.0/24' -o vmbr0 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '192.168.1.0/24' -o vmbr0 -j MASQUERADE
# localnetwork 192.168.1.0

VM1 Guest (using a shared public IP through a reverse proxy)

auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug ens18
iface ens18 inet static
        address 192.168.1.104/24
        gateway 192.168.1.1
        dns-nameservers 8.8.8.8
        dns-search google.com

VM2 Guest (using a own public IP)
In the Proxmox VM settings for this VM, the requested MAC address (11:11:11:11:11:11) from Hetzner must be entered in the network card (vmbr0).

auto lo
iface lo inet loopback

# Internal network interface (for local Proxmox network)
allow-hotplug ens18
iface ens18 inet static
    address 192.168.1.206/24
    dns-nameservers 8.8.8.8
    dns-search google.com

# Public network interface (for external communication)
allow-hotplug ens19
iface ens19 inet static
    address 122.150.130.47
    netmask 255.255.255.192
    gateway 122.150.130.1

Now to a new challenge. My public IP range is now exhausted, and Hetzner has assigned me an additional subnet. However, I cannot request MAC addresses for this subnet. I'm sure the experts here have already done something like this. How can I assign a public IP for additional VMs, such as VM2? I previously managed to get a configuration working but then received an abuse notification. Can I create a vmbr2 on the Proxmox host?

New Subnet

IP: 133.250.101.120 / 29
Gateway: 122.150.130.22
Netmask: 255.255.255.248
Broadcast: 133.250.101.127

Public IP 1: 133.250.101.121 
Public IP 2: 133.250.101.122
Public IP 3: 133.250.101.123
Public IP 4: 133.250.101.124
Public IP 5: 133.250.101.125
Public IP 6: 133.250.101.126

All IPs have been modified from the original. How should the configuration look on the host and on the client VM3, and which MAC address should I enter in Proxmox for VM3?

Thank you for the support.