[SOLVED] Hetzner Proxmox routed config

[Keyinator]

Hello,

I am really new to proxmox and routing.
I want to setup two bridges vmbr0 and vmbr1.
vmbr0 should allow my networks to connect with the internet in a routed config (I only have one public ip-adress and need to have a static mac-adress)
vmbr1 should be an internal network so vms can talk with eachother (I would also like to ssh into those vms from the proxmox node).

My current setup is the following:
I have masked the first three blocks with <Main IP> but have left the last block so errors can be spotted more easily.
<Gateway> is <Main IP>.1

auto lo
iface lo inet loopback
#https://pve.proxmox.com/pve-docs/images/default-network-setup-routed.svg

auto enp9s0
iface enp9s0 inet static
        address  <Main IP>.16/26
        gateway  <Gateway>
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up echo 1 > /proc/sys/net/ipv4/conf/enp9s0/proxy_arp
        up route add -net <Main IP>.0 netmask 255.255.255.192 gw <Gateway> dev enp9s0

iface enp9s0 inet6 static
        address  <Main IPv6>::2
        netmask  128
        gateway  fe80::1


auto vmbr0 #Internet access
iface vmbr0 inet static
        address  192.168.0.1/26
        pointopoint <Gateway>
        gateway <Gateway>
        bridge-ports none #Setting this to enp9s0 seems to cause the whole network to become unavailable
        bridge-stp off
        bridge-fd 1
        bridge_maxage 12

auto vmbr1 #Private network for vms to communicate
iface vmbr1 inet manual
        bridge-ports none
        bridge-stp off
        bridge-fd 0

However I don't know if that's correct and what to implement on the vm's network config.
Can anyone help me with that?

 

[Richard]

auto enp9s0
iface enp9s0 inet static
address <Main IP>.16/26
gateway <Gateway>
post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up echo 1 > /proc/sys/net/ipv4/conf/enp9s0/proxy_arp
up route add -net <Main IP>.0 netmask 255.255.255.192 gw <Gateway> dev enp9s0

I don't think proxy_arp is useful here.

The "up route" line is not necessary (all is routed to <Gateway> by the "gateway" entry)

auto vmbr0 #Internet access
iface vmbr0 inet static
address 192.168.0.1/26
pointopoint <Gateway>
gateway <Gateway>
bridge-ports none #Setting this to enp9s0 seems to cause the whole network to become unavailable
bridge-stp off
bridge-fd 1
bridge_maxage 12

auto vmbr1 #Private network for vms to communicate
iface vmbr1 inet manual
bridge-ports none
bridge-stp off
bridge-fd 0[/CODE]

However I don't know if that's correct and what to implement on the vm's network config.
Can anyone help me with that?

Looks ok, but you need to add NAT for outgoing connection from VMs as follows:

iptables -t nat -A POSTROUTING -o enp9s0 -j MASQUERADE

Incoming connections (from public network) to VMs will be possible only with port forwarding.

 

[Keyinator]

I don't think proxy_arp is useful here.

The "up route" line is not necessary (all is routed to <Gateway> by the "gateway" entry)



Looks ok, but you need to add NAT for outgoing connection from VMs as follows:

iptables -t nat -A POSTROUTING -o enp9s0 -j MASQUERADE

Incoming connections (from public network) to VMs will be possible only with port forwarding.

Thank you very much for your help. I was able to solve the problem (however with 2 ip-adresses as it is way easier). I will share the config which works with hetzner dedicated servers where mac must be masked.

The following code is an example where you have your first ip for management and second ip for public usage (http and https in this case).
Just reuse the last line to fit your needs

auto enp9s0
iface enp9s0 inet static
  address  <IP 1>/26
  gateway  <Gateway 1>

iface enp9s0 inet6 static #THIS IS NOT NEEDED FOR IPV4 ONLY USSAGE
  address  <IPv6 1>
  netmask  128
  gateway  <IPv6 Gateway>

auto vmbr0
iface vmbr0 inet static
  address  10.0.0.1/24
  bridge-ports none
  bridge-stp off
  bridge-fd 0
  post-up   echo 1 > /proc/sys/net/ipv4/ip_forward
  post-up   iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o enp9s0 -j MASQUERADE
  post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o enp9s0 -j MASQUERADE
  up ip route add <IP 2>/32 dev vmbr0

  #Port forward port 80 and 443 to vm and back from and to ip 2
  post-up   iptables -t nat -A PREROUTING  -p tcp -j DNAT          -d <IP 2> --dport 80  --to-destination 10.0.0.2:80
  post-up   iptables -t nat -A PREROUTING  -p tcp -j DNAT          -d <IP 2> --dport 443 --to-destination 10.0.0.2:443

I also had problems with setting up the vm's ethernet as there is obviously no dhcp out of the box.
So here is an example for ubuntu:
(You enter these values when the vm boots up for the first time using the console tab on the proxmox gui)
(Keep in mind that in above example on the last two lines our vm receives ip 10.0.0.2)

• Set ipv4 to manual
• Enter the following values:
  • Subnet: 10.0.0.255/24 (for my example)
  • Address: 10.0.0.2
  • Gateway: 10.0.0.1
  • Nameservers: 8.8.8.8, 1.1.1.1 (Choose whatever you like.)
  • Search domains:

 

[danfreudend]

Thank you very much for your help. I was able to solve the problem (however with 2 ip-adresses as it is way easier). I will share the config which works with hetzner dedicated servers where mac must be masked.

The following code is an example where you have your first ip for management and second ip for public usage (http and https in this case).
Just reuse the last line to fit your needs

auto enp9s0
iface enp9s0 inet static
  address  <IP 1>/26
  gateway  <Gateway 1>

iface enp9s0 inet6 static #THIS IS NOT NEEDED FOR IPV4 ONLY USSAGE
  address  <IPv6 1>
  netmask  128
  gateway  <IPv6 Gateway>

auto vmbr0
iface vmbr0 inet static
  address  10.0.0.1/24
  bridge-ports none
  bridge-stp off
  bridge-fd 0
  post-up   echo 1 > /proc/sys/net/ipv4/ip_forward
  post-up   iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o enp9s0 -j MASQUERADE
  post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o enp9s0 -j MASQUERADE
  up ip route add <IP 2>/32 dev vmbr0

  #Port forward port 80 and 443 to vm and back from and to ip 2
  post-up   iptables -t nat -A PREROUTING  -p tcp -j DNAT          -d <IP 2> --dport 80  --to-destination 10.0.0.2:80
  post-up   iptables -t nat -A PREROUTING  -p tcp -j DNAT          -d <IP 2> --dport 443 --to-destination 10.0.0.2:443

I also had problems with setting up the vm's ethernet as there is obviously no dhcp out of the box.
So here is an example for ubuntu:
(You enter these values when the vm boots up for the first time using the console tab on the proxmox gui)
(Keep in mind that in above example on the last two lines our vm receives ip 10.0.0.2)

• Set ipv4 to manual
• Enter the following values:
  • Subnet: 10.0.0.255/24 (for my example)
  • Address: 10.0.0.2
  • Gateway: 10.0.0.1
  • Nameservers: 8.8.8.8, 1.1.1.1 (Choose whatever you like.)
  • Search domains:

This is very helpful. but I can't figure out which range ip 2 should have? Should it be in the same range as ip 1 or in the 10.0.0.- range?