Share my network configuration on servers hosted by Hetzner

T

Tony

Renowned Member
Nov 4, 2010
113
10
83
Hi,

many of us run Proxmox VE on servers hosted by Hetzner. The network configuration is usually the most tricky part. I am going to post my network configuration here (real IPs changed). I hope it could help people in a similar situation. I also hope to get some feedback and constructive criticism.

If you also run PromoxVE at Hetzner, I am very interested in learning from your network config.

I always install Promox VE as follows:
- order server without OS, server will be in rescue mode when ready
- run stress-testing for 24h
- install debian minimal
- install pve following this Install Proxmox VE on Debian Buster - Proxmox VE

content of /etc/network/interfaces on pve host:
Code: 
auto lo
iface lo inet loopback

auto enp41s0
iface enp41s0 inet static
  address X.Y.Z.150
  netmask 255.255.255.192
  gateway X.Y.Z.129
  up route add -net X.Y.Z.128 netmask 255.255.255.192 gw X.Y.Z.129 dev enp41s0

# switch for extra public IPs: can be additional IPs of a server, or failover IPs
# ref: https://blog.no-panic.at/2016/08/09/proxmox-on-debian-at-hetzner-with-multiple-ip-addresses/
auto vmbr0
iface vmbr0 inet static
  address 1.2.3.4
  netmask 255.255.255.255
  bridge_ports none
  bridge_stp off
  bridge_fd 0
  bridge_maxwait 0
  pre-up brctl addbr vmbr0
  # single extra IP:
  up ip route add 1.2.3.5/32 dev vmbr0

# switch connected to a vSwitch with a public subnet
# ref: https://www.razva.ro/how-to-setup-proxmox-with-hetzner-vswitch-and-public-ip/
iface enp41s0.4000 inet manual
auto vmbr4000
iface vmbr4000 inet manual
# net4000 public
  bridge_ports enp41s0.4000
  bridge_stp off
  bridge_fd 0
  mtu 1400


# switch connected to a vSwitch with a private subnet
iface enp41s0.4001 inet manual
auto vmbr4001
iface vmbr4001 inet manual
# net4001 private
  bridge_ports enp41s0.4001
  bridge_stp off
  bridge_fd 0
  mtu 1400
  up ip addr add 192.168.30.11/24 dev vmbr4001

- content of /etc/network/interfaces of a VM connected to vmbr0:
Code: 
auto lo
iface lo inet loopback

auto ens18
iface ens18 inet static
  address 1.2.3.5
  netmask 255.255.255.255
  post-up ip route add 1.2.3.4 dev ens18
  post-up ip route add default via 1.2.3.4 dev ens18

- content of /etc/network/interfaces of a VM connected to vmbr4000 (public vSwitch):
Code: 
auto lo
iface lo inet loopback


auto ens18
iface ens18 inet static
  address X.Y.Z.37
  netmask 255.255.255.224
  gateway X.Y.Z.33
  mtu 1400

- content of /etc/network/interfaces of a VM connected to vmbr4001 (private vSwitch):
Code: 
auto lo
iface lo inet loopback


auto ens18
iface ens18 inet static
  address 192.168.30.121
  netmask 255.255.255.0
  gateway 192.168.30.1
  mtu 1400
 
Hi Tony,

I have couple of servers in cluster and my own IP addresses segment. I am using very simple bridged network configuration.
When I try to use same conf on server rented from Hetzner with additional IP, virtual host was not able to communicate with outside world.

Node network conf:

Code: 
auto lo
iface lo inet loopback

iface eth0 inet manual

auto vmbr0
iface vmbr0 inet static
        address 194.x.x.50/24
        gateway 194.x.x.1
        bridge-ports eth0
        bridge-stp off
        bridge-fd 0

Virtual host conf.
Code: 
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
        address 194.x.x.60/24
        gateway 194.x.x.1

Could you advice me what is the problem?
Thank you in advance
 
Hey Matus,
the difference is that Hetzner uses a Routed Network setup with MAC Binding for the IPs
That mean you have to route your additional IPs via the Hosts IP.
This Setup is a bit more complex but offers a lot of great features for You as a Customer and also for Hetzner

examples are
- HA IPs between Locations
- Migration of IPs to another Host
- Protect customers against each other, for example if someone configures a DHCP Server on the Interface eg.
- Saving IPs
 
Matus said:
Hi Tony,

I have couple of servers in cluster and my own IP addresses segment. I am using very simple bridged network configuration.
When I try to use same conf on server rented from Hetzner with additional IP, virtual host was not able to communicate with outside world.

Node network conf:

Code: 
auto lo
iface lo inet loopback

iface eth0 inet manual

auto vmbr0
iface vmbr0 inet static
        address 194.x.x.50/24
        gateway 194.x.x.1
        bridge-ports eth0
        bridge-stp off
        bridge-fd 0

Virtual host conf.
Code: 
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
        address 194.x.x.60/24
        gateway 194.x.x.1

Could you advice me what is the problem?
Thank you in advance
Click to expand...
you can look at my config, bridge vmbr0 and the corresponding. It's a working config with IPs changed. Some points:

- ensure you have ip forwarding enabled
- the address of vmbr0 is the same as on main NIC
- for more info read this: https://blog.no-panic.at/2016/08/09/proxmox-on-debian-at-hetzner-with-multiple-ip-addresses/
 
hi Tony,

I used the configuration exactly as above (using my own IP of course) on the host node and guest respectively. But only the guest assigned the additional single iP address could connect to the internet. The guest assigned a vswitch subnet IP could only ping it's own IP and the private IP, but couldn't connect to the internet. Do you know if anything has changed in hetzner's setup?

Regards.
 
virtmin said:
hi Tony,

I used the configuration exactly as above (using my own IP of course) on the host node and guest respectively. But only the guest assigned the additional single iP address could connect to the internet. The guest assigned a vswitch subnet IP could only ping it's own IP and the private IP, but couldn't connect to the internet. Do you know if anything has changed in hetzner's setup?

Regards.
Click to expand...
yes I used to experience the problem with vSwitch that guests cannot talk to the Internet. Things that I tried:
- ask for Hetzner support
- disconnect the server from vSwitch and reconnect (advised by Hetzner support)
- keep pinging the gateway
- check the MTU 1400 everywhere relevant
 
Tony said:
yes I used to experience the problem with vSwitch that guests cannot talk to the Internet. Things that I tried:
- ask for Hetzner support
- disconnect the server from vSwitch and reconnect (advised by Hetzner support)
- keep pinging the gateway
- check the MTU 1400 everywhere relevant
Click to expand...

Okay, so I went to the vSwitch to delete and readd as you've adviced, and behold, my node wasn't in the vswitch 4000 (public) group. I must have accidentally removed it instead when I tried to remove my windows node from the group. I readded it and viola! my guest is communicating with the internet for the first time in 72 straight hours.

Thank you for your suggestion and for the configuration you posted. You've just ended my hours of struggle.

Regards.
 
Hi,

Can someone please advice how to enable IPv6 for my guests using a /27 public subnet via vswitch on hetzner.

IPv4 is working alright on all guest.

Edit: I only have the /64 that is assigned to the main server.

My current interfaces configuration is below:

Code: 
auto lo
iface lo inet loopback
iface lo inet6 loopback

auto enp4s0
iface enp4s0 inet static
        address 46.xx.xx.9
        netmask 255.255.255.192
        gateway 46.xx.xx.1
        up route add -net 46.xx.xx.0 netmask 255.255.255.192 gw 46.xx.xx.1 dev enp4s0

iface enp4s0 inet6 static
        address 2a01:xxx:xxx:xxx::2
        netmask 64
        gateway fe80::1

#start single additional IPv4 address

auto vmbr0
iface vmbr0 inet static
        address 46.xx.xx.9
        netmask 255.255.255.255
        bridge_ports none
        bridge_stp off
        bridge_fd 0
        bridge_maxwait 0
        pre-up brctl addbr vmbr0
        up ip route add 46.xx.xx.56/32 dev vmbr0

iface vmbr0 inet6 static
        address 2a01:xxx:xxx:xxx::3
        netmask 64
        up ip -6 route add 2a01:xxx:xxx:xxx::/64 dev vmbr0

#end single additional IPv4 address

#start vswitch IPv4 /27 public subnet

auto enp4s0.4000
iface enp4s0.4000 inet static
        address 116.xx.xx.1
        netmask 255.255.255.224
        vlan-raw-device enp4s0
        mtu 1400

auto vmbr4000
iface vmbr4000 inet manual
        address 116.xx.xx.1
        netmask 255.255.255.224
        bridge_ports enp4s0.4000
        bridge_stp off
        bridge_fd 0
        mtu 1400

#end vswitch IPv4 /27 public subnet

#start vswitch IPv4 private IP

iface enp4s0.4001 inet manual
auto vmbr4001
iface vmbr4001 inet manual
  bridge_ports enp4s0.4001
  bridge_stp off
  bridge_fd 0
  mtu 1400
  up ip addr add 192.168.100.1/24 dev vmbr4001

#end vswitch IPv4 private IP
 
Last edited:
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom