Proxmox 4 Hetzner extra subnet configuration

asmar · Dec 6, 2015

Hi all,

I've read all relevant post about Hetzner & Proxmox network configuration but doesn't seem to work out on Proxmox 4 or I'm doing something wrong.

My interfaces file is (just changed the number of IPs):

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address 276.9.58.231
netmask 255.255.255.224
gateway 276.9.58.225
up route add -net 276.9.58.224 netmask 255.255.255.224 gw 276.9.58.225 eth0
# default route to access subnet

auto vmbr0
iface vmbr0 inet static
address 276.9.249.145
netmask 255.255.255.248
bridge_ports none
bridge_stp off
bridge_fd 0

My subnet details are:

Subnet: 276.9.249.144 /29
Netmask: 255.255.255.248
Broadcast: 276.9.249.151
Usable IP addresses:

276.9.249.145 to 276.9.249.150

I've tried various configs but no luck, I'm able to create the VM either OpenVZ or KVM but VMs don't have access on Internet.

Any help is much appreciated.

monkfish · Dec 13, 2015

Hello, new user here with various machines at Hetzner, this is what works for me.

I wanted to use separate bridges for each available ip address on my subnet so I ended up with lots of bridge interfaces.
You may want something different, eg 2 ip's on one. For me, I would recommend creating all your bridge interfaces in one go, then you only require one reboot.

So in order to cater for the [rather strange?] Hetzner routing you will need to use one of those addresses as a route to/from the internet for the rest. I used the highest available address. You probably want to move the static management ip off eth0 and assign to vmbr0. I don't appear to have any static route statements on my interfaces BUT I do have "net.ipv4.ip_forward=1" in sysctl.conf.

After building the bridges, you then assign the ip address inside the vm itself. For mine, I actually have pfsense firewalls on each of the bridges with a corresponding "private" bridge for the internal lan. So vmbr0 is main address with all subnet traffic routed via vmbr7. So to build a pfsense router I would have a WAN interface on vmbr7 and a LAN interface on vmbr1. Hope that makes sense. Here is what my config looks like; I've replaced with your addresses given.
Hope that helps

Monk
-----

Code:

[SIZE=2]auto lo
iface lo inet loopback

iface eth0 inet manual

auto vmbr0
iface vmbr0 inet static
        address  276.9.58.231
        netmask  255.255.255.224
        gateway  276.9.58.225
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0

auto vmbr1
iface vmbr1 inet manual
        bridge_ports none
        bridge_stp off
        bridge_fd 0

auto vmbr2
iface vmbr2 inet manual
        bridge_ports none
        bridge_stp off
        bridge_fd 0

auto vmbr3
iface vmbr3 inet manual
        bridge_ports none
        bridge_stp off
        bridge_fd 0

auto vmbr4
iface vmbr4 inet manual
        bridge_ports none
        bridge_stp off
        bridge_fd 0

auto vmbr5
iface vmbr5 inet manual
        bridge_ports none
        bridge_stp off
        bridge_fd 0

auto vmbr6
iface vmbr6 inet manual
        bridge_ports none
        bridge_stp off
        bridge_fd 0

auto vmbr7
iface vmbr7 inet static
        address  276.9.249.150
        netmask  255.255.255.248
        bridge_ports none
        bridge_stp off
        bridge_fd 0[/SIZE]

petrogazz · Dec 31, 2015

I confirm the setup of monkfish works with Promox4.1 and hetzner subnet, thanks for sharing!

monkfish · Dec 31, 2015

petrogazz said:
I confirm the setup of monkfish works with Promox4.1 and hetzner subnet, thanks for sharing!

Hey, you're welcome.
One thing that threw me was the firewalling inside Proxmox and several times I managed to lock myself out.

It was because I had set connections to drop without explicitly allowing traffic to the management address on tcp/22 or tcp/8006.

If you have problems and lock yourself out, boot using the Hetzner resuce interface, mount the partition with /etc on and add "pve-firewall stop" to rc.local then restart the server "normally". When it boots back into proxmox it will effectively stop the firewall and you can reconnect and modify accordingly. Be sure to remove that line after you're done!

I am now experimenting with traffic shaping rules on my virtual ip's.

One more thing I found, a little bit hit and miss, if you use pfsense firewalls with Proxmox, on the Hetzner hardware you may/may not have to disable hardware checksum offloading for the virtual nics. Symptoms of this include interfaces coming up but traffic not being passed, ICMP working but no TCP.

I very much like the service Hetzner provide, just wish they would allow cramming more memory into a machine to allow for funky virtualisation stuff.

Happy New Year

Monk

asmar · Dec 31, 2015

Hi all,

With 4 IPs (not different subnet) this is my working configuration:

auto eth0
iface eth0 inet static
address 192.168.1.210
netmask 255.255.255.192
gateway 192.168.1.193
up route add -net 192.168.1.192 netmask 255.255.255.192 gw 192.168.1.193 eth0
# default route to access subnet

iface eth0 inet6 static
address 2a01:4f8:171:1558::2
netmask 64
gateway fe80::1

auto vmbr0
iface vmbr0 inet static
address 192.168.1.210
netmask 255.255.255.192
bridge_ports none
bridge_stp off
bridge_fd 0
up ip route add 192.168.1.252/32 dev vmbr0
up ip route add 192.168.1.253/32 dev vmbr0
up ip route add 192.168.1.254/32 dev vmbr0
up ip route add 192.168.1.72/32 dev vmbr0

I've just changed the IPs with some fake for the post. The LXC vm's are working fine with the above setup.
If I however want to create a KVM then it doesn't, the VM can't access the Internet at all.
I tried a few combinations but no luck.
Can anyone post an example how I can make the 192.168.1.253 to work with KVM?

Many thanks

petrogazz · Jan 3, 2016

Hi asmar,

have you assigned a separate MAC from the Hetzner robot for the IP of the KVM?

My setup (without subnet) is the following (requires separate MAC for both LXC and KVM)

Code:

    auto lo

    iface lo inet loopback


    auto eth0

    iface eth0 inet manual


    auto  vmbr0

    iface vmbr0 inet static

      address  'MAIN IP'

      broadcast 'BROADCAST'

      netmask  'NETMASK'

      gateway  'GATEWAY IP'

      bridge_ports eth0

      bridge_stp off

      bridge_fd 0

      # default route to access subnet

      up route add -net X.X.X.X netmask 'NETMASK' gw 'GATEWAY' vmbr0


    iface vmbr0 inet6 static

      address x.x.x.x.x.x

      netmask 64

      gateway fe80::1

      up sysctl -p

You must use the seperate MAC given from the hetzner robot, to your proxmox network device config (LXC or KVM)

monkfish · Jan 4, 2016

Petrogazz is right - with separate ip's and not a subnet and attempting to use a bridged configuration, separate MACs must be used otherwise the Hetzner network drops the traffic.

I got some [possibly outdated?] info here when first setting mine up
http://wiki.hetzner.de/index.php/Proxmox_VE/en although the translation to English could do with some work!

asmar · Jan 4, 2016

The LXC templates don't have a separate MAC and they work fine, no network issues at all. I assume if I request a separate MAC I will need to update the network settings on the VM?

Regarding KVM, I tried with MAC but no luck. I tried with the real settings of the IP and also those that recommend on Hetzner wiki but no luck. I will try again later and post my network details here in case I'm missing something.

Thanks a lot for the replies.

asmar · Jan 4, 2016

The only difference in my setup on the host server is that I got "bridge_ports none" instead of "bridge_ports eth0"
I will try to change this and see if that works.
What settings do you have in your KVM VM?

petrogazz said:
Hi asmar,

have you assigned a separate MAC from the Hetzner robot for the IP of the KVM?

My setup (without subnet) is the following (requires separate MAC for both LXC and KVM)

Code:

auto lo iface lo inet loopback auto eth0 iface eth0 inet manual auto vmbr0 iface vmbr0 inet static address 'MAIN IP' broadcast 'BROADCAST' netmask 'NETMASK' gateway 'GATEWAY IP' bridge_ports eth0 bridge_stp off bridge_fd 0 # default route to access subnet up route add -net X.X.X.X netmask 'NETMASK' gw 'GATEWAY' vmbr0 iface vmbr0 inet6 static address x.x.x.x.x.x netmask 64 gateway fe80::1 up sysctl -p

You must use the seperate MAC given from the hetzner robot, to your proxmox network device config (LXC or KVM)

asmar · Jan 4, 2016

Further update, I've requested new MAC for all VMs, update their network MAC address through proxmox gui, added bridge_ports eth0 instead of bridge_ports none and after a reboot server doesn't boot at all.
Do I need to create a new bridge interface and change there the bridge_ports eth0?

Maus112 · Aug 9, 2017

Hello @ all

thank you for your answers.

@monkfish can you show me an example of your vm config?

Thank you

(Sorry for my bad english i hope you understand me)

Maus112

ErTnEc · Sep 13, 2017

Hi Monkfish

Very useful post thankyou, but I'm struggling a little at the moment...

I've a /28 subnet with Hetzner, and I've configured my interfaces similar to how you have, only real difference is vmbr1 is the last IP of the subnet range I have, and vmbr100 is the internal LAN interface:

Code:

auto lo
iface lo inet loopback

iface eth0 inet manual

auto vmbr0
iface vmbr0 inet static
        address  1.2.3.4
        netmask  255.255.255.224
        gateway  1.2.3.10
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0

auto vmbr1
iface vmbr1 inet static
        address 1.2.10.22
        netmask 255.255.255.240
        bridge_ports none
        bridge_stp off
        bridge_fd 0

auto vmbr100
iface vmbr100 inet static
        bridge_ports none
        bridge_stp off
        bridge_fd 0

Now I've already got various VM's running, and they're all fine, however I'm trying to get them all behind pfSense, and running into a little problem.

I've configured the pfSense WAN interface as follows:

IPv4 Address: 1.2.10.9
Subnet mask IPv4: 255.255.255.240
Gateway IPv4: 1.2.10.22

I can ping the pfSense instance from the internet just fine, however, routing traffic through to it fails (TCP for example) and it just times out. ipv4 forwarding is enabled on the Proxmox host though:

# sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1

Now you mention about Hetzners funky routing & needing to use one of the subnet addresses as a route to/from the internet. I've read a few other online posts & blog entries, and one site () mentions about requesing Hetzner reroute traffic from the original single IP (in my example 1.2.3.4) to one of the subnet IP's. Do I request it to be routed to the .22 address as defined on the Proxmox host? This is the only real part I'm unsure about.

Cheers

monkfish said:
Hello, new user here with various machines at Hetzner, this is what works for me.

I wanted to use separate bridges for each available ip address on my subnet so I ended up with lots of bridge interfaces.
You may want something different, eg 2 ip's on one. For me, I would recommend creating all your bridge interfaces in one go, then you only require one reboot.

So in order to cater for the [rather strange?] Hetzner routing you will need to use one of those addresses as a route to/from the internet for the rest. I used the highest available address. You probably want to move the static management ip off eth0 and assign to vmbr0. I don't appear to have any static route statements on my interfaces BUT I do have "net.ipv4.ip_forward=1" in sysctl.conf.

After building the bridges, you then assign the ip address inside the vm itself. For mine, I actually have pfsense firewalls on each of the bridges with a corresponding "private" bridge for the internal lan. So vmbr0 is main address with all subnet traffic routed via vmbr7. So to build a pfsense router I would have a WAN interface on vmbr7 and a LAN interface on vmbr1. Hope that makes sense. Here is what my config looks like; I've replaced with your addresses given.
Hope that helps

Monk
-----

Code:

[SIZE=2]auto lo iface lo inet loopback iface eth0 inet manual auto vmbr0 iface vmbr0 inet static address 276.9.58.231 netmask 255.255.255.224 gateway 276.9.58.225 bridge_ports eth0 bridge_stp off bridge_fd 0 auto vmbr1 iface vmbr1 inet manual bridge_ports none bridge_stp off bridge_fd 0 auto vmbr2 iface vmbr2 inet manual bridge_ports none bridge_stp off bridge_fd 0 auto vmbr3 iface vmbr3 inet manual bridge_ports none bridge_stp off bridge_fd 0 auto vmbr4 iface vmbr4 inet manual bridge_ports none bridge_stp off bridge_fd 0 auto vmbr5 iface vmbr5 inet manual bridge_ports none bridge_stp off bridge_fd 0 auto vmbr6 iface vmbr6 inet manual bridge_ports none bridge_stp off bridge_fd 0 auto vmbr7 iface vmbr7 inet static address 276.9.249.150 netmask 255.255.255.248 bridge_ports none bridge_stp off bridge_fd 0[/SIZE]

virtrel · Oct 4, 2017

Hey everybody.

I also need some assistance with my Hetzner subnet configuration. It would be very kind, if anybody could help me.

Subnet
Gateway: 136.243.37.4
Netzmaske: 255.255.255.248
Broadcast: 148.251.158.167

148.251.158.160
148.251.158.161
148.251.158.162
148.251.158.163
148.251.158.164
148.251.158.165
148.251.158.166
148.251.158.167

Interface from host (Proxmox):

auto lo
iface lo inet loopback

auto eth0
iface eht0 inet static

auto vmbr0
iface vmbr0 inet static
address 136.243.37.4
netmask 255.255.255.255
gateway 136.243.37.1
pointopoint 136.243.37.1
bridge_ports eth0
bridge_stp off
bridge_fd 1
bridge_hello 2
bridge_maxage 12

auto vmbr1
iface vmbr1 inet static
address 140.251.158.166
netmask 255.255.255.248
bridge_ports none
bridge_stp off
bridge_fd 0

Interface from client (VM):

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address 148.251.158.161
netmask 255.255.255.248
gateway 136.243.37.4
pointopoint 136.243.37.4

I am able to ping 136.243.37.4 from the client, but I'm not able to access the internet.
Have I forgotten anything, or is anything wrong with my config?

Search

Search

Proxmox 4 Hetzner extra subnet configuration

asmar

Renowned Member

monkfish

Active Member

petrogazz

New Member

monkfish

Active Member

asmar

Renowned Member

petrogazz

New Member

monkfish

Active Member

asmar

Renowned Member

asmar

Renowned Member

asmar

Renowned Member

Maus112

New Member

ErTnEc

New Member

virtrel

New Member