IPv4 in bridge mode and IPv6 in routed mode for Hetzner dedicated server running Proxmox VE

Tronde

New Member
May 8, 2023
2
0
1
Germany
www.my-it-brain.de
Dear Proxmox-Community,
I'm running Proxmox VE 8.0.4 on Debian GNU/Linux 12 (Bookworm) using a Hetzner dedicated server. The server comes with one IPv4 address, one MAC, and an IPv6 subnet (/64). I have booked an additional IPv4 and MAC address for a specific VM that needs to be reachable via IPv4 from the internet. Here is what I'm trying to accomplish:
  • The host running Proxmox should be reachable via IPv4 and IPv6
  • I would like to have one bridge (vmbr1) for virtual machines (VMs) using RFC1918 IP addresses and connecting to the internet via NAT
  • I would like to have one segment where VMs get an IPv6 address from my subnet and can communicate with the internet in routed mode to avoid MAC abuse errors (Hetzner allows traffic only from dedicated MAC addresses)
  • The 'proxy-vm' should be accessible from the internet via IPv4 on one interface and via IPv6 on another interface
What I have tried so far

Following you find the configuration of my Proxmox-host. The IP addresses where obfuscated using IPs from RFC5737 and RFC3849.

Code:
:~$ cat /etc/network/interfaces
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
    pre-up /sbin/ip addr flush dev eth0 || true

iface eth0 inet6 static
    address 2001:db8:919:1922::1/128
    gateway fe80::1

auto vmbr0
iface vmbr0 inet static
    address 198.51.100.58/27
    gateway 198.51.100.33
    bridge-ports eth0
    bridge-stp off
    bridge-fd 0

iface vmbr0 inet6 static
    address 2001:db8:919:1922::2/64

auto vmbr1
iface vmbr1 inet static
    address 203.0.113.1/24
    bridge-ports none
    bridge-stp off
    bridge-fd 0

    post-up echo 1 > /proc/sys/net/ipv4/ip_forward
    post-up iptables -t nat -A POSTROUTING -s '203.0.113.0/24' -o vmbr0 -j MASQUERADE
    post-down iptables -t nat -D POSTROUTING -s '203.0.113.0/24' -o vmbr0 -j MASQUERADE
   

:~$ cat /etc/network/interfaces.d/vm-routes
iface vmbr0 inet6 static
    up ip -6 route add 2001:db8:919:1922::/64 dev vmbr0
   

:~$ cat /etc/network/interfaces.d/vmbr0-extra
iface vmbr0 inet static
    hwaddress de:ad:be:ef:73:f1


:~$ ip route
default via 144.76.31.33 dev vmbr0 proto kernel onlink 
144.76.31.32/27 dev vmbr0 proto kernel scope link src 144.76.31.58 
144.76.31.56 dev vmbr0 scope link 
192.168.188.0/24 dev vmbr1 proto kernel scope link src 192.168.188.1 

:~$ ip -6 route
2a01:4f8:191:2219::/64 dev vmbr0 proto kernel metric 256 pref medium
2a01:4f8:191:2219::/64 dev vmbr0 metric 1024 pref medium
fe80::/64 dev vmbr1 proto kernel metric 256 pref medium
fe80::/64 dev vmbr0 proto kernel metric 256 pref medium


On my VMs I use RHEL. Here is the IP configuration of the proxy-vm that is connected to vmbr0:

Code:
ip -br a
  lo        UNKNOWN    127.0.0.1/8 ::1/128
  ens18        UP        198.51.100.56/27 2001:db8:919:1922::3/64 fe80::250:56ff:fe00:XXXX/64
 
  ip route
  default via 198.51.100.33 dev ens18 proto static metric 100
  198.51.100.32/27 dev ens18 kernel scope link src 198.51.100.56 metric 100
 
  ip -6 route
  ::1 dev lo proto kernel metric 256 pref medium
  2001:db8:919:1922::/64 dev ens18 proto kernel metric 100 pref medium
  fe80::/64 dev ens18 proto kernel metric 1024 pref medium
  default via fe80::1 dev ens18 proto static metric 100 pref medium

Other VMs have IPv4 disabled and are using an IPv6 address, e.g. 2001:db8:919:1922::4/64 and IPv6 gateway fe80::1. With this configuration they are able to communicate but cause an MAC abuse error as their MAC is visible on the switch the Proxmox-host is connected to. Currently their virtual NICs are disconnected to avoid the MAC abuse issue.

During my search on the web I read about using a router VM with OPNsense. But I like to avoid using an additional VM for that purpose as I believe it's not necessary.

What do I need help with?
  • Can you check my Proxmox-host and client config for any obvious error that I might have missed?
  • Can someone explain to me how a solution would look like?
  • Is it even possible to have IPv4 in bridged but IPv6 in routed mode?
Your help is much appreciated. If any information is missing from my post I'm happy to provide them when you tell me what is needed.

Best regards,
Tronde
 
Last edited:
Hello,

We have same problem. ipv4 vms are running (bridged config) but we couldn't make ipv6 to work (routed config).

Did you find any solution?
 
It's quite frustrating - there are lots of posts like this, but no solutions anywhere.

Like many I have a range of IPv4 addresses that need to work so I have no choice but running in bridge mode, at least for IPv4, and that works fine.

It seems that IPv6 has to run in routed mode. I don't know what is the implications of that, how do you set that up?
 
We make it work after some hours :)

/etc/network/interfaces file of the server
Code:
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback
iface lo inet6 loopback
iface enp7s0 inet manual

auto vmbr0
iface vmbr0 inet static
  address 123.123.123.123/26 #main ip address of the server
  hwaddress 54:32:13:33:59:33
  gateway 123.123.123.1
  pointopoint 123.123.123.1
  bridge_ports enp7s0
  bridge_stp off
  bridge_fd 0

iface vmbr0 inet6 static
  address 2a01:444:333:ccc::2/64 #an ipv6 address from the subnet that Hetzner assigned
  gateway fe80::1

auto vmbr1
iface vmbr1 inet static
  address 123.123.4.1 #ipv4 subnet from Hetzner to use with vms that has ipv4, this ip will also be the gateway ip of vms that are using this subnet
  netmask 255.255.255.240
  bridge_ports none
  bridge_stp off
  bridge_fd 0

So we make ipv6 available for vmbr0 network. Now if you want a server that has an ip from ipv4 subnet and also an ipv6 address, you have to add 2 nics to the vm (vmbr0 for ipv6, vmbr1 for ipv4) and after install you can give an ipv6 address from ipv6 subnet like below:

Code:
auto eth0
iface eth0 inet static
  address 123.123.4.2
  netmask 255.255.255.240
  gateway 123.123.4.1

auto eth1
iface eth1 inet6 static
  address 2a01:444:333:ccc::4/64
  gateway fe80::1

This config works for us.
 
Last edited:
Thanks - but at the same time I made my post here, I contacted Hetzner support, who referred me to this page. It actually seems to be explaining it quite well. I hope this can help others as well.

It seems, with a IPv4 range, what I need (and use) is routed network, sorry for the misunderstanding. The effective difference, if I understand it correct, is that with routed network you lose and IP address for the gateway, but can then allocate one or more IP addresses to VM's as you please, whereas with bridged network, you need to request and assign a MAC address to each VM and only one IP address can be allocated per VM.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!