[wrong] Network configuration for two Proxmox in Hetzner with vSwitch (VM host bridged)

F

Fra

Renowned Member
Dec 10, 2011
143
10
83
Since is tooks awhile for us to reach this working network setup, we want to share in case somebody else need it


The two hosts are in Hetzner; they have only one network card, one public IP address (used by the VM & CT to reach the internet: none of the VM have incoming internet traffic); they are both connected to an Hetzner vSwitch, which is also connected to the Hetzner network cloud (subnet 10.15.200.0/24, gateway 10.15.200.1), where we have a wireguard server that handle VPN with which VPN users are able to connect to all CT and VM.
The VM and CT have IP in 10.15.200.0/24, with gateway either on 10.15.200.201 or 10.15.200.202, so we can move them from one proxmox to the other with no issue.

(it's a working configuration, but with the public IP redacted)

Any advice is welcome, of course:



Bash: 
auto lo
iface lo inet loopback
 
iface enp5s0 inet manual
        mtu 1400
        post-up echo 1 > /proc/sys/net/ipv4/conf/enp5s0/proxy_arp
 
iface enp5s0.4001 inet manual
        mtu 1400
 
auto vmbr0
iface vmbr0 inet static
        address 248.51.123.20/27
        gateway 248.51.123.1
        bridge-ports enp5s0
        bridge-stp off
        bridge-fd 0
        mtu 1400
        pointopoint 248.51.123.1
        post-up echo 1 > /proc/sys/net/ipv4/conf/vmbr0/proxy_arp
 
auto vmbr1
iface vmbr1 inet static
        address 10.15.200.202/24
        bridge-ports enp5s0.4001
        bridge-stp off
        bridge-fd 0
        mtu 1400
        up   ip route add 10.15.0.0/16 via 10.15.200.1 dev vmbr1
        down ip route del 10.15.0.0/16 via 10.15.200.1 dev vmbr1
        post-up echo 1 > /proc/sys/net/ipv4/conf/vmbr1/proxy_arp
        post-up   iptables -t nat -A POSTROUTING -s '10.15.200.0/24' -o vmbr0 -j SNAT --to-source 248.51.123.20
        post-down iptables -t nat -D POSTROUTING -s '10.15.200.0/24' -o vmbr0 -j SNAT --to-source 248.51.123.20

Bash: 
auto lo
iface lo inet loopback
 
iface enp5s0 inet manual
        mtu 1400
        post-up echo 1 > /proc/sys/net/ipv4/conf/enp5s0/proxy_arp
 
iface enp5s0.4001 inet manual
        mtu 1400
 
auto vmbr0
iface vmbr0 inet static
        address 248.51.186.174/26
        gateway 248.51.186.129
        bridge-ports enp5s0
        bridge-stp off
        bridge-fd 0
        mtu 1400
        pointopoint 248.51.186.129
        post-up echo 1 > /proc/sys/net/ipv4/conf/vmbr0/proxy_arp
 
auto vmbr1
iface vmbr1 inet static
        address 10.15.200.201/24
        bridge-ports enp5s0.4001
        bridge-stp off
        bridge-fd 0
        mtu 1400
        up   ip route add 10.15.0.0/16 via 10.15.200.1 dev vmbr1
        down ip route del 10.15.0.0/16 via 10.15.200.1 dev vmbr1
        post-up echo 1 > /proc/sys/net/ipv4/conf/vmbr1/proxy_arp
        post-up   iptables -t nat -A POSTROUTING -s '10.15.200.0/24' -o vmbr0 -j SNAT --to-source 248.51.186.174
        post-down iptables -t nat -D POSTROUTING -s '10.15.200.0/24' -o vmbr0 -j SNAT --to-source 248.51.186.174
 
Last edited:
QQ请问加了一个网卡，，，插的光纤，，，，为什么无法识别。。目前网络不可用
 
Tony said:
Hi,

I also use Proxmox with Hetzner vSwitch, following the tutorial by @Razva in this forum and the script Script to install Proxmox 5.x and 6.x on a Dedicated Hetzner Server – schaal @it UG (schaal-it.com). It works, but not really stable, sometime the VMs on different nodes cannot talk to each other.

Your config does set proxy_arp for the bridge. How did you arrive at that setting?
Click to expand...
I found that setup in several places, for example https://www.flomain.de/2015/05/how-to-proxmox-networking/ (very good one, to gain the big picture, btw)

We are used to enable it with other wireguard server and we are just using it https://docs.sweeting.me/s/wireguard#WireGuard-Documentation (if you want I can turn it off and see the effect).

As to how is reliable the vSwitch in Hetzner, we just started to use it, and yes, we too experimented delays sometimes (with a Proxmox setup on top of Debian), now with the ISO installation everything appears to work better, but I will let you know (we are testing it now). By the way, I spotted this one that could have an effect in our previous installation https://forum.proxmox.com/threads/wrong-elaboration-of-etc-network-interface-by-proxmox-gui.91271/
 
chenjie said:
QQ请问加了一个网卡，，，插的光纤，，，，为什么无法识别。。目前网络不可用
Click to expand...
uh, that is a bit generic, did you experience it with the above config? and by the way, I am the last one here, to be able to give advice, but if you post your detailed issue somebody for sure can give you a feed.
 
Fra said:
I found that setup in several places, for example https://www.flomain.de/2015/05/how-to-proxmox-networking/ (very good one, to gain the big picture, btw)

We are used to enable it with other wireguard server and we are just using it https://docs.sweeting.me/s/wireguard#WireGuard-Documentation (if you want I can turn it off and see the effect).

As to how is reliable the vSwitch in Hetzner, we just started to use it, and yes, we too experimented delays sometimes (with a Proxmox setup on top of Debian), now with the ISO installation everything appears to work better, but I will let you know (we are testing it now). By the way, I spotted this one that could have an effect in our previous installation https://forum.proxmox.com/threads/wrong-elaboration-of-etc-network-interface-by-proxmox-gui.91271/
Click to expand...

We've removed all the proxy_arp setting, and everything stil works.
 
Fra said:
Since is tooks awhile for us to reach this working network setup, we want to share in case somebody else need it


The two hosts are in Hetzner; they have only one network card, one public IP address (used by the VM & CT to reach the internet: none of the VM have incoming internet traffic); they are both connected to an Hetzner vSwitch, which is also connected to the Hetzner network cloud (subnet 10.15.200.0/24, gateway 10.15.200.1), where we have a wireguard server that handle VPN with which VPN users are able to connect to all CT and VM.
The VM and CT have IP in 10.15.200.0/24, with gateway either on 10.15.200.201 or 10.15.200.202, so we can move them from one proxmox to the other with no issue.

(it's a working configuration, but with the public IP redacted)

Any advice is welcome, of course:



Bash: 
auto lo
iface lo inet loopback
 
iface enp5s0 inet manual
        mtu 1400
        post-up echo 1 > /proc/sys/net/ipv4/conf/enp5s0/proxy_arp
 
iface enp5s0.4001 inet manual
        mtu 1400
 
auto vmbr0
iface vmbr0 inet static
        address 248.51.123.20/27
        gateway 248.51.123.1
        bridge-ports enp5s0
        bridge-stp off
        bridge-fd 0
        mtu 1400
        pointopoint 248.51.123.1
        post-up echo 1 > /proc/sys/net/ipv4/conf/vmbr0/proxy_arp
 
auto vmbr1
iface vmbr1 inet static
        address 10.15.200.202/24
        bridge-ports enp5s0.4001
        bridge-stp off
        bridge-fd 0
        mtu 1400
        up   ip route add 10.15.0.0/16 via 10.15.200.1 dev vmbr1
        down ip route del 10.15.0.0/16 via 10.15.200.1 dev vmbr1
        post-up echo 1 > /proc/sys/net/ipv4/conf/vmbr1/proxy_arp
        post-up   iptables -t nat -A POSTROUTING -s '10.15.200.0/24' -o vmbr0 -j SNAT --to-source 248.51.123.20
        post-down iptables -t nat -D POSTROUTING -s '10.15.200.0/24' -o vmbr0 -j SNAT --to-source 248.51.123.20

Bash: 
auto lo
iface lo inet loopback
 
iface enp5s0 inet manual
        mtu 1400
        post-up echo 1 > /proc/sys/net/ipv4/conf/enp5s0/proxy_arp
 
iface enp5s0.4001 inet manual
        mtu 1400
 
auto vmbr0
iface vmbr0 inet static
        address 248.51.186.174/26
        gateway 248.51.186.129
        bridge-ports enp5s0
        bridge-stp off
        bridge-fd 0
        mtu 1400
        pointopoint 248.51.186.129
        post-up echo 1 > /proc/sys/net/ipv4/conf/vmbr0/proxy_arp
 
auto vmbr1
iface vmbr1 inet static
        address 10.15.200.201/24
        bridge-ports enp5s0.4001
        bridge-stp off
        bridge-fd 0
        mtu 1400
        up   ip route add 10.15.0.0/16 via 10.15.200.1 dev vmbr1
        down ip route del 10.15.0.0/16 via 10.15.200.1 dev vmbr1
        post-up echo 1 > /proc/sys/net/ipv4/conf/vmbr1/proxy_arp
        post-up   iptables -t nat -A POSTROUTING -s '10.15.200.0/24' -o vmbr0 -j SNAT --to-source 248.51.186.174
        post-down iptables -t nat -D POSTROUTING -s '10.15.200.0/24' -o vmbr0 -j SNAT --to-source 248.51.186.174
Click to expand...

no need for all the proxy_arp in the above
Fra said:
Since is tooks awhile for us to reach this working network setup, we want to share in case somebody else need it


The two hosts are in Hetzner; they have only one network card, one public IP address (used by the VM & CT to reach the internet: none of the VM have incoming internet traffic); they are both connected to an Hetzner vSwitch, which is also connected to the Hetzner network cloud (subnet 10.15.200.0/24, gateway 10.15.200.1), where we have a wireguard server that handle VPN with which VPN users are able to connect to all CT and VM.
The VM and CT have IP in 10.15.200.0/24, with gateway either on 10.15.200.201 or 10.15.200.202, so we can move them from one proxmox to the other with no issue.

(it's a working configuration, but with the public IP redacted)

Any advice is welcome, of course:



Bash: 
auto lo
iface lo inet loopback
 
iface enp5s0 inet manual
        mtu 1400
        post-up echo 1 > /proc/sys/net/ipv4/conf/enp5s0/proxy_arp
 
iface enp5s0.4001 inet manual
        mtu 1400
 
auto vmbr0
iface vmbr0 inet static
        address 248.51.123.20/27
        gateway 248.51.123.1
        bridge-ports enp5s0
        bridge-stp off
        bridge-fd 0
        mtu 1400
        pointopoint 248.51.123.1
        post-up echo 1 > /proc/sys/net/ipv4/conf/vmbr0/proxy_arp
 
auto vmbr1
iface vmbr1 inet static
        address 10.15.200.202/24
        bridge-ports enp5s0.4001
        bridge-stp off
        bridge-fd 0
        mtu 1400
        up   ip route add 10.15.0.0/16 via 10.15.200.1 dev vmbr1
        down ip route del 10.15.0.0/16 via 10.15.200.1 dev vmbr1
        post-up echo 1 > /proc/sys/net/ipv4/conf/vmbr1/proxy_arp
        post-up   iptables -t nat -A POSTROUTING -s '10.15.200.0/24' -o vmbr0 -j SNAT --to-source 248.51.123.20
        post-down iptables -t nat -D POSTROUTING -s '10.15.200.0/24' -o vmbr0 -j SNAT --to-source 248.51.123.20

Bash: 
auto lo
iface lo inet loopback
 
iface enp5s0 inet manual
        mtu 1400
        post-up echo 1 > /proc/sys/net/ipv4/conf/enp5s0/proxy_arp
 
iface enp5s0.4001 inet manual
        mtu 1400
 
auto vmbr0
iface vmbr0 inet static
        address 248.51.186.174/26
        gateway 248.51.186.129
        bridge-ports enp5s0
        bridge-stp off
        bridge-fd 0
        mtu 1400
        pointopoint 248.51.186.129
        post-up echo 1 > /proc/sys/net/ipv4/conf/vmbr0/proxy_arp
 
auto vmbr1
iface vmbr1 inet static
        address 10.15.200.201/24
        bridge-ports enp5s0.4001
        bridge-stp off
        bridge-fd 0
        mtu 1400
        up   ip route add 10.15.0.0/16 via 10.15.200.1 dev vmbr1
        down ip route del 10.15.0.0/16 via 10.15.200.1 dev vmbr1
        post-up echo 1 > /proc/sys/net/ipv4/conf/vmbr1/proxy_arp
        post-up   iptables -t nat -A POSTROUTING -s '10.15.200.0/24' -o vmbr0 -j SNAT --to-source 248.51.186.174
        post-down iptables -t nat -D POSTROUTING -s '10.15.200.0/24' -o vmbr0 -j SNAT --to-source 248.51.186.174
Click to expand...
forget about the above configuration: it is wrong! the setup apparently works but it is not stable: there is a conflict on routing.


sorry for the useless noise: anyhow if somebody get an Hetzner vSwitch working setup that accomplish the following needs, please share :)

Needs:
* VM reach the internet with the proxmox (so with IP: 248.51.186.174): no need for the VM to be reachable from the internet.
* VM and promox host have IPs belonging to the same subnet: 10.15.200.0/24
* VM and promox reach 10.15.0.0/16 via Hetzner vSwitch on 10.15.200.1
* an hetzner cloud host attached to the vSwitch (with ip 10.15.0.2) can reach all IP in 10.15.200.0/24
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom