Hello,
I use Proxmox since version 4.4 in my homelab, and recently set up my first Proxmox instance on a Hetzner Root Server. The machine has a single public IPv4 and the usual /64 IPv6 subnet. While most of the setup is working, I have an issue where I need some more ideas.
For IPv6, the host and every VM has an address from the subnet. This is the most straightforward configuration and works flawless.
For IPv4, the public address is assigned to the host. For the VMs, I use a routed setup with masquerading on the host. This is also working flawless.
Additionally, the machine and the VMs have to be accessible from Hetzner cloud servers within a private network. To make this work, I connected a vSwitch to the cloud network and to the server. Hetzner uses VLAN tag 4000 for vSwitch packages. I created an additional interface for this VLAN, and an additional VM Bridge. The bridge has an IP address from the vSwitch private subnet. All VMs connected to this VM Bridge also get an IP address from this subnet. For the VMs, this is working as it should, the VMs can access the cloud servers and vice versa.
On the host, I experience a strange issue. Right after reboot, the connection works incoming and outgoing. After some minutes, it stops working. The host is inaccessible from all cloud servers. In the other direction, the cloud servers are also at first inaccessible from the host. However, after a few packages, the connection comes back alive and works for a few minutes in both directions - until the issue starts again.
When I "fix" the connection with a ping, it takes 3-4 lost packages until the connection works.
When I "fix" it using traceroute to a cloud server, it takes from 14 to 16 until the vSwitch answers.
The issue is only on the host - on the VMs it's always working, independently from the current state on the host.
What could be the issue? Honestly I'm running out of ideas, and I couldn't find a similar case in the web. I'll append my /etc/network/interfaces configuration with the IPs masked.
One note, I really don't want to have more than one public IPv4 address. There will be quite some VMs, and I need the public IPv4 just for API provides who think it's fine to not offer IPv6 for their APIs. All incoming traffic goes through a load balancer and the private network. In times of short IPv4 supplies, I don't want to waste them just for having IPv4 internet connectivity.
In case you need any more information, feel free to ask. Looking forward for any ideas.
TheJKM
I use Proxmox since version 4.4 in my homelab, and recently set up my first Proxmox instance on a Hetzner Root Server. The machine has a single public IPv4 and the usual /64 IPv6 subnet. While most of the setup is working, I have an issue where I need some more ideas.
For IPv6, the host and every VM has an address from the subnet. This is the most straightforward configuration and works flawless.
For IPv4, the public address is assigned to the host. For the VMs, I use a routed setup with masquerading on the host. This is also working flawless.
Additionally, the machine and the VMs have to be accessible from Hetzner cloud servers within a private network. To make this work, I connected a vSwitch to the cloud network and to the server. Hetzner uses VLAN tag 4000 for vSwitch packages. I created an additional interface for this VLAN, and an additional VM Bridge. The bridge has an IP address from the vSwitch private subnet. All VMs connected to this VM Bridge also get an IP address from this subnet. For the VMs, this is working as it should, the VMs can access the cloud servers and vice versa.
On the host, I experience a strange issue. Right after reboot, the connection works incoming and outgoing. After some minutes, it stops working. The host is inaccessible from all cloud servers. In the other direction, the cloud servers are also at first inaccessible from the host. However, after a few packages, the connection comes back alive and works for a few minutes in both directions - until the issue starts again.
When I "fix" the connection with a ping, it takes 3-4 lost packages until the connection works.
When I "fix" it using traceroute to a cloud server, it takes from 14 to 16 until the vSwitch answers.
The issue is only on the host - on the VMs it's always working, independently from the current state on the host.
What could be the issue? Honestly I'm running out of ideas, and I couldn't find a similar case in the web. I'll append my /etc/network/interfaces configuration with the IPs masked.
One note, I really don't want to have more than one public IPv4 address. There will be quite some VMs, and I need the public IPv4 just for API provides who think it's fine to not offer IPv6 for their APIs. All incoming traffic goes through a load balancer and the private network. In times of short IPv4 supplies, I don't want to waste them just for having IPv4 internet connectivity.
In case you need any more information, feel free to ask. Looking forward for any ideas.
TheJKM
Code:
source /etc/network/interfaces.d/*
auto lo
iface lo inet loopback
iface lo inet6 loopback
auto enp27s0
iface enp27s0 inet static
address <PUBLIC_IP>/32
gateway <PUBLIC_IP_GATEWAY>
pointopoint <PUBLIC_IP_GATEWAY>
up route add -net <PUBLIC_IP_NET_BASE> netmask 255.255.255.192 gw <PUBLIC_IP_GATEWAY> dev enp27s0
iface enp27s0 inet6 static
address <PUBLIC_IP_V6>/128
gateway fe80::1
auto enp27s0.4000
iface enp27s0.4000 inet manual
mtu 1400
auto vmbr0
iface vmbr0 inet static
address 192.168.0.1/24
bridge-ports none
bridge-stp off
bridge-fd 0
post-up iptables -t nat -A POSTROUTING -s '192.168.0.0/24' -o enp27s0 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s '192.168.0.0/24' -o enp27s0 -j MASQUERADE
iface vmbr0 inet6 manual
address <ANOTHER_PUBLIC_IP_V6>/64
up ip -6 route add <PUBLIC_IP_V6_PREFIX>/64 dev vmbr0
auto vmbr4000
iface vmbr4000 inet static
address 10.0.30.10/24
bridge-ports enp27s0.4000
bridge-stp off
bridge-fd 0
mtu 1400
up ip route add 10.0.0.0/16 via 10.0.30.1 dev vmbr4000
down ip route del 10.0.0.0/16 via 10.0.30.1 dev vmbr4000