Proxmox + Hetzner | VMs without public IP can't access internet

[speedbird]

Hello all
The following scenario...
My proxmox server at Hetzner is running with the following network config:

auto lo
iface lo inet loopback
iface enp4s0 inet manual

auto vmbr0
iface vmbr0 inet static
        address xx.xx.xx.58/27
        gateway xx.xx.xx.29
        bridge-ports enp4s0
        bridge-stp off
        bridge-fd 0

post-up iptables -A INPUT -i vmbr0 -p tcp --destination-port 111 -j REJECT || true
post-up iptables -A FORWARD -i vmbr0 -p tcp --destination-port 111 -j REJECT || true
post-up iptables -A INPUT -i vmbr0 -p udp --destination-port 111 -j REJECT || true
post-up iptables -A FORWARD -i vmbr0 -p udp --destination-port 111 -j REJECT || true


auto vmbr1
iface vmbr1 inet static
 address 10.10.0.1
 netmask 255.255.255.0
 bridge_ports none
 bridge_stp off
 bridge_fd 0

post-up iptables -t nat -A POSTROUTING -s '10.10.0.0/24' -o vmbr0 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s '10.10.0.0/24' -o vmbr0 -j MASQUERADE

I have 2 VMs running there, which have separate IPs, bought as single IPs via robot, assigned via MAC address. Works 1a.

Now I want to create 1-2 more VMs, which don't need a public IP, but still should have access to the network, e.g. for updates / downloads etc..


I tried the following:

Create a new VM, use vmbr1 for network.

On the VM, set IP to 10.10.0.5, gateway 10.10.0.1, netmask to 255.255.255.0 and dns to 10.10.0.1 / 8.8.8.8 / nothing

so... no matter what DNS i set (10.10.0.1 or 8.8.8.8 or nothing), I can't get the VM to reach the internet. I can't ping any IP, nothing. I can reach other VMs on the same network, like 10.10.0.2 and I can also reach the 10.10.0.5 from the .2 vm, that's all fine but I can't get out to like download updates or anything.

So what am I doing wrong here?

 

[Spoonman2002]

Makes sense because only vmbr0 has a Gateway.
I think you need to install a virtual router, like pfSense/OPNsense/IPFire on your PVE host.
Sorry, I totally missed the last two NAT rules....

 

[speedbird]

Makes sense because only vmbr0 has a Gateway.

But shouldn't the NAT routing of vmbr1 deal with that or do I really need another router vm to make this happen?

 

[Spoonman2002]

But shouldn't the NAT routing of vmbr1 deal with that or do I really need another router vm to make this happen?

Remove the quotes '10.10.0.0/24'
Should be 10.10.0.0/24

 

[bobmc]

I think you're missing the option to enable forwarding on the host, your network config file should include

post-up echo 1 > /proc/sys/net/ipv4/ip_forward

these configs also normally include the following commands

post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1 post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1

as @Spoonman2002 said, remove the quotes around the ip addresses

 

[speedbird]

Remove the quotes '10.10.0.0/24'
Should be 10.10.0.0/24

Removes the quotes.

Added the ip_forward post-up as well

Looks like this now:

auto lo
iface lo inet loopback

iface enp4s0 inet manual

auto vmbr0
iface vmbr0 inet static
        address xx.xx.xx.58/27
        gateway xx.xx.xx.29
        bridge-ports enp4s0
        bridge-stp off
        bridge-fd 0

post-up iptables -A INPUT -i vmbr0 -p tcp --destination-port 111 -j REJECT || true
post-up iptables -A FORWARD -i vmbr0 -p tcp --destination-port 111 -j REJECT || true
post-up iptables -A INPUT -i vmbr0 -p udp --destination-port 111 -j REJECT || true
post-up iptables -A FORWARD -i vmbr0 -p udp --destination-port 111 -j REJECT || true

auto vmbr1
iface vmbr1 inet static
 address 10.10.0.1
 netmask 255.255.255.0
 bridge_ports none
 bridge_stp off
 bridge_fd 0

post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up iptables -t nat -A POSTROUTING -s 10.10.0.0/24 -o vmbr0 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s 10.10.0.0/24 -o vmbr0 -j MASQUERADE

Still nothing I don't get it.

post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1

post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1

What does that do? Never saw that in any configs before.

 

[Spoonman2002]

Removes the quotes.

Added the ip_forward post-up as well

Looks like this now:

auto lo
iface lo inet loopback

iface enp4s0 inet manual

auto vmbr0
iface vmbr0 inet static
        address xx.xx.xx.58/27
        gateway xx.xx.xx.29
        bridge-ports enp4s0
        bridge-stp off
        bridge-fd 0

post-up iptables -A INPUT -i vmbr0 -p tcp --destination-port 111 -j REJECT || true
post-up iptables -A FORWARD -i vmbr0 -p tcp --destination-port 111 -j REJECT || true
post-up iptables -A INPUT -i vmbr0 -p udp --destination-port 111 -j REJECT || true
post-up iptables -A FORWARD -i vmbr0 -p udp --destination-port 111 -j REJECT || true

auto vmbr1
iface vmbr1 inet static
 address 10.10.0.1
 netmask 255.255.255.0
 bridge_ports none
 bridge_stp off
 bridge_fd 0

post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up iptables -t nat -A POSTROUTING -s 10.10.0.0/24 -o vmbr0 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s 10.10.0.0/24 -o vmbr0 -j MASQUERADE

Still nothing I don't get it.





What does that do? Never saw that in any configs before.

Did you do a "systemctl restart networking" ?
Or via the webgui "Apply Configuration" ?

 

[speedbird]

Yes. I even rebooted Proxmox entirely to be sure.

 

[bobmc]

can you post a traceroute from on of the vm's on vmbr1