[SOLVED] Create Private Network Bridge with NAT

Nanja · Apr 22, 2023

Hi, I was following a guide and they stopped the guide before it tells you what you are supposed to setup on the VM.

This is how guide told me to set it up

Code:

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

auto eno1
iface eno1 inet manual
        dns-nameservers 8.8.8.8

iface eno2 inet manual

auto vmbr0
iface vmbr0 inet static
        address myip/24
        gateway myip
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0

auto vmbr1
iface vmbr1 inet static
        address 10.10.10.1
        netmask 255.255.255.0
        bridge_ports none
        bridge_stp off
        bridge_fd 0
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o vmbr0 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o vmbr0 -j MASQUERADE

On the VM I set it up like this, but like I said.. the guide never covered what you are supposed to put on the VM.

Code:

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

allow-hotplug ens18
iface ens18 inet static
    address 10.10.10.2/24
    gateway 10.10.10.0
    dns-nameservers 1.1.1.1 1.0.0.1 8.8.8.8 8.8.4.4
    dns-search mydomain

If no one can help me, could I atleast get a link to a better guide for setting up a VM. I wish to have multiple VMs using my same public IP.

Edit: maybe I should've said how I set it up on VM doesn't seem to work and I can't ping outside world.

remark · Apr 22, 2023

Hi there!
Just find any guide to setup NAT with iptables for typical "office network". Say, you VMs are "office computers" connected to same LAN as your proxmox host.
Your proxmox IP will be default gateway for "office computers".

But IMHO better way is to setup router as VM, not to give public IP to host. pfSense may be as one of. Give that router VM two virtual network adapters, first one wil be "WAN" and second will be "LAN".

Sleipnir · Apr 23, 2023

Hello,

On the VM, gateway cannot be a network address. Try 10.10.10.1 instead of 10.10.10.0

You also need to copy/paste from the documentation and adapt to your needs, you mistyped 3 times in a row in /etc/network/interfaces !

Nanja · Apr 24, 2023

What do I do now? I corrected the 10.10.10.0 to 10.10.10.1
Now I can enter vm and ping something, but suddenly it stops after first ping and can't ping anymore.
I imagine it's a firewall thing?

I have this setup on host machine, like a guide told me

Code:

iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o vmbr0 -j MASQUERADE
iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o vmbr0 -j MASQUERADE
iptables -A POSTROUTING -t nat -s 10.10.10.0/24 -j MASQUERADE

I thought that was all I needed was those 3 things.
Do I need to add some rules to the iptables on VM too?

remark · Apr 24, 2023

You do not need add any iptables rules on VM. Just set right ip address, gateway and DNS.

Code:

iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o vmbr0 -j MASQUERADE
iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o vmbr0 -j MASQUERADE
iptables -A POSTROUTING -t nat -s 10.10.10.0/24 -j MASQUERADE

First line adds specified iptables rule (-A option)
Second line removes that added rule (-D option)
Third line adds rule that different from 1st.

To understand iptables rule managing, you can issue 'iptables -L -t nat' after each line. You will see current rules and can understand what was added/removed by each iptables command.

remark · Apr 24, 2023

auto vmbr0
iface vmbr0 inet static
address myip/24
gateway myip
bridge-ports eno1
bridge-stp off
bridge-fd 0

What specified in the 'gateway' parameter? It should be gateway ip, given by ISP.

Nanja · Apr 24, 2023

remark said:
What specified in the 'gateway' parameter? It should be gateway ip, given by ISP.

A lot of guides have told me to set it up like this.. where I remove the ip and gateway from eno1 and put it on vmbr0

& my iptables

Code:

root@dedi:~# iptables -L -t nat
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
MASQUERADE  all  --  10.10.10.0/24        anywhere

No sure why, but once I boot up vm.. if I am fast enough, I can ping something but then it stops and gets cut off by something?
I really wish datacenter gave me extra IP... this is rather difficult. I am only NAT since I only have one ip.

Sleipnir · Apr 24, 2023

You don't seem to have corrected the file on Proxmox itself. There are 3 times the same mistake so vmbr1 cannot work correctly. Don't use any documentation, use the one from the Proxmox team. Or just check how vmbr0, which is correct, differs from vmbr1.

remark · Apr 24, 2023

Is your vmbr1 do not have slave interface?
Please do not follow guides blind. Try to understand.
Your ISP is gateway for your Proxmox host.
Your Proxmox host is gateway for your VMs on Proxmox host.
So, your Proxmox host acts as a router.

And try to comment out 'auto eno1' in /etc/network/interfaces on Proxmox host.

Sleipnir · Apr 24, 2023

There's no slave interface with NAT. You're perfectly right when you say that PVE is the router in this case, though it should point to another router. 104.194.x.x may be correct but I doubt it, it should be a local address.

Nanja · Apr 24, 2023

I read proxmox network configuration documentation, so I tried to setup what they said.

Keeping isp ip and gateway on eno1.
I messed up though, since I tried to change it in web interface... Clicked apply and it took forever to load and then server became not accessible anymore.

Anyways, this is what is on proxmox site, after ISP reinstall my OS I will try this.
Thanks for assistance, I think this will work.

Code:

auto lo
iface lo inet loopback

auto eno1
#real IP address
iface eno1 inet static
        address  198.51.100.5/24
        gateway  198.51.100.1

auto vmbr0
#private sub network
iface vmbr0 inet static
        address  10.10.10.1/24
        bridge-ports none
        bridge-stp off
        bridge-fd 0

        post-up   echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up   iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o eno1 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o eno1 -j MASQUERADE

remark · Apr 24, 2023

Keep us updated and good luck!

Sleipnir · Apr 24, 2023

Hopefully it'll work.

remark · Apr 24, 2023

As an addition.
First. If you do not pass eno1 network to VMs, then you do not need eno1 bridged. So you assign public ip to eno1 directly, this looks right.
Second. If you have pubilc ip on your network interface, take care of security, at least setup firewall on Proxmox host. But do it carefully, read documentation before take any steps.

Sleipnir · Apr 24, 2023

if eno1 is not in the loop then we are far from nat

Nanja · Apr 24, 2023

remark said:
As an addition.
First. If you do not pass eno1 network to VMs, then you do not need eno1 bridged. So you assign public ip to eno1 directly, this looks right.
Second. If you have pubilc ip on your network interface, take care of security, at least setup firewall on Proxmox host. But do it carefully, read documentation before take any steps.

My ssh somehow reconnected, so I started working on it. I got the VM to be able to ping, it won't resolve domains.

Like I can go to websites as well using IP's, but can't resolve domains. What do I do next? I mean I have dns set to cloudflare on vm.
Is it because I set my /etc/host file to this on vm

Code:

10.10.10.5 dedi.something.com dedi

Which is also like my host machine

Code:

realip dedi.something.com dedi

remark · Apr 24, 2023

eno1 must be physically connected to an ISP or upstream router.

Sleipnir · Apr 24, 2023

Or you can connect your DNS server to vmbr1 so that your VMs can benefit from it.

remark · Apr 24, 2023

Nanja said:
My ssh somehow reconnected, so I started working on it. I got the VM to be able to ping, it won't resolve domains.
View attachment 49568

Like I can go to websites as well using IP's, but can't resolve domains. What do I do next? I mean I have dns set to cloudflare on vm.
Is it because I set my /etc/host file to this on vm

Code:

10.10.10.5 dedi.something.com dedi

Which is also like my host machine

Code:

realip dedi.something.com dedi

Setup DNS on your host.
Go to your host Web interface -> System -> DNS
Specify one or more DNS servers there.

If your VM cannot resolve domains, setup DNS on VM, this depends on what OS you run on it.
common for linux if edit /etc/resolv.conf:
nameserver 1.1.1.1
nameserver 8.8.8.8

But it may vary on different distributions and direct editing of resolv.conf may be undesired.

remark · Apr 24, 2023

Sleipnir said:
Or you can connect your DNS server to vmbr1 so that your VMs can benefit from it.

This required to setup an own DNS server, at least caching, right?

[SOLVED] Create Private Network Bridge with NAT

New Member

Renowned Member

Member

New Member

Renowned Member

Renowned Member

New Member

Member

Renowned Member

Member

New Member

Renowned Member

Member

Renowned Member

Member

New Member

Renowned Member

Member

Renowned Member

Renowned Member