Setup VM LAN to talk to internet in masquerade mode

[renatom]

I have a single server attached to the internet on a public IP.

I have 4 different VM's attached to vmbr1. Their Ip's are 10.0.0.101, 10.0.0.102, 10.0.0.103 and 10.0.0.104. Gateway 10.0.0.254 for all of them.

I cannot make then go to the internet. They cannot ping 8.8.8.8 or any other internet address,

/etc/network/interfaces:

auto lo
iface lo inet loopback

auto eno1
iface eno1 inet manual

auto eno2
iface eno2 inet manual

auto vmbr0
iface vmbr0 inet static
address 140.221.22.23/24
gateway 140.221.22.254
bridge-ports eno1
bridge-stp off
bridge-fd 0
hwaddress A4:BF:01:1E:58:48

auto vmbr1
iface vmbr1 inet static
address 10.0.0.254/24
bridge-ports none
bridge-stp off
bridge-fd 0
post-up echo 1 > /proc/sys/net/ipv4/ip_foward
post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o eno1 -j MASQUERADE
post-up iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o eno1 -j MASQUERADE
post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1


Questions:
a) What am I missing here to make it work?

b) I need to bond eno1 and eno2? What is the correct way of doing it?

 

[shrdlicka]

Hi,
a)
- what does your config in one of the VMs look like?
- is there a firewall active in proxmox?

b)
did you take a look at this: https://pve.proxmox.com/wiki/Network_Configuration#_linux_bond

 

[renatom]

OK. I got how to build the bond and end with the following configuration:

/etc/network/interfaces:

auto lo
iface lo inet loopback

auto eno1
iface eno1 inet manual

auto eno2
iface eno2 inet manual

auto bond0
iface bond0 inet manual
bond-slaves eno1 eno2
bond-miimon 100
bond-mode active-backup
bond-primary eno1

auto vmbr0
iface vmbr0 inet static
address 141.121.18.23/24
gateway 141.121.18.254
bridge-ports bond0
bridge-stp off
bridge-fd 0
hwaddress A4:BF:01:44:27:BA

auto vmbr1
iface vmbr1 inet static
address 10.0.0.1/24
bridge-ports none
bridge-stp off
bridge-fd 0

post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o bond0 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o bond0 -j MASQUERADE

My VM configuration:
# cat 180.conf
boot: order=ide0;ide2;net0
cores: 8
ide0: local:180/vm-180-disk-0.qcow2,size=64G
machine: pc-i440fx-6.2
memory: 4096
meta: creation-qemu=6.2.0,ctime=1657882056
name: server2012
net0: e1000=22:32:3D:AB:10:47,bridge=vmbr1,firewall=1
numa: 0
onboot: 1
ostype: win8
scsihw: virtio-scsi-pci
smbios1: uuid=f8a3c049-98f2-4187-ac52-133435dc5df5
sockets: 1
vmgenid: bf23dbc2-eae2-46cd-bab1-3d1fc3683dfe


I'm running Windows Server with the following configuration:



Windows still is not able to ping or connect the internet (ping 8.8.8.8).

Here is the iptables configuration:

# iptables -t nat -L -n -v
Chain PREROUTING (policy ACCEPT 2108 packets, 138K bytes)
pkts bytes target prot opt in out source destination

Chain INPUT (policy ACCEPT 920 packets, 47205 bytes)
pkts bytes target prot opt in out source destination

Chain OUTPUT (policy ACCEPT 204 packets, 13697 bytes)
pkts bytes target prot opt in out source destination

Chain POSTROUTING (policy ACCEPT 294 packets, 19797 bytes)
pkts bytes target prot opt in out source destination
0 0 MASQUERADE all -- * bond0 10.0.0.0/24 0.0.0.0/0


What shall be missing? Help appreciated.

 

[shrdlicka]

Did you check if the firewall is active?
see in the WebGui:

• Datacenter/Firewall/Options
• [Node name]/Firewall/Options

Can you ping the gateway aka 10.0.0.1?

 

[renatom]

Firewall is off for the server:


And for the VM's:



I cannot ping the gateway (10.0.0.1) from the VMs.

In the meantime, I've created another VM attached to vmbr1. One VM can ping the other VM. No VM can ping the gateway (10.0.0.1 - vmbr1).

Seens that this is the problem, not pinging the gateways.... but why not?

My new VM (Linux) OS config:

iface ens18 inet static
address 10.0.0.100/24
gateway 10.0.0.1
dns-nameservers 10.0.0.1
dns.search mydomain.com

 

[renatom]

I've create a new VM: 10.0.0.110 with gateway 10.0.0.1
That one pings the gateway (10.0.0.1), does not ping internet (8.8.8.8) and does not ping other VM (10.0.0.100 or 10.0.0.180). All 3 VMS are in vmbr1.

Don't know what can be...

 

[shrdlicka]

A very stupid suggestion: Did you try restarting the host system?

Can your host system ping all the VMs?

 

[renatom]

Rebooted the system.

Now:
- All VMs pings each other
- All VM's pings gateway (10.0.0.1)
- Host pings all VMS

But:
- No VM pings Internet (8.8.8.8) or access it
- Host pings internet (8.8.8.8) and access it

 

[renatom]

By the way, there is something very important I've not mentioned.

I'm using OVH, so as far as I know at OVH things are different and the normal NAT won't work. Is that true? Do I really need a failover Ip?

 

[bobmc]

post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o bond0 -j MASQUERADE

I would suggest you try
post-up iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -o vmbr0 -j MASQUERADE

If you're still having issues after that, I would suggest you forget about bonding as I doubt you will see much benefit and it just adds complication