Allowing "other" subnet from router?

[wmorrison]

My main home router (Alpine Linux, nftables and dnsmasq) uses subnet 192.168.1.0/24, all working.

My Proxmox machine is on that subnet, but I have modified /etc/network/interfaces as follows:

# Top lines are original as installed, subnet 192.168.1.0/24 auto lo iface lo inet loopback iface eno1 inet manual auto vmbr0 iface vmbr0 inet static address 192.168.1.123/24 gateway 192.168.1.1 bridge-ports eno1 bridge-stp off bridge-fd 0 # Added lines below, subnet 192.168.2.0/24, for VMs using vmbr1 auto vmbr1 iface vmbr1 inet static address 192.168.2.1/24 netmask 255.255.255.0 bridge-ports none bridge-stp off bridge-fd 0 post-up echo 1 > /proc/sys/net/ipv4/ip_forward post-up iptables -t nat -A POSTROUTING -s '192.168.2.0/24' -o eno1 -j MASQUERADE post-down iptables -t nat -D POSTROUTING -s '192.168.2.0/24' -o eno1 -j MASQUERADE

If I create a VM using vmbr1 and IP in range 192.168.2.0/24, it can ping vmbr1 and other IPs in range 192.168.2.0/24

But it cannot access anything outside that range.

On my router, tshark sees ping requests from 192.168.2.0/24, but I'm guessing they are dropped?

I'm not sure how to modify /etc/nftables.nft (on my router, not Proxmox) to allow traffic from this "second" subnet. Ideas?

BTW, I'm doing this because I want to set up multiple VMs that require very specific hostname/dns rules, and I'd rather keep this configuration on the Proxmox machine than "pollute" my main router's configuration. Just want to make minimal changes there, once, to allow the traffic, if possible. So that if I "break" anything, it's just the Proxmox machine rather than my whole network.

 

[wmorrison]

Got it working. The missing piece was ARP proxying. Without this, pings/connections from VMs inside the "second" network failed because the router was sending arp "who has" requests that the VM never received (so couldn't answer.)

tshark is very useful debugging this kind of stuff.

That's configured in /etc/sysctl.conf

Two settings, one should be there by default, can't remember whether I de-commented it or the Proxmox install did.

net.ipv4.ip_forward=1

The ARP proxy one was not present in the file, so I added

net.ipv4.conf.all.proxy_arp=1

So basically, those two settings plus adding this to /etc/network/interfaces

# whatever name doesn't conflict, I just used the next available one auto vmbr1 iface vmbr1 inet static # network address for the "internal Proxmox subnet" address 192.168.2.1/24 bridge-ports none bridge-stp off bridge-fd 0

Then on my main router on 192.168.1.0/24, added a route to 192.168.2.0/24

ip route add 192.168.2.0/24 via 192.168.1.1

And for nftables, masquerade the new network when destination is the world.

Don't think I'm forgetting anything... That should be enough if you only care about IP connectivity, but for DNS, well, that's why I'm running a second dnsmasq on the Proxmox machine, servicing only the "2" network, but each of the two dnsmasq's use each other as a second nameserver. Also using separate domains for each network. That's fairly straightforward so I won't add steps unless someone asks.

Again, the only reason I did this (besides the learning experience) is so I can have custom dns settings for VMs (when I put them on vmbr1) without "polluting" my main router's configuration.