Routes to other networks not working

[jaywll]

Hello! I'm running into networking problem with my proxmox guest VMs that I'm scratching my head over, and I'm hoping the community can help me out.

Here's my setup:

• 192.168.0.1: Router / default gateway / access point
• 192.168.0.5 / 192.168.10.5: DHCP server, VPN server, other stuff
• 192.168.0.10: Proxmox server with a few VMs including windows 11 and linux (mostly ubuntu)
• 192.168.0.20: (Soon to be shut down) XCP-NG server with a few VMs
• 192.168.0.x: Other clients

As you can see, 192.168.0.5 is connected to two networks, and is responsible for routing traffic between them. I don't think it's important, but it's running zentyal. The router is just a home router, an Asus ZenWifi. On the router, there's a static route defined that specifies 192.168.0.5 as the gateway for 192.168.10.0/24.

Almost everything works great: all devices on the 192.168.10.x network can reach everything (including the VMs) on the 192.168.0.x network just fine. All the physical devices (including the proxmox server itself) and XCP-NG VMs on the 192.168.0.x network can reach everything on the 192.168.10.x network.

The problem is the proxmox VMs. None of them can reach the 192.168.10.x network at all, and I can't for the life of me figure out why. It's got to be proxmox, but with both the server and all the VMs sharing vmbr0 I'm having a tough time understanding why traffic from the server is routed correctly to the other network but traffic from the VMs is getting stuck somewhere. I haven't changed any firewall settings on the proxmox server. Maybe I need to to get this setup working?

When I try and ping a client on the 192.168.10.x network from one of the proxmox VMs I get 100% packet loss:

From _gateway (192.168.0.1): icmp_seq=2 Redirect Host(New nexthop: percy.redacted.com (192.168.0.5))

Any help or guidance is greatly appreciated!

 

[mira]

What are the IPs you configured inside the VM?

Please provide the output of ip r from inside the VM (Linux).

In addition, please provide the network config (cat /etc/network/interfaces) and the ip r of the PVE host.


Did you enable the firewall on your PVE/the VMs?

 

[jaywll]

The IPs inside the VMs are assigned by DHCP. They're reserved addresses so they stay consistent.

ip r from inside one of the VMs:

default via 192.168.0.1 dev ens18 proto dhcp src 192.168.0.23 metric 100
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1
172.18.0.0/16 dev br-6a014a7c9e29 proto kernel scope link src 172.18.0.1
192.168.0.0/24 dev ens18 proto kernel scope link src 192.168.0.23
192.168.0.1 dev ens18 proto dhcp scope link src 192.168.0.23 metric 100

/etc/network/interfaces from the pve host:

#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eno1 inet manual

iface eno2 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.0.10/24
        gateway 192.168.0.1
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0

ip r from the pve host:

default via 192.168.0.1 dev vmbr0 proto kernel onlink
192.168.0.0/24 dev vmbr0 proto kernel scope link src 192.168.0.10

I haven't changed the firewall settings on either the host or any of the VMs.

I also need to correct something I said in my original post, that the pve host can reach the other networks just fine - it can reach the other networks but "just fine" turned out to be a stretch, and there's definitely something going on. On the host pinging a device on the other network seems to have a ~40% success rate (which I guess is better than 0% on the VMs, but still).

root@sybill:~# ping -c3 whirlwind
PING whirlwind.redacted.com (192.168.10.11) 56(84) bytes of data.
From 192.168.0.1 (192.168.0.1): icmp_seq=2 Redirect Host(New nexthop: percy.redacted.com (192.168.0.5))
64 bytes from 192.168.10.11 (192.168.10.11): icmp_seq=3 ttl=63 time=71.5 ms

--- whirlwind.redacted.com ping statistics ---
3 packets transmitted, 1 received, 66.6667% packet loss, time 2024ms
rtt min/avg/max/mdev = 71.476/71.476/71.476/0.000 ms

I guess a potential solution would be to manually create the relevant route on the pve host and each of the VMs, but I'd really prefer to avoid doing that if I can. I plan to make changes to the setup in the future, so having the routes defined in a single spot (on the router) is attractive to me if it's possible.