[SOLVED] Strange routing - host can't connect to internet via VM gateway and PCI passthrough'd NIC.

taurolyon

Member
Sep 7, 2021
9
1
8
Sorry for the long post. I'm having to connect to my host box by KVM and there's no easy way to copy-paste from there. I had to install LXDE and a web browser to configure my firewall VM.

Anywho... My setup is on a dedicated server with a single IP address and network uplink. I am trying to setup OPNsense/PFsense as a VM with the network card passed through as the WAN, the LAN is connected to the bridge vmbr0.

I did try shifting this to vmbr1 in a later attempt so my logs below will reference vmbr1. I'm trying to use only using one bridge so I use these interchangeably.

My bad network diagram:
Untitled Diagram.drawio.png

What works:
OPNsense picked up the Intel NIC via PCI Passthrough without a problem and I was able to configure the static IPs for the LAN and WAN sides.
OPNsense can connect to the internet and communicate properly.
Guest machines can connect to vmbr0, is assigned a DHCP by OPNsense, and communicate with the internet.
Proxmox host can ping the gateway firewall IP.

What doesn't work
Proxmox host cannot communicate with the internet.

What's weird:
When VM100 is running: a tap100i0 device with a link-local IP is created.Screenshot 2023-02-11 012542.png

This tap device creates a default entry on my routing table
Screenshot 2023-02-11 012751.png
I attempted to remove the default, even tried a flush and this happens:
Screenshot 2023-02-11 014522.png
This default route remains... and any attempts to add or change the default root produces the "File exists" error.

My /etc/network/interfaces:
Screenshot 2023-02-11 013002.png
(disregard eno1, as it is passed through to VM100 and isn't present after the VM is running)
Probably bad idea: I attempted to bridge to lo, but it disregards this as the mac is all zeros (invalid).
Ping attempts from the host:
Screenshot 2023-02-11 013547.png

I'm not sure what I'm missing.
Does Proxmox allow for such a configuration where there is no physical network device on the host box?
 
So, the answer was to use ip route replace instead of add/del.

I fixed it with:
Code:
sudo ip route replace default via 172.16.0.1 dev vmbr1
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!