Please help me checking and confirming network layout

[StefanoA]

Hi,
I'm a newbie with Proxmox and I'd like to have my network structure checked and confirmed before moving on
So apologies in advance for any wrong term or dumb question

This is what I am planning



I don't really want to touch anything in the ISP modem/router, can't even set it to bridge mode so let's just leave it (192.168.1.x subnet)

My main LAN is supposed to be managed by the virtualized OpenWRT in the 192.168.0.x subnet.
Wired and wireless, needs to include everything: the main PC, other virtualized machines/services in PM, mobile devices and IoT devices

The host computer I'm using has 4 x 2.5Gb Intel NICs i226 and I'm planning to reserve one to Proxmox management interface and the other 3 as in the diagram
I'm also planning to connect the PM bridge vmbr1 as virtual eth3 in OpenWRT in order to connect any VM/CT to the main LAN directly into PM and get IP address, etc

I also installed tailscale on the host so that I can access it from outside and independently from which LAN address it actually gets and without port forwarding

I think overall the above layout makes sense and specially allowing me connection to the host even in case of problems with the virtualized router,
the only disadvantages I can see are:

1) the host cannot access directly the main LAN 192.168.0.x
2) the host can access internet only through 100Mbps connection
3) every download/transfer to/from the host will be limited to 100Mbps
4) one physical LAN port of the host machine is kind of wasted

Is there any way to share the 1Gbps connection to the ISP router between PM management interface and the WAN port of OpenWRT using the same physical port?
I think this could solve all above drawbacks

thanks in advance

 

[Richard]

Is there any way to share the 1Gbps connection to the ISP router between PM management interface and the WAN port of OpenWRT using the same physical port?

Yes, assign simply an IP address to the bridge where the physical NIC is assigned to.
The diagram does not present the Prox Host configuration precisely, clearer woud be the content of /etc/network/interfaces or simply the output of

pvereport

 

[StefanoA]

Hi Richard, thanks for your reply

Here is /etc/network/interfaces
root@proxmox:/etc/network# cat interfaces # network interface settings; autogenerated # Please do NOT modify this file directly, unless you know what # you're doing. # # If you want to manage parts of the network configuration manually, # please utilize the 'source' or 'source-directory' directives to do # so. # PVE will preserve these directives, but will NOT read its network # configuration from sourced files, so do not attempt to move any of # the PVE managed interfaces into external files! auto lo iface lo inet loopback iface enp2s0 inet manual iface enp3s0 inet manual iface enp4s0 inet manual iface enp5s0 inet manual auto vmbr0 iface vmbr0 inet static address 192.168.0.10/24 gateway 192.168.0.1 bridge-ports enp2s0 bridge-stp off bridge-fd 0 auto vmbr1 iface vmbr1 inet manual bridge-ports none bridge-stp off bridge-fd 0

but note that since my first post I changed the static address of proxmox to 192.168.0.10 and the gateway accordingly and I connected the LAN port to the dumb AP, so now the host lies in the main subnet 192.168.0.0/24, just temporarily

From your reply I understand that the static IP i provide to the bridge will be the one of proxmox management interface while any other VM/CT connected to the same bridge can either have its own static ip or get it by DHCP, In this way I can have both Proxomox and the WAN of OpenWRT connected to the same 1Gbps port of the ISP router, ok

My main concern/issue is this one: I'd like to have my proxmox to be in the same subnet of the virtualized router 192.168.0.0/24, so all devices and services can communicate among them.
But in case of any problem to the virtualized router the whole network would go down and I wouldn't be able to connect to proxmox anymore, for example to restart it.

So how can I have proxmox connected "normally" to gateway 192.168.0.1 / static ip 192.168.0.10
and in case of problems to gateway 192.168.1.1 / static ip 192.168.1.10 ?
Is it possible to have two configurations without conflicts?

thanks

 

[Richard]

Hi Richard, thanks for your reply

Here is /etc/network/interfaces
root@proxmox:/etc/network# cat interfaces # network interface settings; autogenerated # Please do NOT modify this file directly, unless you know what # you're doing. # # If you want to manage parts of the network configuration manually, # please utilize the 'source' or 'source-directory' directives to do # so. # PVE will preserve these directives, but will NOT read its network # configuration from sourced files, so do not attempt to move any of # the PVE managed interfaces into external files! auto lo iface lo inet loopback iface enp2s0 inet manual iface enp3s0 inet manual iface enp4s0 inet manual iface enp5s0 inet manual auto vmbr0 iface vmbr0 inet static address 192.168.0.10/24 gateway 192.168.0.1 bridge-ports enp2s0 bridge-stp off bridge-fd 0 auto vmbr1 iface vmbr1 inet manual bridge-ports none bridge-stp off bridge-fd 0

According to the above enp3,4,5s0 are not configured at all - therefore it is not possible to use them for traffic to a VM too. If you want to use them of the mentioned openwrt VM you have to assign them to bridges and also assign the virtual NICs in the VMs to this bridges too.

 

[StefanoA]

Hi Richard,

enp3,4,5s0 are passed though to OpenWRT VM,
enp3 is the wan
the others are connected to OWRT br-lan and to vmbr1 of PVE and form my main LAN 192.168.0.x



I understand ports (either virtual or physical) are always passed in this order from PVE to OpenWRT

PVE OpenWRT
vmbr0 - ETH0
vmbr1 - ETH1 - LAN - Used to connect all PVE VMs and CTs
enp3s0 - ETH2 - WAN
enp4s0 - ETH3 - LAN - Connected to main PC
enp5s0 - ETH4 - LAN - Connected to dumb AP for WIfi

 

[Richard]

enp3,4,5s0 are passed though to OpenWRT VM,

Does "passed through" mean PCi pass-through using IOMMU? Since this is an experimental feature it's not reliable and may cause the problems you have.

 

[StefanoA]

Yes, I'm using IOMMU because I thought this way I can reduce the overhead and latencies to the router part of the system making it closer to a bare metal soft-router. Would you rather recommend to use VirtIO network cards instead for all NICs?

Actually I haven't had any problem/issue with this setup so far (running about one month)

My main concern is how to manage issues with the virtualized router and how to maintain the connection to the Proxmox host in this case
So again, what happens if I connect the Proxmox host to two interfaces:

vmbr1 / NO physical port / static 192.168.0.10 / gateway 192.168.0.1 (internet coming from OpenWRT VM)

AND

vmbr0 / enp3s0 / static 192.168.1.10 / gateway 192.168.1.1 (internet coming from my ISP)

so that when everything's fine my host is connected and reachable through my 192.168.0.x LAN for management, backup, etc
BUT in case of problems with OpenWRT it will still be reachable from 192.168.1.x subnet ?
is this possible / advisable ? If I try to set a second gateway I get an error

thanks