Proxmox accessible via switch but not directly.

[NiKTaMeR]

So here is my situation.

Desktop -> fiber -> VM OPNsense

VM OPNsense (DHCP server) -> Switch + WAP -> Rest of network

I can reach ProxMox perfectly when connected through wifi but not when through fiber.

Server:
- 2 x 2 SFP+ (enpXsXfX)
-1 GbE (enp3s0)
-1 old 100Mb NIC (enp5s1)

Everything is bridged and the enp5s1 is connected to the switch.

I'm probably missing somthing really dumb but have been playing with it for hours and can't figure it out. When i remove the bridge from enp5s1, I just completely lose access from everywhere (had to restore though console).

Here is my interfaces file:

auto lo
iface lo inet loopback


auto enp3s0
iface enp3s0 inet manual


auto enp1s0f0
iface enp1s0f0 inet manual


auto enp1s0f1
iface enp1s0f1 inet manual


iface enp2s0f0 inet manual


iface enp2s0f1 inet manual


auto enp5s1
iface enp5s1 inet manual


auto vmbr0
iface vmbr0 inet static
    address 192.168.2.2/24
    gateway 192.168.2.1
    bridge-ports enp5s1
    bridge-stp off
    bridge-fd 0
#100Mbit


auto vmbr1
iface vmbr1 inet manual
    bridge-ports enp1s0f0
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094
#Wan Bridge


auto vmbr2
iface vmbr2 inet manual
    bridge-ports enp3s0
    bridge-stp off
    bridge-fd 0
#Lan Bridge


auto vmbr3
iface vmbr3 inet manual
    bridge-ports enp1s0f1
    bridge-stp off
    bridge-fd 0
#10Gbe Desktop


auto vmbr4
iface vmbr4 inet manual
    bridge-ports enp2s0f0
    bridge-stp off
    bridge-fd 0
#Unused fiber

Thanks in advance for any insights,
Cheers.

 

[ph0x]

So you try to reach the host through the 10GbE NIC which is bridged by vmbr3?
Since it doesn't have an address, I guess vmbr3 is one of the interfaces of the firewall VM (like you stated above). Then I would say the error lies inside the firewall VM.

 

[NiKTaMeR]

So you try to reach the host through the 10GbE NIC which is bridged by vmbr3?
Since it doesn't have an address, I guess vmbr3 is one of the interfaces of the firewall VM (like you stated above). Then I would say the error lies inside the firewall VM.

Could be, but there is something I'm not getting if it is.

I have attached a graphical representation of my setup to make it clearer (for both you AND I )

The reason I don't think the issue is in the FW is because 3 NICs are bridged as LAN in OPNsense (vmbr2-3-4) and vmbr2 goes back through ethernet to the Wireless router (set in AP mode).

The Proxmox server is connected to the switch via vmbr0 (100mbit nic) through ethernet.

Now from my desktop connected via fiber I can connect everything on my network except ProxMox. I was even able to ssh into OPN, and ssh into ProxMox from that session. If I disable the adapter and connect through wifi it works perfecly.
Since the the fiber link (vmbr3) is bridged the the link that goes in the switch (vmbr2), I don't understand how I can access one way but not the other ?

P.S. quick addendum, vmbr1 is an SFP+ ONT

 

[ph0x]

So you can ssh to the Proxmox box but not reach the GUI?

 

[NiKTaMeR]

So you can ssh to the Proxmox box but not reach the GUI?

Nope can't ssh directly (when connected with fiber), can ssh to OPNsense and then ssh to Proxmox from within that shell.

It might just be me muddying the water.

 

[ph0x]

Did you allow ssh connections from WAN to LAN in OPNSense?

 

[NiKTaMeR]

No, I thought this would be an undue risk. However both devices are on the LAN, so not sure how this would help.

 

[ph0x]

Ah, you're right, I had a twist in my mind.

 

[ph0x]

I guess it's a routing issue because of the big bridge inside the firewall.
Maybe you could try with different interfaces inside the firewall and work with routing rather than bridging.

 

[NiKTaMeR]

Quick update, couldn't live with the annoyance and couldn't fin an answer so decided to start from scratch ... A few hours later and 5-6 mumbled curse words later, everything is working. Thanks for trying ph0x, much appreciated.

 

[ph0x]

At least it works now.