Proxmox has no internet connection, BUT OPNSense and Docker and Portainer does !

[Grazor]

HI All Im Very new to all this, so ill describe it best I can, if you need any information or screenshots plz let me know ill provide as fast as I can

SO I Installed PROXMOX VIA Usb install undated it as normal no problem, then OPNSense undated as normal, this is when I lost connection to the internet for ProxMOX only.
since then ive installed PORTAINER and DOCKER with no trouble at all. everything has internet axcess and can me updated BUT PROXMOX.
Does Anyone have an area for me to check or Idears what is going on ?
thank you in Advance

 

[Dunuin]

First you should show us your network setting...

cat /etc/network/interfaces
cat /etc/hosts
cat /etc/resolv.conf
ip addr

...and tell us what IP your gateway is. Should the PVE node use the OPNsense as a gateway or some other router?
Which bridges is OPNsense connected to or do you passthough a NIC? Any vlans in use?

 

[Grazor]

I will get all that info for you now mate thank you for reading this

 

[Grazor]

 

[Grazor]

there is all requested network settings ill now try an answer the rest best I can mate

 

[bbgeek17]

you should not configure systems with multiple NICs on the same subnet. Its a special configuration that requires expert understanding of Linux routing. There are very few cases where this type of setup is useful. Incorrect configurations are common and lead to strange results.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[Grazor]

Im connecting using PPPOE directly to the Router which is running Proxmox and Opnsense is runnin inside Proxmox as a VM
the Router is then connected to a switch with Wifi that connects to all devices in the house.
the Internal Gateway is 192.168.1.1

 

[Dunuin]

Like bbgeek17 already said, first you should get rid of one of the IPs. So either use 192.168.1.100 on vmbr0 or 192.168.1.101 on vmbr1 but not both. And don't set a gateway twice per host.

 

[Grazor]

Thank you very much i will get it sorted asap and see what happens
Thank you Both Dunuin and bbgeek17

 

[coolgen]

too bad no further follow up on this subject as i have about the same issue which i cannot seem to resolve

 

[NWAdmin]

too bad no further follow up on this subject as i have about the same issue which i cannot seem to resolve

I had the same issue. I resolved it by adding my dns servers manually to /etc/resolv.conf .
In my case, proxmox was not able to gather its dns servers, as proxmox starts before OPNsense is running. After adding the dns servers and rebooting, everything worked.

In my case:
nameserver 192.168.50.10 - my local pihole
nameserver 192.168.50.1 - my OPNsense LAN
nameserver 1.1.1.1 - Cloudflare just in case something fails

 

[Kingneutron]

@NWAdmin I do the same thing. Never understood why anyone would want to virtualize a main Internet router under its own host server tho. Move the router to a separate host or run it bare-metal and then you don't have the chicken-and-egg problem.

 

[Dunuin]

@NWAdmin I do the same thing. Never understood why anyone would want to virtualize a main Internet router under its own host server tho. Move the router to a separate host or run it bare-metal and then you don't have the chicken-and-egg problem.

Or better run it with failover on two nodes. Als long as you never shutdown both nodes at the same time there are no chickens or eggs.

In my case:
nameserver 192.168.50.10 - my local pihole
nameserver 192.168.50.1 - my OPNsense LAN
nameserver 1.1.1.1 - Cloudflare just in case something fails

With that you stab pi-hole in the back.
When using any ad/tracker blocking DNS server this should be your only DNS server (or use two ad blocking DNS servers for redundancy). Otherwise clients might use the non-blocking DNS servers and you defeat the whole purpose of pi-hole.
For example when using windows. "primary DNS server" doesn't mean it will be prefered over the "secondary DNS server" with the secondary only being used when the primary is not avaiable. It will choose one of them at random and then might be switching between 1.1.1.1 and pi-hole and only half of the time ads will be blocked and you will still be tracked.

I wrote a tutorial here on how to set up two Pihole LXCs for redundancy:
Post in thread 'Pi-hole LXC with gravity-sync?' https://forum.proxmox.com/threads/pi-hole-lxc-with-gravity-sync.109881/post-645646

And there is a tutorial on how to set up OPNsense with failover:
https://www.thomas-krenn.com/en/wiki/OPNsense_HA_Cluster_configuration