[SOLVED] pfSense 2.6 unable to ping gateway or beyond

robotdog

New Member
Dec 23, 2021
5
0
1
53
Hi all,

Problem Statement:

After installing pfSense 2.6 into my ProxMox host, no VMs or PCs can get to the internet. They cannot ping the gateway.
My usual diagnostic is to ping from the WAN interface and the LAN interface both in the GUI and from the shell.
I can ping 192.168.50.4 from any VM or PC and I can ping 192.168.1.71 from any VM or PC, but I cannot ping the GW (192.168.1.254)
I can also ping the gateway and internet from the proxmox host, with no issues.

Note: occasionally/sporadically, my pings will make it through the gateway and onto the final destination.

What am I doing wrong here? I'm pretty sure I've misconfigured the pfsense install or the proxmox networking side, even though the VMs worked for months prior to this recent installation. I just don't understand why the pfsense vm won't let me out. :-( Any help would be appreciated, before I go completely bald.

Thx.

Configuration:

ATT Fiber to ATT Modem(BGW320-500): This is the Gateway (IP: 192.168.1.254)
Modem connect direct (CAT 6) to the PFSense WAN interface 192.168.1.71.
LAN interface is 192.168.50.4. Only one subnet 192.168.50.0/24

Map:

{{Internet}} --> [ATT modem] (GW:192.168.1.254) --> pfSense WAN (vmbr1:192.168.1.71) --> pfSense LAN (vmbr2:192.168.50.4)



Proxmox host is 7.1-7 and has 5 NICs. One onboard RealTek 1GB nic and a 4 port QNAP PCIe card.

root@proxmox-01:~# root@proxmox-01:~# ip -c a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp12s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast master vmbr0 state DOWN group default qlen 1000 link/ether 1c:1b:0d:97:a9:b6 brd ff:ff:ff:ff:ff:ff 3: enp15s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr1 state UP group default qlen 1000 link/ether 24:5e:be:7a:bb:38 brd ff:ff:ff:ff:ff:ff 4: enp16s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr2 state UP group default qlen 1000 link/ether 24:5e:be:7a:bb:37 brd ff:ff:ff:ff:ff:ff 5: enp18s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master vmbr3 state DOWN group default qlen 1000 link/ether 24:5e:be:7a:bb:36 brd ff:ff:ff:ff:ff:ff 6: enp19s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master vmbr4 state DOWN group default qlen 1000 link/ether 24:5e:be:7a:bb:35 brd ff:ff:ff:ff:ff:ff 7: vmbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000 link/ether 1c:1b:0d:97:a9:b6 brd ff:ff:ff:ff:ff:ff inet 192.168.50.3/24 scope global vmbr0 valid_lft forever preferred_lft forever 8: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether 24:5e:be:7a:bb:38 brd ff:ff:ff:ff:ff:ff inet 192.168.1.71/24 scope global vmbr1 valid_lft forever preferred_lft forever inet6 fe80::265e:beff:fe7a:bb38/64 scope link valid_lft forever preferred_lft forever 9: vmbr2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether 24:5e:be:7a:bb:37 brd ff:ff:ff:ff:ff:ff inet 192.168.50.5/24 scope global vmbr2 valid_lft forever preferred_lft forever inet6 fe80::265e:beff:fe7a:bb37/64 scope link valid_lft forever preferred_lft forever 10: vmbr4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000 link/ether 24:5e:be:7a:bb:35 brd ff:ff:ff:ff:ff:ff inet 192.168.60.1/24 scope global vmbr4 valid_lft forever preferred_lft forever inet6 fe80::265e:beff:fe7a:bb35/64 scope link valid_lft forever preferred_lft forever 11: vmbr3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000 link/ether 24:5e:be:7a:bb:36 brd ff:ff:ff:ff:ff:ff inet6 fe80::265e:beff:fe7a:bb36/64 scope link valid_lft forever preferred_lft forever



2022-08-03 18_11_40-__TRUENAS_adahnert_User Downloads.png

the default route is 192.168.1.254

root@proxmox-01:~# ip -c r default via 192.168.1.254 dev vmbr1 proto kernel onlink 192.168.1.0/24 dev vmbr1 proto kernel scope link src 192.168.1.71 192.168.50.0/24 dev vmbr2 proto kernel scope link src 192.168.50.5 192.168.50.0/24 dev vmbr0 proto kernel scope link src 192.168.50.3 linkdown 192.168.60.0/24 dev vmbr4 proto kernel scope link src 192.168.60.1 linkdown


pfsense configuration:
vtnet0 is 192.168.1.71
vtnet1 is 192.168.50.4

I have turned off the "Offload Checksums" checkmark in Advanced/Networking
I have unchecked the "Private Networks" for both WAN and LAN.
In my rules, I have opened the floodgates with an PASS ANY/ANY just to make sure nothing was blocked.

2.6.0-RELEASE][root@pfSense.fastlane]/root: ifconfig vtnet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: WAN options=800b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE> ether 1e:c0:8c:ae:1e:5a inet6 fe80::1cc0:8cff:feae:1e5a%vtnet0 prefixlen 64 scopeid 0x1 inet 192.168.1.71 netmask 0xffffff00 broadcast 192.168.1.255 media: Ethernet 10Gbase-T <full-duplex> status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> vtnet1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=800b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE> ether c2:aa:30:31:13:af inet6 fe80::c0aa:30ff:fe31:13af%vtnet1 prefixlen 64 scopeid 0x2 inet 192.168.50.4 netmask 0xffffff00 broadcast 192.168.50.255 media: Ethernet 10Gbase-T <full-duplex> status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> enc0: flags=0<> metric 0 mtu 1536 groups: enc nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> pflog0: flags=100<PROMISC> metric 0 mtu 33160 groups: pflog pfsync0: flags=0<> metric 0 mtu 1500 groups: pfsync
 

vesalius

Active Member
Aug 19, 2020
162
43
28
one issue, is that both proxmox and pfsense are trying to claim 192.168.1.71. I would first change the vmbr1 CIDR to something in the 192.168.1.0/24 subnet but not 192.168.1.71, 192.168.1.72 for example. Removing/changing the vmbr1 CIDR at the Proxmox host network level will not affect pfsense's use of that IP over the vmbr1 bridge.

Next question is do you want proxmox to use the ATT modem as its gateway or pfsense? You can consider removing the gateway from vmbr1 and adding a gateway to vmbr2 using the 192.168.50.4 pfsense IP.
 

robotdog

New Member
Dec 23, 2021
5
0
1
53
AWESOME! Something so simple, seems to have corrected the problem. Not sure why it didn't make sense to me before. thanks very much vesalius, your help is much appreciated.

For your second question, I "think" I want proxmox to keep the ATT gateway. However, I have an additional ISP that is only for work related items and if I can make it work, would like to use it as a 'fail over' in case the ATT fiber connection goes down. How does that change things?
 

vesalius

Active Member
Aug 19, 2020
162
43
28
Easiest, is to set up multiwan failover in pfSense. Plenty of online guides and videos about this. Plug the second isp into an unused physical port on the Proxmox host, set up a linux bridge (could use vmbr3 if nothing else does) with that physical interface (label it WAN2). Give the the Linux bridge to pfSense and then follow a guide to set up multiwan failover.

after that all of your vm/LXC using the pfsense Lan/vmbr2 will get failover without any changes. the Proxmox host would as well if you shift the gateway to vmbr2 as I mentioned was possible above. If you do make this gateway change just be sure to select the option to have the pfSense VM start automatically on boot.

BTW this is mirrrors my own setup except I use an OPNsense VM to handle multiwan failover.
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!