[SOLVED] pfSense 2.6 unable to ping gateway or beyond

robotdog · Aug 4, 2022

Hi all,

Problem Statement:

After installing pfSense 2.6 into my ProxMox host, no VMs or PCs can get to the internet. They cannot ping the gateway.
My usual diagnostic is to ping from the WAN interface and the LAN interface both in the GUI and from the shell.
I can ping 192.168.50.4 from any VM or PC and I can ping 192.168.1.71 from any VM or PC, but I cannot ping the GW (192.168.1.254)
I can also ping the gateway and internet from the proxmox host, with no issues.

Note: occasionally/sporadically, my pings will make it through the gateway and onto the final destination.

What am I doing wrong here? I'm pretty sure I've misconfigured the pfsense install or the proxmox networking side, even though the VMs worked for months prior to this recent installation. I just don't understand why the pfsense vm won't let me out. :-( Any help would be appreciated, before I go completely bald.

Thx.

Configuration:

ATT Fiber to ATT Modem(BGW320-500): This is the Gateway (IP: 192.168.1.254)
Modem connect direct (CAT 6) to the PFSense WAN interface 192.168.1.71.
LAN interface is 192.168.50.4. Only one subnet 192.168.50.0/24

Map:
{{Internet}} --> [ATT modem] (GW:192.168.1.254) --> pfSense WAN (vmbr1:192.168.1.71) --> pfSense LAN (vmbr2:192.168.50.4)

Proxmox host is 7.1-7 and has 5 NICs. One onboard RealTek 1GB nic and a 4 port QNAP PCIe card.


root@proxmox-01:~# root@proxmox-01:~# ip -c a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever

2: enp12s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast master vmbr0 state DOWN group default qlen 1000
link/ether 1c:1b:0d:97:a9:b6 brd ff:ff:ff:ff:ff:ff

3: enp15s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr1 state UP group default qlen 1000
link/ether 24:5e:be:7a:bb:38 brd ff:ff:ff:ff:ff:ff

4: enp16s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr2 state UP group default qlen 1000
link/ether 24:5e:be:7a:bb:37 brd ff:ff:ff:ff:ff:ff

5: enp18s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master vmbr3 state DOWN group default qlen 1000
link/ether 24:5e:be:7a:bb:36 brd ff:ff:ff:ff:ff:ff

6: enp19s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master vmbr4 state DOWN group default qlen 1000
link/ether 24:5e:be:7a:bb:35 brd ff:ff:ff:ff:ff:ff

7: vmbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
link/ether 1c:1b:0d:97:a9:b6 brd ff:ff:ff:ff:ff:ff
inet 192.168.50.3/24 scope global vmbr0
valid_lft forever preferred_lft forever

8: vmbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 24:5e:be:7a:bb:38 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.71/24 scope global vmbr1
valid_lft forever preferred_lft forever
inet6 fe80::265e:beff:fe7a:bb38/64 scope link
valid_lft forever preferred_lft forever

9: vmbr2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 24:5e:be:7a:bb:37 brd ff:ff:ff:ff:ff:ff
inet 192.168.50.5/24 scope global vmbr2
valid_lft forever preferred_lft forever
inet6 fe80::265e:beff:fe7a:bb37/64 scope link
valid_lft forever preferred_lft forever

10: vmbr4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
link/ether 24:5e:be:7a:bb:35 brd ff:ff:ff:ff:ff:ff
inet 192.168.60.1/24 scope global vmbr4
valid_lft forever preferred_lft forever
inet6 fe80::265e:beff:fe7a:bb35/64 scope link
valid_lft forever preferred_lft forever

11: vmbr3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
link/ether 24:5e:be:7a:bb:36 brd ff:ff:ff:ff:ff:ff
inet6 fe80::265e:beff:fe7a:bb36/64 scope link
valid_lft forever preferred_lft forever

2022-08-03 18_11_40-__TRUENAS_adahnert_User Downloads.png

the default route is 192.168.1.254

  
root@proxmox-01:~# ip -c r
default via 192.168.1.254 dev vmbr1 proto kernel onlink
192.168.1.0/24 dev vmbr1 proto kernel scope link src 192.168.1.71
192.168.50.0/24 dev vmbr2 proto kernel scope link src 192.168.50.5
192.168.50.0/24 dev vmbr0 proto kernel scope link src 192.168.50.3 linkdown
192.168.60.0/24 dev vmbr4 proto kernel scope link src 192.168.60.1 linkdown

pfsense configuration:
vtnet0 is 192.168.1.71
vtnet1 is 192.168.50.4

I have turned off the "Offload Checksums" checkmark in Advanced/Networking
I have unchecked the "Private Networks" for both WAN and LAN.
In my rules, I have opened the floodgates with an PASS ANY/ANY just to make sure nothing was blocked.


2.6.0-RELEASE][root@pfSense.fastlane]/root: ifconfig
vtnet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: WAN
        options=800b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE>
        ether 1e:c0:8c:ae:1e:5a
        inet6 fe80::1cc0:8cff:feae:1e5a%vtnet0 prefixlen 64 scopeid 0x1
        inet 192.168.1.71 netmask 0xffffff00 broadcast 192.168.1.255
        media: Ethernet 10Gbase-T <full-duplex>
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
vtnet1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=800b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE>
        ether c2:aa:30:31:13:af
        inet6 fe80::c0aa:30ff:fe31:13af%vtnet1 prefixlen 64 scopeid 0x2
        inet 192.168.50.4 netmask 0xffffff00 broadcast 192.168.50.255
        media: Ethernet 10Gbase-T <full-duplex>
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
enc0: flags=0<> metric 0 mtu 1536
        groups: enc
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=100<PROMISC> metric 0 mtu 33160
        groups: pflog
pfsync0: flags=0<> metric 0 mtu 1500
        groups: pfsync

vesalius · Aug 4, 2022

one issue, is that both proxmox and pfsense are trying to claim 192.168.1.71. I would first change the vmbr1 CIDR to something in the 192.168.1.0/24 subnet but not 192.168.1.71, 192.168.1.72 for example. Removing/changing the vmbr1 CIDR at the Proxmox host network level will not affect pfsense's use of that IP over the vmbr1 bridge.

Next question is do you want proxmox to use the ATT modem as its gateway or pfsense? You can consider removing the gateway from vmbr1 and adding a gateway to vmbr2 using the 192.168.50.4 pfsense IP.

robotdog · Aug 4, 2022

AWESOME! Something so simple, seems to have corrected the problem. Not sure why it didn't make sense to me before. thanks very much vesalius, your help is much appreciated.

For your second question, I "think" I want proxmox to keep the ATT gateway. However, I have an additional ISP that is only for work related items and if I can make it work, would like to use it as a 'fail over' in case the ATT fiber connection goes down. How does that change things?

vesalius · Aug 4, 2022

Easiest, is to set up multiwan failover in pfSense. Plenty of online guides and videos about this. Plug the second isp into an unused physical port on the Proxmox host, set up a linux bridge (could use vmbr3 if nothing else does) with that physical interface (label it WAN2). Give the the Linux bridge to pfSense and then follow a guide to set up multiwan failover.

after that all of your vm/LXC using the pfsense Lan/vmbr2 will get failover without any changes. the Proxmox host would as well if you shift the gateway to vmbr2 as I mentioned was possible above. If you do make this gateway change just be sure to select the option to have the pfSense VM start automatically on boot.

BTW this is mirrrors my own setup except I use an OPNsense VM to handle multiwan failover.

robotdog · Aug 5, 2022

Thanks again vesalius, that is what I'm going to go with then.

Hebert · Oct 13, 2022

Hi, I'm having a similar problem.

I'm new to using proxmox, I'm using virtualized with VirtualBox as a home test environment for a college project, I'm going to implement everything on a real server.

I have an environment with a Pfsense virtual machine and Windows XP.

use this tutorial to do Pfsense configuration on proxmox: LINK

By default, Proxmox puts my enp0s3 interface on BRIDGE vmbr0 as a network interface to access a graphical interface via WEB. The vmbr1 Bridge I created to connect the PFsense LAN interface and the Windows XP machine is working as a virtual switch.

Pfsense config

After configuring the Windows XP machine with PFsense LAN Gateway, I can access PFsense via Windows XP ping the Pfsense LAN and WAN interfaces, but I can't access the internet, as far as I've tested, I can only access the WAN interface.

I also tried to ping through my external network (main network where Proxmox is), either through the machine terminal or through the proxmox terminal and it does not recognize my Pfsense WAN interface. However the WAN interface is taking DHCP ip through my gateway 192.168.0.1

I believe that the problem is similar to that of the colleague above, but I was not able to adapt the solution to my scenario. Could you help me with this?

Search

Search

[SOLVED] pfSense 2.6 unable to ping gateway or beyond

robotdog

New Member

vesalius

Renowned Member

robotdog

New Member

vesalius

Renowned Member

robotdog

New Member

Hebert

New Member

We value your privacy