Proxmox / pfSense / VMs and an ARP Table

[panflorek]

Hello All,
I have a problem I don't understand. Probably I miss something simple

I have a simple lab as on the picture below:



The problem is with the VM's "Services" and "Workstation".
Everything works - I can ping across the network, ssh etc.
But..
Whenever I reboot the VM's, i'm loosing network connectivity. I can ping/ssh only across 192.168.3.0/24 subnet.
The only solution to it, is to log into the pfSense and clean the ARP table.



Then the ARP table get's repopulated with the same values and everything works as before.
For example, if I reboot the "Services" VM, I need to clean only the ARP entry for this VM and everything works back again.

Why could this be happening? What might be the possible cause? It drives me craaaaaaazy
I tried both static IPs and DHCP Lease - same issue - need to clean the ARP table by hand after every reboot.

It's possible that it's more related to pfSense itself, but maybe someone came across the same issue already? (yeah I goooooooogled a lot).
EDIT: The same happens with opnsense

 

[Richard]

Whenever I reboot the VM's, i'm loosing network connectivity. I can ping/ssh only across 192.168.3.0/24 subnet.
The only solution to it, is to log into the pfSense and clean the ARP table.

AFAIU access from private "homerouter" subnet 192.168.2.0/24 does not work (access from external is another topic, will only work with packet forwarding). I would trace the packet flow via tcpdump at all nodes which are involved (i.e. lab router, vmbr0@host, pfsense, vmbr1@host and finally inside the VMs) and have a look where packets are lost.

 

[lkiesow]

Whenever I reboot the VM's, i'm loosing network connectivity. I can ping/ssh only across 192.168.3.0/24 subnet.
The only solution to it, is to log into the pfSense and clean the ARP table.

I'm wondering if this is the same problem I described in
https://forum.proxmox.com/threads/invalid-arp-responses-cause-network-problems.118128/

You could try the solution I used there and adjust arp_ignore on your PVE server. To do this temporarily for testing, try running the following command and check if the issue still persists:

echo 2 > /proc/sys/net/ipv4/conf/all/arp_ignore