I have a 3 node cluster. One of the VMs is my gateway, which happens to be pfsense. I use vmbr0 for LAN, as per default, and I create a vmbr1 for the WAN interface. For the purposes of troubleshooting, I have my ONT connected directly to a proxmox node, and I physically move the WAN uplink cable to another proxmox node when I migrate the gateway VM.
The problem is that on 2 of my 3 nodes, pfsense will lose the WAN uplink. I will see something like this in its system.log:
That mac address does not exist on my system, and keep in mind the ONT is connected directly to the node, so it must be coming from the node. It looks like a Supermicro onboard NIC's mac. All three nodes are Supermicro MBs, and the two that have this problem are identical, but none have this mac. It could be a mac address that existed in my cluster at some point, but what is it doing showing up here? (vtnet1 on the pfsense side is vmbr1 on the proxmox side, if that is not obvious). I have installed proxmox from scratch, and pfsense as well, yet this phantom interface returned with the same mac address, to fight with my gateway for the ONT, and bring my WAN down.
Nothing appears in the node's logs when this takes place, until I remove the cable to follow the VM back to the one node that does not have this problem, at which point I see the link go down.
I'm out of ideas. I'm currently running grep on root for the mac address, but I already know it will not show up. Any help is much appreciated.
The problem is that on 2 of my 3 nodes, pfsense will lose the WAN uplink. I will see something like this in its system.log:
Oct 8 20:25:58 pfSense kernel: arp: 00:25:90:ed:70:b6 is using my IP address 173.63.36.16 on vtnet1!
Oct 8 20:29:43 pfSense kernel: arp: 00:25:90:ed:70:b6 is using my IP address 173.63.36.16 on vtnet1!
Oct 8 20:33:29 pfSense kernel: arp: 00:25:90:ed:70:b6 is using my IP address 173.63.36.16 on vtnet1!
Oct 8 20:33:40 pfSense rc.gateway_alarm[69376]: >>> Gateway alarm: WAN_DHCP (Addr:173.63.36.1 Alarm:1 RTT:4.752ms RTTsd:8.147ms Loss:21%)
Oct 8 20:33:40 pfSense check_reload_status[398]: updating dyndns WAN_DHCP
Oct 8 20:33:40 pfSense check_reload_status[398]: Restarting IPsec tunnels
Oct 8 20:33:40 pfSense check_reload_status[398]: Restarting OpenVPN tunnels/interfaces
Oct 8 20:33:40 pfSense check_reload_status[398]: Reloading filter
Oct 8 20:33:41 pfSense php-fpm[369]: /rc.openvpn: Gateway, none 'available' for inet, use the first one configured. 'WAN_DHCP'
Oct 8 20:33:41 pfSense php-fpm[369]: /rc.openvpn: Gateway, none 'available' for inet6, use the first one configured. 'WAN_DHCP6'
Oct 8 20:33:41 pfSense php-fpm[86041]: /rc.dyndns.update: phpDynDNS (
That mac address does not exist on my system, and keep in mind the ONT is connected directly to the node, so it must be coming from the node. It looks like a Supermicro onboard NIC's mac. All three nodes are Supermicro MBs, and the two that have this problem are identical, but none have this mac. It could be a mac address that existed in my cluster at some point, but what is it doing showing up here? (vtnet1 on the pfsense side is vmbr1 on the proxmox side, if that is not obvious). I have installed proxmox from scratch, and pfsense as well, yet this phantom interface returned with the same mac address, to fight with my gateway for the ONT, and bring my WAN down.
Nothing appears in the node's logs when this takes place, until I remove the cable to follow the VM back to the one node that does not have this problem, at which point I see the link go down.
I'm out of ideas. I'm currently running grep on root for the mac address, but I already know it will not show up. Any help is much appreciated.
Last edited: