Major network issues after live migration

[Weidah]

I can't seem to figure this out. I've tried everything, flushing ARP tables, trying to change MAC address (which works occasionally) and so on.

I've got a cluster of two machines.
One machine with OPNSense is using SR-IOV (Not migrating that obviously)
The other machines using linux bridges.
Everything is set up to be identical.

I migrate the machine to the second host, and it can ping EVERYTHING on the network BUT the OPNSense gateway.
There's got to be some strange ARP issue going on or something that the first host still thinks that the migrated machine is connected to it's bridge.

I've tried flushing it, checking that there is ARP messages going out with Wireshark. The first machine just doesn't seem to care?
The most interesting thing is I can't even ping the machines using SR-IOV.


And then out of the blue, they'll work for no obvious reason. I have no idea why.
Like just now I try to ping google.com and I get a response and the gateway is accessible on OPNSense
I migrated an Ubuntu machine and it just works. Has internet. Can ping google.com gateway accessible on OPNSense
I migrate it back. Doesn't work again.


This happens even if I turn off the machine and then migrate.

 

[Weidah]

Completely confused now. Seems it has a lot to do with SR-IOV and bridge forwarding again... Didn't do enough testing to prove this is the issue. But seems more testing is ahead of me.

But for now it seems if I move the VMs to a separate bridge which doesn't have a SR-IOV PF of my 10GBnic in it there's no issues.

Any way to run a script after a migration?