Losing migration would be extremely inconvenient, however it seems to be the only practical solution.
Under regular load, the latency between us and the remote site stays between 2 and 4ms, however under heavy load that increases to as much as 20.
This is most likely the leading cause of all...