Hi José,
I just want to confirm my understanding, correct me in any point where I am wrong:
1) Three buildings, each one holds exactly one node (server), in your logs it was marked srv1, srv2, srv3;
2) in each of those 3 cases, your network setup is identical, meaning:
3) from a node there are 2 metallic GbE connections to the same switch "in the cabinet" for that particular node - these are not link aggregate, they are completely different subnets and routed separately later on;
4) from the switch "in the cabinet", you have a OS2 (?) fibre to another switch "for the inter-building connections";
5) this second switch "for the inter-buidling connections" is located at the central building, the other two buildings also connect to it via OS2(?) fibre;
6) the inter-building connections are your own fibre cable, at no point does the traffic route through the public internet, not even via VPN.
Correct me in the above where I got it wrong - I am sure I would not have guessed it all correctly.
Also can you post ping times inbetween the srv[1-3]? Meaning from srv1 to srv2, from srv1 to srv3, BUT ALSO from srv2 to srv3?
Also, as you have two interfaces for each node, can you do the ping for both of the subnets?
Finally,
if you can afford to do that, can you keep one of the pings running (e.g. srv1 to srv2) while you manually reboot the switch "for the inter-building connections" and post it as well?