Maybe some "energy saving" setting?... I'm shooting blindly, just saw a neighbour thread https://forum.proxmox.com/threads/network-drops-out-randomly.166414/
Edit: ... or bad network interface... I can see posts about issues with some Intel NICs. I don't know your NIC, but you may try to search...