[SOLVED] VMs loosing network after some time

[semira uthsala]

Hi all,

I'm running a 4 node pve cluster (6-1.3) with ovs installed. All networks configured using ovs and all interfaces are LACP bonded.

OVS BOND (LACP) -> OVS BRIDGE -> VMS
                -> Intports

Recently I noticed Windows VMs using intel E1000 drivers disconnecting after some time. When I disable and enable network interface from windows it starts again. Then I switched this to virtio driver and keep monitoring

And again today, VM that recently moved from another platform having the same issue, This time Debian 8 Jessie. But this VM transfer a lot of traffic (20GB+) every hour since this is used for speed testing. When I restart the networking within the VM it works again. (systemctl restart networking). I tried to change driver from virtio to intelE1000 without any luck

This VMs also have multiple security groups enabled with PVE firewall

below are my bond and bridge configuration

allow-vmbr1 bond3
iface bond3 inet manual
        ovs_bonds eno1 eno2
        ovs_type OVSBond
        ovs_bridge vmbr1
        ovs_options bond_mode=balance-tcp lacp=active

allow-ovs vmbr1
iface vmbr1 inet manual
        ovs_type OVSBridge
        ovs_ports bond3 intport1230

 

[semira uthsala]

Update on this.

I did dist-upgrade from Debian 8 to Debian 9 and my disconnect issue is fixed. I did a continuous ping to gateways and I can see sometimes dropping one or two packets but not disconnect completely as before

Networking of Windows VMs with virtio drivers so far stable. I'll keep monitor and posted

 

[semira uthsala]

2nd update on this.

Today I moved another VM to pve. This time RHEL7. after some time VM loses the connectivity as before. Never change or update anything from the VM side. using virtio driver with qemu-guest-agent enabled. we are planning to go full production with proper licensing and all. but this issue slowing us down. Need to solve this issue as soon as possible

 

[semira uthsala]

I found this is due to network bug on our setup that another platform (openstack neutron) keeps holding the IPs even after those instances are shutdown. I'll remove my post as it's not really ovs issues. my setup working perfectly fine now

 

[christensensbt]

Do you think your issue is related to this https://forum.proxmox.com/threads/kvm-windows-vms-and-losing-network-connectivity.46763/?

It sounds similar, other than the fact that you've experienced it on Linux Guests and not just Windows

 

[semira uthsala]

Do you think your issue is related to this https://forum.proxmox.com/threads/kvm-windows-vms-and-losing-network-connectivity.46763/?

It sounds similar, other than the fact that you've experienced it on Linux Guests and not just Windows

No. This is not proxmox issue at all. This happened due to openstck neutron keep holding the IP addresses even after instance shutdown. So lose of connection due to ip conflict.

And the windows VM network issue with intel NIC still there. Not sure how I can fix it since I'm running some software that spesifically requird intel NICs. all the windows VMs change to virtio driver running smoothly

 

[christensensbt]

No. This is not proxmox issue at all. This happened due to openstck neutron keep holding the IP addresses even after instance shutdown. So lose of connection due to ip conflict.

And the windows VM network issue with intel NIC still there. Not sure how I can fix it since I'm running some software that spesifically requird intel NICs. all the windows VMs change to virtio driver running smoothly

Oh ok, I just noticed that you had already posted in that other thread as well. My mistake.

 

[mykeebot]

Hello @semira uthsala,

Are you sure about that it was not a Proxmox issue/bug? I'm experiencing the same with Proxmox 8.0 with a Rock Linux 8 VM. In case of a certain time of inactivity the VM drops the network and I cannot reach it. I have to restart it from Proxmox, it solves the problem. I added some details to this ticket...

https://forum.proxmox.com/threads/c...amount-time-of-inactivity.136330/#post-604366

Thanks,
Mike