CentOS 6 VM cannot communicate on the network

[Kenneth_H]

Hi
We are facing some issues with Linux virtual machines on our Proxmox installation.
More specifically, the issue is with VM's running CentOS 6.8.
We have been trying to migrate from Citrix XenServer to Proxmox by using rsync to copy data and configuration.
However we face an issue where a VM cannot communicate with the network or internet when the VM NIC is connected to a network bridge that is based on a Linux Network Bond.


If I fx. install CentOS 7 instead, then it connects with no issues at all.
The issue is not specific to the usage of virtIO, as we have tried then VMXNET3 and Intel E1000 variants as well.
If I connect the VM to a bridge that is directly attached to a network card it works.

I have tried to delete and recreate the VM without success.

One thing that I noticed. Once a CentOS 6 VM has been booted it seems to cause a generic connection issue for all VM's on vmbr1, as I can poweroff any CentOS 7 VM and start it again. After this the VM can also not connect to anything. Rebooting the host solves the issue, but it can be reproduced when the CentOS 6 VM is booted again.

 

[manu]

I am a bit surprised of such a behaviour. Does your CentOS 6 VM has a unique mac address vs other VMs ?

 

[Kenneth_H]

Hi
Yes, it does have a unique MAC-address. I checked the entire network before booting it the first time to make that there were not duplicates.
It seems to do something weird in the network stack when CentOS 6 tries to communicate on the network.
Generally we face an issue with the bonded connection behind vmbr1, where a Linux VM cannot get an IP-address from DHCP, but if I try with a Windows Server 2012 R2 or Windows Server 2016, it does get an IP-address without no issues. In the DHCP-server we can see that it responds to the request, but the response never returns to the VM.

Is there a way for us to troubleshoot the issue? Any logfiles on the host that we can look at?

 

[Kenneth_H]

Hi again
After doing some more testing on the issue, we found that installing CentOS 6 VM on newer hardware the issue does not occur, but upon shutting down and cold booting a CentOS 7 on the primary host, we see the same behaviour.
We are going to test if we can reproduce this again without a CentOS 6 VM booted.

Would knowledge about the two hardware servers help finding the issue?

 

[Kenneth_H]

We have now tested a bit more on this and it seems the network stack on the primary server has a delay of about 30 seconds from network init in the OS until we can communicate.
This also explains why we can reproduce the issue after a reboot on CentOS 7.

Our primary server is an older SuperMicro X8 system with 4x Intel Xeon X7550 processors and all NIC's are using the Intel 82576 network driver.

The other server we tested on does not have this delay on the VM networking. This is a newer Fujitsu RX100S8 with 2x Intel i210 network cards.

 

[manu]

Hi Kenneth

> the network stack on the primary server has a delay of about 30 seconds from network init in the OS until we can communicate.

do you mean here the 30 second delay is occuring on the physical host ( it takes 30s seconds to login via ssh, although the boot process is finished ) or in the VM ( it takes 30 seconds to login via ssh in the VM, although the boot process of the VM has finished )