Problems Importing from ESXi

[Verulam]

Hi there

I've run into a very odd situation. My first import of a Linux box from ESXi (7.0.3) to PVE (8.2.4) ran perfectly using the wizard. However when trying other hosts I quickly ran into trouble with various error messages such as:

Connection error - Timeout.

or

create storage failed: cfs-lock 'file-storage_cfg' error: got lock request timeout (500)

or

is not a vim server

I couldn't work out why these other imports failed when my first attempt succeeded - I even re-imported it again to verify. I can see on the ESXi hosts that the PVE login is accepted as root. I've made the following config changes on ESXi:

........
<soap>
<sessionTimeout>0</sessionTimeout>
<maxSessionCount>0</maxSessionCount>
</soap>
........

I looked for variables that were different like hardware, ESXi exact version, size of VM, time of day, atmospheric pressure etc but couldn't see an issue. Then, when looking at the size of mounted datastores on the ESXi hosts (which I don't think is a factor) I noticed that the number of VMs on the host was different.

My first migration was from a host with only two VMs on it. All my other attempts were on hosts with at least 10 VMs. So after migrating some workloads I found the following: that PVE can connect successfully to ESXi as long as the number of VMs on ESXi is 4 or less. As soon as I have 5 VMs on ESXi it errors.

I know PVE isn't searching the many TBs of datastores for VMX files (and it fails far too quickly anyway) but I can't see why having 5 VMs on an ESXi host would spook it?

Was wondering if anyone else had some across this?

Thanks.

 

[dakralex]

Thank you for your thorough description of your problem. Unfortunately I could not recreate your problem (in respect to the amount of VMs on ESXi) on a local setup. I would be interested if there is something of interest in your syslog for the ESXi fuse mount point (or PVE storage and esxi-import-tool in general). Have you checked if there the manifest.json in /run/pve/import/esxi/<storage-name>/ looks correct to you? Have you tried to reconnect to the mount point (e.g. recreating the storage, remounting)?

 

[dakralex]

Another idea that just crossed my mind, if it's possible: Have you tried to restart the ESXi hosts? If it's not possible, have you tried to restart the management agents on the ESXi hosts as described in [1]?

[1] https://knowledge.broadcom.com/external/article/320280/restarting-the-management-agents-in-esxi.html

 

[Verulam]

Thanks for getting back to me @dakralex , I'm not surprised you cant recreate the problem - it has to be something odd at our end...

I'll take a look at the manifest & import directory. Yes, I have tried to reconnect to the storage as well as restart hostd and reboot hosts.

I went back to my first guest that successfully imported and migrated it to another host - it then failed to import. Migrating VMs that didn't import to the first guest's host did allow them to import. So it is something to do with some of our ESXi host's config which is another breadcrumb to help work out what is going on.

Also gives me a temporary workaround to get some VMs off ESXi and into PVE which is what we're trying to achieve.