[SOLVED] Can't migrate running vm with mapped resources

[MisterDeeds]

Dear all

I have two Proxmox servers, each with an identical GPU. For this I have created an entry under the Resource Mappings.


I then passed this mapping to a VM as PCI passthrough. However, the VM cannot be migrated in online mode and must be switched off.


Is this generally not possible or am I doing something wrong?

Thank you and best regards

 

[leesteken]

I have two Proxmox servers, each with an identical GPU. For this I have created an entry under the Resource Mappings.

I then passed this mapping to a VM as PCI passthrough. However, the VM cannot be migrated in online mode and must be switched off.

Is this generally not possible or am I doing something wrong?

I'm almost sure that migration of a running VM with PCIe passthrough is not possible, as KVM/QEMU would need to save the GPU state and recreate it on the other node (highly device specific) or transfer PCIe command between nodes (high unlikely).

 

[MisterDeeds]

I'm almost sure that migration of a running VM with PCIe passthrough is not possible, as KVM/QEMU would need to save the GPU state and recreate it on the other node (highly device specific) or transfer PCIe command between nodes (high unlikely).

Ohh, all right. Too bad but thank you for the explanation! BR

 

[colnetwork]

Hello, is there a possibility for these type of cases where PCIe passtrough is being utilized that pve could shutrdown the VM first and then migrate it to the other identical host? this for example in a scenario were the hosting node goes down unexpectedly. Thanks

 

[MrPete]

How certain are you that the other host is 100% identical? Exact same devices in every PCIe slot? Same firmware versions? (I can kinda guarantee the S/N's are different but that is hopefully never an issue.)

You can certainly come close even if PVE won't directly do it:

• Shut down VM
• Edit the VM *.conf file to comment out the mapped device
• Migrate to new host
• Edit to uncomment (ie activate)
• Off you go

 

[leesteken]

Exact same devices in every PCIe slot?

That problem can be solved with mapped devices, if the same devices are used. No need to edit the VM configuration.

Same firmware versions? (I can kinda guarantee the S/N's are different but that is hopefully never an issue.)

If the VM is shut down and restarted on another node (with mapped devices), and the device are identical or similar enough, then it should work in principle as no active state needs to be transferred.

 

[MrPete]

That problem can be solved with mapped devices, if the same devices are used. No need to edit the VM configuration.

We're talking PCIe Pass Through here. The VM configuration includes the exact PCIe slot. Here's what the *.conf line it looks like for an nVidia GPU... with the final .* removed so both the GPU and audio devices are passed through (required for the nVidia driver/installer to recognize the card):

hostpci0: 0000:01:00,pcie=1