How to (safely) backup a VM with PCIe passthrough?

[abceleung]

Hi, I have a Nvidia 3090 that is passed through into a Debian VM (for running LLM). I want to perform scheduled backup for this VM.

In Proxmox web UI, I clicked "datacenter", then created a backup job ("stop" mode) that runs at 6am every day. It worked on the first day, but the backup job failed yesterday and I can't boot the VM anymore, so I had to delete the VM and recreated it. I haven't setup the backup job yet.

Is it possible to safely backup a VM with PCIe passthrough devvice? If yes, how do I do it?
Thanks.

Related (sorta): I just read an article from TrueNAS: Yes, You Can (Still) Virtualize TrueNAS. Quote:

When using PCI passthrough, TrueNAS will treat the storage controller just like it’s been installed on physical hardware. This direct access to the PCI device does prevent several advanced features of virtualization from functioning, including (but not limited to) the following:

• Hot adding and removing of virtual devices
• Suspend and resume of the VM
• “Record and replay” functionality
• Fault tolerance and high availability, including “live migration” features
• VM snapshots (note – ZFS snapshots inside of TrueNAS do work)

 

[LnxBil]

Is it possible to safely backup a VM with PCIe passthrough devvice? If yes, how do I do it?

I could imagine an offline backup / stop backup should work. The VM is stopped, backupped and started again.

 

[abceleung]

I could imagine an offline backup / stop backup should work. The VM is stopped, backupped and started again.

That was what I thought too. I will setup stop backup again today and see if it works. Hopefully it works this time.