How do I only enable hook script when spinning up VMs but not for snapshot backups

[ndejay]

Hi,

I am running a PVE 6.3 server with several VMs with PCIe passthrough (GPU, USB, etc.). I'm using a hook script for a few of these VMs (as a hookscript: ... in /etc/pve/qemu-server/XXX.conf). I am also running regular backups (snapshot mode) of these VMs using PVE, but I would like the hook script NOT to be run then. Is there a simple way to conditionally run the hook script ONLY when I launch VMs outside of backups?

Thank you!

 

[Alwin Antreich]

The hook only understands the phases, as seen in the link. Though, those aren't specific to the condition of the start. But the backup in snapshot mode doesn't trigger a cold-start.
https://git.proxmox.com/?p=pve-docs...e403f1b0d1021aca4a111e57036c5e4114cb5;hb=HEAD

 

[ndejay]

Thank you, that makes sense. Do you by any chance if there is a way to conditionally passthough PCIe depending on whether a VM is cold-started or snapshotted (as in a backup)?

 

[Alwin Antreich]

or snapshotted (as in a backup)?

The snapshot mode for the VM backup doesn't restart the VM. No anew PCIe passthrough needs to be done.
https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_backup_modes

What is the actual issue you are facing during backup?

 

[ndejay]

The issue is that I am unable to run a backup of a VM while another is running.

The VMs in question are (1) a Windows 10 VM and (2) an OS X VM. I am doing a PCIe passthrough of a GPU and USB controller to both, such that only of these two VMs can be run at one time.

While there is no issue with the GPU passthrough, the PCIe USB controller gets initialized by the PVE host too early, so I am using the following hook script to release it from the PVE host so that it can be used on either VM using VFIO:

#!/usr/bin/env bash


if [ "$2" == "pre-start" ]
then


echo "Calling Inateck KT4004 VFIO script."


# First release devices from their current driver (by their PCI bus IDs)
echo 0000:08:00.0 > /sys/bus/pci/devices/0000:08:00.0/driver/unbind


# Then attach them by ID to VFIO
echo 1b73 1100 > /sys/bus/pci/drivers/vfio-pci/new_id
fi

When the OS X VM is running, and a backup for the Windows 10 VM runs, the hook script hangs, presumably because the PCIe USB controller is currently in use by the OS X VM. This problem only occurs when the OS X VM is running.

INFO: Starting Backup of VM 100 (qemu)
INFO: Backup started at 2021-02-07 10:59:19
INFO: status = stopped
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: VM Name: windows10
INFO: include disk 'scsi0' 'local-lvm:vm-100-disk-0' 128G
INFO: include disk 'efidisk0' 'local-lvm:vm-100-disk-2' 4M
INFO: creating vzdump archive '/mnt/pve/slim-backup/dump/vzdump-qemu-100-2021_02_07-10_59_19.vma.zst'
INFO: starting kvm to execute backup task
Calling Inateck KT4004 VFIO script.

I would like to be able to back up the Windows 10 VM while the OS X VM is running (and vice-versa). I also don't understand why the backup mode is set to "stop" in the above example, as I clearly set the mode to snapshot (see below). Is it running in "stop" mode because the VM is down?



Thank you!

 

[Alwin Antreich]

Is it running in "stop" mode because the VM is down?

The KVM process will start for the vzdump backup, the VM doesn't actually boot though.

The VMs in question are (1) a Windows 10 VM and (2) an OS X VM. I am doing a PCIe passthrough of a GPU and USB controller to both, such that only of these two VMs can be run at one time.

The script will need to check if the device has already been removed. If so, then it skips the unbind and passthrough. Otherwise the issue will arise on any start of both VMs.

 

[ndejay]

Thanks, that makes a lot of sense! Do you know if there's a way to do a conditional passthrough at the KVM configuration?

$ cat /etc/pve/qemu-server/100.conf
...
hostpci1: 08:00.0,pcie=1
...

 

[Alwin Antreich]

The VM config can't be changed at that stage, it's locked. You would need to try and play with the qemu monitor commands.

 

[ndejay]

Gotcha! Then it doesn't sound like there's a way to automate this w/o any manual intervention i.e. turning off one VM (or play around with qemu monitor) before a backup job starts, right?

 

[Alwin Antreich]

Monitor commands could be executed by the hook script. But I am not sure if the PCI passthrough is adjustable on a running VM.

When you script the whole startup (no GUI), then you could use qm set to change the VM config and run it.

 

[ndejay]

Thank you for your help, I'll try to implement this and report back!