[SOLVED] Hot swapping physical disks passed through to a VM

S

sesquipedality

New Member
May 19, 2020
12
3
3
45
I have a virtual server running in qemu on proxmox. It has two physical disks attached to the host using a virtio: directive in 101.cfg as folllows

virtio2: /dev/disk/by-id/ata-vol1,backup=0
virtio3: /dev/disk/by-id/ata-vol2,backup=0

These contain a mirrored ZFS volume from which my files are served using nextcloud. This all works fine because these disks are never removed from the guest.

I also have a rotation of three other physical disks to which I back up (the physical controller is an LSI raid controller in JBOD configuration (the physical hardware is an HP Proiliant DL380e). I do this by zfs sending an updated snapshot from the storage array to whatever backup disk is present on the system. I had added these to the VM as follows:

virtio4: /dev/disk/by-id/ata-backup1-volname,backup=0
virtio5: /dev/disk/by-id/ata-backup2-volname,backup=0
virtio6: /dev/disk/by-id/ata-backup3-volname,backup=0

This sort of works but the problem is that none of those disks are hot swappable, so if I remove and insert disks in rotation, the guest stops being able to see them. I have disk hotplug enabled in my qemu config.

My question is what is the correct way to dynamically attach and detach these disks to my VM as they are inserted and removed? Thanks.
 
Last edited:
hi,

sesquipedality said:
This sort of works but the problem is that none of those disks are hot swappable, so if I remove and insert disks in rotation, the guest stops being able to see them. I have disk hotplug enabled in my qemu config.
Click to expand...

virtio disks can be hotplugged and hot-unplugged (i have just tested this on an alpine linux guest )

how are you removing the disks?

you can go to VM -> Hardware -> virtio[X] -> Detach to detach a disk when the VM is running, and afterwards double click the unused disk and add it back and it should work.
 
Thanks for the response. I am exporting the pool (automatically in the backup script) in the guest and then pulling the disk.

Detaching via the GUI seems to result in the drive config simply disappearing - possibly because the GUI doesn't really support the addition of virtio passthrough devices (although since 6.2 the entries do appear in the GUI once you have created them).

Ideally I'm looking for a solution I can run non-interactively when a disk is inserted or removed.
 
I have managed to find a partial answer. I can use "qm monitor" to attach and detach the disk I am taking out by attaching and detaching virtio-scsi "pci cards" as follows:

To attach

qm> drive_add 0 file=/dev/disk/by-id/ata-<disk-id>,if=none,id=backup_vol,cache=none,detect-zeroes=on
qm> device_add virtio-blk-pci,drive=backup_vol,id=backup_scsi_controller

To detach

qm> device_del backup_scsi_controller

It doesn't seem to be possible however to run qm monitor commands from a shell script, and qm monitor will not take arguments or input from stdin. It might be possible to create a socket into qemu-agent and inject this using JSON, but I know nothing about JSON and am really hoping there is an easier way.

(Source: https://blog.chrishowie.com/2019/09/19/hot-swapping-virtio-disks-on-qemu/ via Google cache)
 
Can anyone help me with how to pass these commands to qemu in a script via a socket, please? (Since I think that's the only way this will work.)
 
in qemu source scripts/qmp/qmp-shell can work for you to send commands to the unix socket
 
I have been somewhat around the houses on this one, and indeed did get to the point where I was able to send QMP commands to a guest via a UNIX socket. This is supported by the default Proxmox config using a socket at /var/run/qemu-server/<server-id>.qmp. I can then interactively send QMP JSON messages using

rlwrap -C qmp socat - UNIX:/var/run/qemu-server/101.qmp

(rlwrap is not installed by default, but can be installed using "apt install rlwrap")

However, the QMP message format was as opaque as I thought it was going to be and has no direct equivalent to "drive_add" for reasons of ideological purity (which are perhaps understandable but nonetheless hugely unhelpful in the context of my present difficulty). There is a "blockdev_add" command that may well have done what I wanted assuming I could work out the correct syntax.

I therefore took a step back and came to the conclusion that "qm monitor" probably wasn't accepting input from stdin because of readline support. This lead me to think of "expect". A quick "apt install expect" later, and I was able to create scripts to attach and detach drives as follows:

qemu-drive-attach <vm-id> <device-name> <path-to-block-device>

with qemu-drive-attach script (device-name is arbitrary - I use "virtio8" for consistency with the proxmox naming scheme)

Code: 
!/usr/bin/expect
set vm_id               [lindex $argv 0];
set device_name         [lindex $argv 1];
set device_file         [lindex $argv 2];

spawn qm monitor $vm_id

expect "qm> "
send "drive_add 0 file=$device_file,if=none,id=drive-$device_name,cache=none,detect-zeroes=on\r"

expect "OK" {
    expect "qm> "
    send "device_add virtio-blk-pci,drive=drive-$device_name,id=$device_name\r"
}

expect "qm> "
send "quit\r"

send_user "\n"

And for detaching

qemu-drive-detach <vm-id> <device-name>

with qemu-drive-detach script

Code: 
#!/usr/bin/expect
set vm_id               [lindex $argv 0];
set device_name         [lindex $argv 1];

spawn qm monitor $vm_id
expect "qm> "
send "device_del $device_name\r"

expect "qm> "
send "quit\r"

send_user "\n"

Error checking with these scripts leaves something to be desired, particularly as device_add and device_del are silent on success, but this will enable external block device drives to be hotswapped into the VM from a script.
 
Last edited:
You must log in or register to reply here.
Top Bottom