How to make file based pipes of the host available as serial devices to the guest using hot plugging?

ams_tschoening

New Member
Jun 28, 2021
22
6
3
43
I host multiple different VMs running MySQL and Postgres databases and have a snapshots based backup of their data files in place already. That is, the host accesses the VMs using SSH, created snapshots of the contained file system, mounts those, mounts the while file system using SSHFS at the host and reads all files of interest using a tool named BorgBackup.

While that works in general, I would like to additionally create dumps e.g. using "pg_dump" and back those up by the host as well. The important point ist that I would like to avoid additionally storing the dumps within the VM, because for some VMs that would simply take too much space. Backup times would increase as well: The dump would need to be written within the VM, read from there afterwards etc. The host regularly creates snapshots using "zfs-auto-snapshot" as well, so a lot of changes within the VMs might be preserved longer than actually useful.

OTOH, BorgBackup allows to read special files like FIFOs as if they were normal files. Some additional tool named BorgMatic uses that behavior to exactly do what I want, only within VMs, which is not how my setup works currently.

--read-specialopen and read block and char device files as well as FIFOs as if they were regular files. Also follows symlinks pointing to these kinds of files.

I've already did some research and found that QEMU supports named pipes in the host as backends for virtual serial devices within the VM and that reads exactly like what I need in the end: Create some pipes in the host, pass them through into the VMs, let "pg_dump" write into the serial ports in the VMs and let BorgBackup/BorgMatic read those special pipes in the host.

Bash:
[host]# mkfifo /tmp/serial0
[host]# qemu-kvm -enable-kvm -m 1024 -nographic \
            -device virtio-serial-pci -chardev pipe,id=ch0,path=/tmp/serial0
            -device virtserialport,chardev=ch0,name=serial0 \
            -drive file=disk.img,if=virtio,boot=on

What I don't understand yet is how to make those virtual serial ports available when the VM is already running?

The above example code is for starting the VM and when looking at the shell tools "qm" or "pvesh" I didn't recognize any corresponding options to reconfigure a VM to have some virtual serial device at runtime. The only thing in that direction I was able to find so far was the following which looks like sending commands to the QEMU guest agent?

Bash:
echo '{ "execute": "qmp_capabilities" }
{ "execute" : "chardev-add", "arguments" : { "id" : "charch1", "backend" : { "type" : "socket", "data" : {"wait": false, "server": true, "addr": { "type": "unix", "data": { "path": "tty.sock" } } } } } }
{ "execute": "device_add", "arguments": { "driver": "virtserialport", "chardev":"charch1", "id": "channel1", "name": "sh.hyper.channel.1" } }' |
socat - unix-connect:hypervisor.sock

The reason I want to add/delete the necessary pipes on runtime is that I would like to first retrieve all database names in some VM and then create the necessary pipes, devices etc. based on this. This way I don't need to restart VMs with new pipes/devices only because some database has been added, which can easily be forgotten as well.

Any help where I would need to look at would be appreciated, thanks!
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!