RAW SCSI device (LTO tape) in qemu

P

paulatz

New Member
Jul 27, 2023
15
2
3
Hello, I have a HPE LTO tape reader+changer connected via a SAS card that I want to use from inside a qemu VM on proxmox.
After a bit of reading, I have been able to connect it like follows (I put here the full process, in case other people are interested) :
On the pve, I find out which are the "raw" scsi devices that correspond to the tape reader and tape changer:
Code: 
# ls -l /dev/sg*|grep tape
crw-rw---- 1 root tape  21,  7 11 avril 14:23 /dev/sg7
crw-rw---- 1 root tape  21,  8 11 avril 14:23 /dev/sg8
In order to find which is the reader and which is the changer, I can check with dmesg, although it is not strictly necessary:
Code: 
# dmesg|grep sg[78]
[101770.048401] st 1:0:2:0: Attached scsi generic sg7 type 1
[101770.721470] ch 1:0:2:1: Attached scsi generic sg8 type 8
So it is st->sg7 (the tape drive) and ch->st8 (the tape changer robot)

I can then add the device by hand in the qemu configuration file, in my case /etc/pve/nodes/pve01/qemu-server/123.conf, by adding the following line (I have the line unrolled it for readability, but it has to be put on a single line, or potentially several "args" lines?):
Code: 
args: -drive id=footape,
        if=none,
        format=raw,
        readonly=off,
        file=/dev/sg7
    -device virtio-scsi-pci,
        id=scsi2 
    -device scsi-generic,
        bus=scsihw0.0,
        drive=footape
    -drive id=foochgr,
        if=none,
        format=raw,
        readonly=off,
        file=/dev/sg8
    -device virtio-scsi-pci,
        id=scsi3
    -device scsi-generic,
        bus=scsihw0.0,
        drive=foochgr

This works perfectly fine, the two devices appear inside the VM and I can use them.
I have, however, two questions:
1. Is I correct to use the same virtual bus for both? Would it make any difference to use different buses?
2. The configuration is a bit fragile, i.e. the number of the devices can change after a reboot. It usually does not, but it will if, for example, I modify the iSCSI drives which are also attached. Is there a way to make this more solid? I could make a script to modify the file during boot, or poke around with udev configuration, but they both seem dangerous to me.
Also, I use scsi virtual devices 2 and 3, because I already have 0 and 1 configured directly from proxmox interface. If I add a new drive in proxmox and forget to update the file by hand, problems could arise. Ideally, a native proxmox supports for raw scsi device could solve all these problems.

Thank you anyway, and I hope this could be useful to someone else.
 
you could also try passing through those two devices as scsi2/3, e.g. like this:

Code: 
qm set XXX -scsi2 /dev/sg7

this will either result in scsi-hd (not what you want), or scsi-generic/scsi-block, depending on the scan logic in /usr/share/perl5/PVE/QemuServer/Drive.pm
 
Thank you Fabian, if I understand correctly what you mean, neither option would work for me, and from a quick test, they do not. The point is that a LTO tape is not a block device, but a character device. I.e. if I setup a loop device, I can attach it in eitehr way:
Code: 
#qm set 128 --scsi2 /dev/loop0,scsiblock=0
update VM 128: -scsi2 /dev/loop0,scsiblock=0
# qm set 128 --scsi3 /dev/loop0,scsiblock=1
update VM 128: -scsi3 /dev/loop0,scsiblock=1
But with a character device, it will not:
Code: 
# qm set 128 --scsi2 /dev/sg7,scsiblock=0
update VM 128: -scsi2 /dev/sg7,scsiblock=0
volume /dev/sg7 does not exist

I.e. it does not work either with the dvd r/w, if I target the raw scsi interface (/dev/sg9) or the dvd-writer device file (/dev/sch0), although it works with the read-only interface, /dev/r0, which is a block device.
 
oh, that seems to be just an error from an attempt to query the size when adding a new disk.. if you want, you could try once more by putting it directly into the config to skip that check? if it works, feel free to file a bug to make that check non-fatal for pass-through disks..
 
Hello, if I naively add
scsi2: /des/sg7
it fails with error
Code: 
kvm: -drive file=/dev/sg7,if=none,id=drive-scsi2,format=raw,cache=none,aio=io_uring,detect-zeroes=on: Could not open '/dev/sg7': filesystem does not support O_DIRECT
TASK ERROR: start failed: QEMU exited with code 1
I think it is the missing "scsi-generic" option that is the key point. Anyway, I'll see to open a bug report after a few more tries.
thank you for your help
 
could you run

Code: 
perl -e 'use strict; use warnings; use PVE::QemuServer::Drive; use Data::Dumper; print Dumper(PVE::QemuServeR::Drive::path_is_scsi("/dev/sg7"))';
perl -e 'use strict; use warnings; use PVE::QemuServer::Drive; use Data::Dumper; print Dumper(PVE::QemuServeR::Drive::path_is_scsi("/dev/sg8"))'

and post the output here?
 
Sure, sorry for the delay, I took a day off:
Code: 
# perl -e 'use strict; use warnings; use PVE::QemuServer::Drive; use Data::Dumper; print Dumper(PVE::QemuServeR::Dri
ve::path_is_scsi("/dev/sg7"))';
Undefined subroutine &PVE::QemuServeR::Drive::path_is_scsi called at -e line 1, <DATA> line 960.
#perl -e 'use strict; use warnings; use PVE::QemuServer::Drive; use Data::Dumper; print Dumper(PVE::QemuServeR::Dri
ve::path_is_scsi("/dev/sg8"))'
Undefined subroutine &PVE::QemuServeR::Drive::path_is_scsi called at -e line 1, <DATA> line 960.

I'm not sure that it went as expected. We're using proxmox 7.4-17, we're planning to upgrade to v8, but probably not before the summer break.
 
paulatz said:
I'm not sure that it went as expected. We're using proxmox 7.4-17, we're planning to upgrade to v8, but probably not before the summer break.
Click to expand...

Yep, we're still running 7.4-17 at work in production as it's been incredibly stable. No issues. Now the dev team just released PVE 8.2.2 (congratz btw!) and was like holy cow. I'm a bit behind. I already upgraded my home lab and test PVE servers to 8.2.2 and it's been running great. Lots of improvements and bug fixes.

I too will upgrade the production servers to 8.2.x series soon to make use of these enhancements.
 
ah, in that case the command is

Code: 
perl -e 'use strict; use warnings; use PVE::QemuServer; use Data::Dumper; print Dumper(PVE::QemuServer::path_is_scsi("/dev/sg7"))';

the code is largely the same, just in a different module ;)
 
and yes, passing through the whole HBA is also an option if it is possible, and probably gives you better performance :)
 
fabian said:
ah, in that case the command is

Code: 
perl -e 'use strict; use warnings; use PVE::QemuServer; use Data::Dumper; print Dumper(PVE::QemuServer::path_is_scsi("/dev/sg7"))';

the code is largely the same, just in a different module ;)
Click to expand...
Hello, here is the output (the device node changed in the meanwhile, following a reboot of the MSL) :
Code: 
# perl -e 'use strict; use warnings; use PVE::QemuServer; use Data::Dumper; print Dumper(PVE::QemuServer::path_is_sc
si("/dev/sg17"))';
$VAR1 = {
'vendor' => 'HPE',
'revision' => '3330',
'product' => 'MSL3040',
'type' => 8,
'removable' => 1
};
root@pve01:~# perl -e 'use strict; use warnings; use PVE::QemuServer; use Data::Dumper; print Dumper(PVE::QemuServer::path_is_sc
si("/dev/sg16"))';
$VAR1 = {
'removable' => 1,
'vendor' => 'HPE',
'product' => 'Ultrium 8-SCSI',
'revision' => 'Q387',
'type' => 1
};


And yes, the biggest problem turned out to be the scsi device number changing, but I wrote a new udev rule to generate stable names for all scsi devices (instead of just tape and disk, which would not include the tape changer), and use those instead. So I have everything working, even if it is not supported in the webgui.
 
so the tape itself (type 1) should be passed through using scsi-generic, but the library has type 8 which is not handled at all right now. feel free to add an enhancement request, but probably passing through the whole HBA makes more sense in any case.
 
You must log in or register to reply here.
Top Bottom