Limit on Number of Pass Through SATA Drives

[Mallard612]

I've been searching for quite some time and haven't found a clear answer. There was a question from 2012, but the answer was that the limit on the number of SATA drives was going away. So I need some help/clarification.

I'm in the process of deploying a home server with Proxmox. I'll be using Fedora as my Docker host and utilizing Rockstor to manage my storage needs. I have a mix of 10 different sized HDD's which is why I'm looking at btrfs. When I modify the .conf file for the Rockstor VM, I'm only able to pass 5 HDD's via SATA. When I tried SCSI, I could only pass 2. So my question is, is there a limit on the number of pass-through drives and, if not, what should the configuration look like to pass all of the HDD's via SATA into the VM? Rockstor currently requires that they are passed via SATA.

I've been adding them by using the following:

nano /etc/pve/nodes/proxmox/qemu-server/114.conf

I then modify the config to include:

sata0: /dev/disk/by-id/{serial number}
sata1: /dev/disk/by-id/{serial number}, etc.

 

[dietmar]

See the manual page:

# man qm.conf

You can have up to 6 SATA devices (stata0-satat5).

 

[ratbuddy]

Any chance this could be revisited? I'm passing through 6 SATA drives to a guest, but would like to pass the full 8 which my motherboard supports. Is this an arbitrary or technical limitation?

 

[SebRider]

Hello,

I am also interested.
Is it possible to pass the limit to 8 or more via an order or a configuration file?
Indeed my virtualized device does not support anything other than SATA.

Thank you
(Sorry for my translation)

 

[SebRider]

Hi,

I have a soluce !

args: -device ahci,id=ahci1,multifunction=on,bus=pci.0,addr=0xb -drive 'file=/dev/disk/by-id/ata-ST500DM009-2F110A_ZA480LWF,if=none,id=drive-sata6,format=raw,aio=threads,detect-zeroes=unmap' -device ide-hd,bus=ahci1.1,drive=drive-sata6,id=sata6

Found on another forum.

 

[Smash_Override]

Sorry for the newb question. I am new to linux and proxmox. What am I doing wrong?

I'm running Proxmox VE 7.2-3 and just like everyone else, trying to pass a hard drive from SATA6 through to a TrueNAS VM.

Am I supposed to add that line of code as a new line inside the .conf file? I assumed I was and now the VM wont start. I did change the hard drive name and the line of code I added looked like this...

args: -device ahci,id=ahci1,multifunction=on,bus=pci.0,addr=0xb -drive 'file=/dev/disk/by-id/{ata-ST12000NM0008-2H3101_ZHZ1F9PC},if=none,id=drive-sata6,format=raw,aio=threads,detect-zeroes=unmap' -device ide-hd,bus=ahci1.1,drive=drive-sata6,id=sata6

I deleted the line of code and it started up no problem. Thanks for the help in advance.

 

[tig3r]

Perhaps follow this guide, it worked fine for me.
https://dannyda.com/2020/08/26/how-to-passthrough-hdd-ssd-physical-disks-to-vm-on-proxmox-vepve/ or the official
https://pve.proxmox.com/wiki/Passthrough_Physical_Disk_to_Virtual_Machine_(VM)

Also even tho Sata is limited between 0-5 (n) you could use Scsi that supports 30 drives

 

[Smash_Override]

WOOOOOOOOOOOO!!!!!!!!! Just followed the official proxmox instruction and WHAMMY! There it is. So just to make sure I am understanding what is really happeneing....Proxmox basically build a virtual scsi interface that controls that SATA port?

Thanks for the help! I may have gotten way to excited the moment I saw the drive show up in my VM lol.

 

[Dunuin]

Disk passthrough isn't showing the VM the real physical disk. The VM is still working with a virtual disk, just that the blocks of that disk will be mapped to that physical disk. So VM writes to virtual disk, QEMU does some virtualization magic and writes that to the physical disk. As the VM isn't interacting with the real physical disk, it is no problem to use a virtual IDE or SCSI bus with physical SATA disks, as QEMU does the translation.

 

[tig3r]

Disk passthrough isn't showing the VM the real physical disk. The VM is still working with a virtual disk, just that the blocks of that disk will be mapped to that physical disk. So VM writes to virtual disk, QEMU does some virtualization magic and writes that to the physical disk. As the VM isn't interacting with the real physical disk, it is no problem to use a virtual IDE or SCSI bus with physical SATA disks, as QEMU does the translation.

I do find that quite logical to understand, but when you add encryption to the mix I have trouble to wrap my head around it.

I have a few drives that holds data, the drives comes from a bare metal Ubuntu and the drives are encrypted with luks.
PVE can of course see the drives but not the partitions, nor mount them since they are encrypted. The passthrough to my VM works fine and in the VM I do decrypt and mount as intended.


Just having hard time understanding the mechanics of QEMU with the encrypted drives that PVE sees

 

[Dunuin]

It doesn't need to know anything about partitions and so on, as it just works on the block level. You got 512B sectors on the virtual disk and these get mapped to the (probably 4K) sectors of your physical disk. So both should contain the same bits when reading the disks from first to last sector.

 

[gaiex]

Hi,

I have a soluce !

args: -device ahci,id=ahci1,multifunction=on,bus=pci.0,addr=0xb -drive 'file=/dev/disk/by-id/ata-ST500DM009-2F110A_ZA480LWF,if=none,id=drive-sata6,format=raw,aio=threads,detect-zeroes=unmap' -device ide-hd,bus=ahci1.1,drive=drive-sata6,id=sata6

Found on another forum.

Hello all,

After adding the "args" to the VM conf. file the drives apear in the VM when booted but not at hardware section on the proxmox VM.

Any thing missing besside the "args" line ?

Thanks

 

[Grantman]

No. That is how it works (in my experience)

I have been able to add additional SATA devices using "args" lines and they do not show up under Hardware. There is a soft limit on SATA devices at 6 which is reflected in the hardware configuration page and the args line manually added to the VM config file is just a work around.

I did not find any issues using 4 additional SATA devices (10 total) on a heavily used file server but I would still consider this an unsupported configuration at best.

Good luck


Grantman

 

[gaiex]

No. That is how it works (in my experience)

I have been able to add additional SATA devices using "args" lines and they do not show up under Hardware. There is a soft limit on SATA devices at 6 which is reflected in the hardware configuration page and the args line manually added to the VM config file is just a work around.

I did not find any issues using 4 additional SATA devices (10 total) on a heavily used file server but I would still consider this an unsupported configuration at best.

Good luck


Grantman

Thanks for the reply,

I've added a total of 12 SATA drives to my VM in two separate AHCI SATA args onf the VM.conf.
This way I could use 12 disks passthrough to my VM from my netapp shelf.

 

[cpa.vipin]

Thanks for the reply,

I've added a total of 12 SATA drives to my VM in two separate AHCI SATA args onf the VM.conf.
This way I could use 12 disks passthrough to my VM from my netapp shelf.

I want to add 8 sata drives to a VM. I was able to add 6th sata drive, however for 7th and 8th I am not sure what to values need to edit to below args.
Any help on this will be greatly appreciated.

args: -device ahci,id=ahci1,multifunction=on,bus=pci.0,addr=0xb -drive 'file=/dev/disk/by-id/ata-xx-xx,if=none,id=drive-sata6,format=raw,aio=threads,detect-zeroes=unmap' -device ide-hd,bus=ahci1.1,drive=drive-sata6,id=sata6

 

[ti9errr]

What parameter to set for a SSD?

 

[solonion]

I want to add 8 sata drives to a VM. I was able to add 6th sata drive, however for 7th and 8th I am not sure what to values need to edit to below args.
Any help on this will be greatly appreciated.

args: -device ahci,id=ahci1,multifunction=on,bus=pci.0,addr=0xb -drive 'file=/dev/disk/by-id/ata-xx-xx,if=none,id=drive-sata6,format=raw,aio=threads,detect-zeroes=unmap' -device ide-hd,bus=ahci1.1,drive=drive-sata6,id=sata6

You can try something like this:

args: -device ahci,id=ahci1,multifunction=on,bus=pci.0,addr=0xb -drive 'file=/dev/disk/by-id/ata-ST16000NM000J-2TW103_ZR5DKGFB,if=none,id=drive-sata6,format=raw,aio=threads,detect-zeroes=unmap' -device ide-hd,bus=ahci1.0,drive=drive-sata6,id=sata6 -drive 'file=/dev/disk/by-id/ata-ST16000NM000J-2TW103_ZR5D694X,if=none,id=drive-sata7,format=raw,aio=threads,detect-zeroes=unmap' -device ide-hd,bus=ahci1.1,drive=drive-sata7,id=sata7 -drive 'file=/dev/disk/by-id/ata-ST16000NM000J-2TW103_ZR60X3P0,if=none,id=drive-sata8,format=raw,aio=threads,detect-zeroes=unmap' -device ide-hd,bus=ahci1.2,drive=drive-sata8,id=sata8

 

[Zorbatron]

Instead of passing through all of the drives individually, it would be better to pass through the while SATA controller. (Unless your boot drive is attached to it...)

 

[gohan]

You can try something like this:

args: -device ahci,id=ahci1,multifunction=on,bus=pci.0,addr=0xb -drive 'file=/dev/disk/by-id/ata-ST16000NM000J-2TW103_ZR5DKGFB,if=none,id=drive-sata6,format=raw,aio=threads,detect-zeroes=unmap' -device ide-hd,bus=ahci1.0,drive=drive-sata6,id=sata6 -drive 'file=/dev/disk/by-id/ata-ST16000NM000J-2TW103_ZR5D694X,if=none,id=drive-sata7,format=raw,aio=threads,detect-zeroes=unmap' -device ide-hd,bus=ahci1.1,drive=drive-sata7,id=sata7 -drive 'file=/dev/disk/by-id/ata-ST16000NM000J-2TW103_ZR60X3P0,if=none,id=drive-sata8,format=raw,aio=threads,detect-zeroes=unmap' -device ide-hd,bus=ahci1.2,drive=drive-sata8,id=sata8

Thank you. The soluion worked for me. I also have a new question. Is there anyway to enable ssd emulation for these drive?

I've tried but got this error

kvm: -drive file=/dev/mapper/pve-vm--100--disk--1,if=none,id=drive-sata6,ssd=1,format=raw,aio=threads,detect-zeroes=unmap: Block format 'raw' does not support the option 'ssd'
TASK ERROR: start failed: QEMU exited with code 1

and remove the format=raw

WARNING: Image format was not specified for '/dev/mapper/pve-vm--100--disk--1' and probing guessed raw.
         Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions.
kvm: -drive file=/dev/mapper/pve-vm--100--disk--1,if=none,id=drive-sata6,ssd=1,aio=threads,detect-zeroes=on: Block format 'raw' does not support the option 'ssd'
TASK ERROR: start failed: QEMU exited with code 1