Passing through a LSI SAS Controller

jackydany

Member
Jul 20, 2016
26
1
23
43
Hi,

after fixing my W10 OVMF problems with the help of this forum, i am continuing my installtion.

I want to passthrough a LSI HBA SAS Controller 9201-16e to a freenas VM.
I flashed the controller with the latest Firmware, IT Mode. Bios is empty. So its just a HBA in IT Mode passing through as JBOD.
I tested with a hardware Freenas Device, working properly. As devices i am using a Netapp DS4243 with a sbb2-601 controller for the sff-8088 ports.
I inserted a sata drive as well as 2 sas drives and all of them are recognized on the Freenas Hardware.
Also they are recognized in Proxmox properly and i coulid add a disk if i would like to.

But when i try to passing it to the VM, its not working and i cant see why.

Code:
cat /etc/pve/qemu-server/105.conf
bios: ovmf
bootdisk: scsi0
cores: 4
efidisk0: wonderland-zfs:vm-105-disk-0,size=1M
hostpci0: 82:00,pcie=1,rombar=0
machine: q35
memory: 16384
name: freetest
net0: virtio=06:0A:D3:D7:61:C6,bridge=vmbr0,firewall=1
numa: 0
ostype: other
scsi0: local-zfs:vm-105-disk-0,size=32G
scsihw: virtio-scsi-pci
smbios1: uuid=4f625084-0e02-4102-8789-66af8ebc1ee2
sockets: 2
vmgenid: cd5cc229-4786-41f9-8d4e-bcc8315e172e

lspci:
82:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2116 PCI-Express Fusion-MPT SAS-2 [Meteor] (rev 02)

Code:
pveversion -v
proxmox-ve: 6.1-2 (running kernel: 5.3.18-3-pve)
pve-manager: 6.1-8 (running version: 6.1-8/806edfe1)
pve-kernel-helper: 6.1-8
pve-kernel-5.3: 6.1-6
pve-kernel-5.3.18-3-pve: 5.3.18-3
pve-kernel-5.3.18-2-pve: 5.3.18-2
pve-kernel-5.3.10-1-pve: 5.3.10-1
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.0.3-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 2.0.1-1+pve8
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.15-pve1
libpve-access-control: 6.0-6
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.0-17
libpve-guest-common-perl: 3.0-5
libpve-http-server-perl: 3.0-5
libpve-storage-perl: 6.1-5
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 3.2.1-1
lxcfs: 4.0.1-pve1
novnc-pve: 1.1.0-1
openvswitch-switch: 2.12.0-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.1-3
pve-cluster: 6.1-4
pve-container: 3.0-23
pve-docs: 6.1-6
pve-edk2-firmware: 2.20200229-1
pve-firewall: 4.0-10
pve-firmware: 3.0-7
pve-ha-manager: 3.0-9
pve-i18n: 2.0-4
pve-qemu-kvm: 4.1.1-4
pve-xtermjs: 4.3.0-1
qemu-server: 6.1-7
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-1
zfsutils-linux: 0.8.3-pve1


I already passed through a AMD Radeon GPU (2 different ones at the same time) to my new W10 VM. Thats working.
I am passing through 2 USB Sticks to my iobroker VM. Working like hell :D

I serached a lot but everything i tried didnt succeed.
I also tried romfile=path for the romfile of the card. No success. rombar=1 also. CPU = host same.


Would be nice to get this running. But i dont know where i am stuck :D

Thanks and happy easter to all of you!
 
Code:
dmesg | grep Adding
[    2.239195] pci 0000:00:00.0: Adding to iommu group 0
[    2.239553] pci 0000:00:01.0: Adding to iommu group 1
[    2.239809] pci 0000:00:02.0: Adding to iommu group 2
[    2.240213] pci 0000:00:02.2: Adding to iommu group 3
[    2.240505] pci 0000:00:03.0: Adding to iommu group 4
[    2.240911] pci 0000:00:03.2: Adding to iommu group 5
[    2.241242] pci 0000:00:05.0: Adding to iommu group 6
[    2.241277] pci 0000:00:05.2: Adding to iommu group 6
[    2.241308] pci 0000:00:05.4: Adding to iommu group 6
[    2.241667] pci 0000:00:11.0: Adding to iommu group 7
[    2.241983] pci 0000:00:1a.0: Adding to iommu group 8
[    2.242351] pci 0000:00:1c.0: Adding to iommu group 9
[    2.242772] pci 0000:00:1c.7: Adding to iommu group 10
[    2.243242] pci 0000:00:1d.0: Adding to iommu group 11
[    2.243516] pci 0000:00:1e.0: Adding to iommu group 12
[    2.245740] pci 0000:00:1f.0: Adding to iommu group 13
[    2.245782] pci 0000:00:1f.2: Adding to iommu group 13
[    2.245814] pci 0000:00:1f.3: Adding to iommu group 13
[    2.246082] pci 0000:01:00.0: Adding to iommu group 14
[    2.246549] pci 0000:03:00.0: Adding to iommu group 15
[    2.246596] pci 0000:03:00.1: Adding to iommu group 15
[    2.246878] pci 0000:04:00.0: Adding to iommu group 16
[    2.247229] pci 0000:04:00.1: Adding to iommu group 17
[    2.247478] pci 0000:06:00.0: Adding to iommu group 18
[    2.247823] pci 0000:06:00.1: Adding to iommu group 19
[    2.248071] pci 0000:06:00.2: Adding to iommu group 20
[    2.248471] pci 0000:06:00.3: Adding to iommu group 21
[    2.248844] pci 0000:07:00.0: Adding to iommu group 22
[    2.249103] pci 0000:08:00.0: Adding to iommu group 23
[    2.249492] pci 0000:08:00.1: Adding to iommu group 24
[    2.249773] pci 0000:0a:00.0: Adding to iommu group 25
[    2.250233] pci 0000:7f:08.0: Adding to iommu group 26
[    2.250281] pci 0000:7f:08.3: Adding to iommu group 26
[    2.250314] pci 0000:7f:08.4: Adding to iommu group 26
[    2.250627] pci 0000:7f:09.0: Adding to iommu group 27
[    2.250663] pci 0000:7f:09.3: Adding to iommu group 27
[    2.250697] pci 0000:7f:09.4: Adding to iommu group 27
[    2.251136] pci 0000:7f:0a.0: Adding to iommu group 28
[    2.251172] pci 0000:7f:0a.1: Adding to iommu group 28
[    2.251205] pci 0000:7f:0a.2: Adding to iommu group 28
[    2.251238] pci 0000:7f:0a.3: Adding to iommu group 28
[    2.251540] pci 0000:7f:0b.0: Adding to iommu group 29
[    2.251576] pci 0000:7f:0b.3: Adding to iommu group 29
[    2.252085] pci 0000:7f:0c.0: Adding to iommu group 30
[    2.252121] pci 0000:7f:0c.1: Adding to iommu group 30
[    2.252155] pci 0000:7f:0c.2: Adding to iommu group 30
[    2.252189] pci 0000:7f:0c.6: Adding to iommu group 30
[    2.252224] pci 0000:7f:0c.7: Adding to iommu group 30
[    2.252621] pci 0000:7f:0d.0: Adding to iommu group 31
[    2.252659] pci 0000:7f:0d.1: Adding to iommu group 31
[    2.252695] pci 0000:7f:0d.2: Adding to iommu group 31
[    2.252730] pci 0000:7f:0d.6: Adding to iommu group 31
[    2.253008] pci 0000:7f:0e.0: Adding to iommu group 32
[    2.253044] pci 0000:7f:0e.1: Adding to iommu group 32
[    2.253532] pci 0000:7f:0f.0: Adding to iommu group 33
[    2.253571] pci 0000:7f:0f.1: Adding to iommu group 33
[    2.253609] pci 0000:7f:0f.2: Adding to iommu group 33
[    2.253646] pci 0000:7f:0f.3: Adding to iommu group 33
[    2.253684] pci 0000:7f:0f.4: Adding to iommu group 33
[    2.253721] pci 0000:7f:0f.5: Adding to iommu group 33
[    2.253758] pci 0000:7f:0f.6: Adding to iommu group 33
[    2.254192] pci 0000:7f:10.0: Adding to iommu group 34
[    2.254232] pci 0000:7f:10.1: Adding to iommu group 34
[    2.254277] pci 0000:7f:10.2: Adding to iommu group 34
[    2.254316] pci 0000:7f:10.3: Adding to iommu group 34
[    2.254354] pci 0000:7f:10.4: Adding to iommu group 34
[    2.254393] pci 0000:7f:10.5: Adding to iommu group 34
[    2.254432] pci 0000:7f:10.6: Adding to iommu group 34
[    2.254472] pci 0000:7f:10.7: Adding to iommu group 34
[    2.254865] pci 0000:7f:11.0: Adding to iommu group 35
[    2.255231] pci 0000:7f:13.0: Adding to iommu group 36
[    2.255270] pci 0000:7f:13.1: Adding to iommu group 36
[    2.255309] pci 0000:7f:13.4: Adding to iommu group 36
[    2.255350] pci 0000:7f:13.5: Adding to iommu group 36
[    2.255392] pci 0000:7f:13.6: Adding to iommu group 36
[    2.255780] pci 0000:80:00.0: Adding to iommu group 37
[    2.256023] pci 0000:80:01.0: Adding to iommu group 38
[    2.256378] pci 0000:80:01.1: Adding to iommu group 39
[    2.256623] pci 0000:80:02.0: Adding to iommu group 40
[    2.256984] pci 0000:80:03.0: Adding to iommu group 41
[    2.257303] pci 0000:80:05.0: Adding to iommu group 42
[    2.257349] pci 0000:80:05.2: Adding to iommu group 42
[    2.257390] pci 0000:80:05.4: Adding to iommu group 42
[    2.257731] pci 0000:82:00.0: Adding to iommu group 43
[    2.258167] pci 0000:84:00.0: Adding to iommu group 44
[    2.258220] pci 0000:84:00.1: Adding to iommu group 44
[    2.258528] pci 0000:85:00.0: Adding to iommu group 45
[    2.258580] pci 0000:85:00.1: Adding to iommu group 45
[    2.258997] pci 0000:ff:08.0: Adding to iommu group 46
[    2.259043] pci 0000:ff:08.3: Adding to iommu group 46
[    2.259088] pci 0000:ff:08.4: Adding to iommu group 46
[    2.259402] pci 0000:ff:09.0: Adding to iommu group 47
[    2.259448] pci 0000:ff:09.3: Adding to iommu group 47
[    2.259495] pci 0000:ff:09.4: Adding to iommu group 47
[    2.259950] pci 0000:ff:0a.0: Adding to iommu group 48
[    2.259996] pci 0000:ff:0a.1: Adding to iommu group 48
[    2.260044] pci 0000:ff:0a.2: Adding to iommu group 48
[    2.260088] pci 0000:ff:0a.3: Adding to iommu group 48
[    2.260379] pci 0000:ff:0b.0: Adding to iommu group 49
[    2.260424] pci 0000:ff:0b.3: Adding to iommu group 49
[    2.260869] pci 0000:ff:0c.0: Adding to iommu group 50
[    2.260915] pci 0000:ff:0c.1: Adding to iommu group 50
[    2.260960] pci 0000:ff:0c.2: Adding to iommu group 50
[    2.261005] pci 0000:ff:0c.6: Adding to iommu group 50
[    2.261051] pci 0000:ff:0c.7: Adding to iommu group 50
[    2.261395] pci 0000:ff:0d.0: Adding to iommu group 51
[    2.261442] pci 0000:ff:0d.1: Adding to iommu group 51
[    2.261492] pci 0000:ff:0d.2: Adding to iommu group 51
[    2.261538] pci 0000:ff:0d.6: Adding to iommu group 51
[    2.261954] pci 0000:ff:0e.0: Adding to iommu group 52
[    2.262003] pci 0000:ff:0e.1: Adding to iommu group 52
[    2.262522] pci 0000:ff:0f.0: Adding to iommu group 53
[    2.262572] pci 0000:ff:0f.1: Adding to iommu group 53
[    2.262619] pci 0000:ff:0f.2: Adding to iommu group 53
[    2.262665] pci 0000:ff:0f.3: Adding to iommu group 53
[    2.262713] pci 0000:ff:0f.4: Adding to iommu group 53
[    2.262761] pci 0000:ff:0f.5: Adding to iommu group 53
[    2.262810] pci 0000:ff:0f.6: Adding to iommu group 53
[    2.263255] pci 0000:ff:10.0: Adding to iommu group 54
[    2.263308] pci 0000:ff:10.1: Adding to iommu group 54
[    2.263356] pci 0000:ff:10.2: Adding to iommu group 54
[    2.263405] pci 0000:ff:10.3: Adding to iommu group 54
[    2.263455] pci 0000:ff:10.4: Adding to iommu group 54
[    2.263506] pci 0000:ff:10.5: Adding to iommu group 54
[    2.263555] pci 0000:ff:10.6: Adding to iommu group 54
[    2.263604] pci 0000:ff:10.7: Adding to iommu group 54
[    2.264002] pci 0000:ff:11.0: Adding to iommu group 55
[    2.264379] pci 0000:ff:13.0: Adding to iommu group 56
[    2.264430] pci 0000:ff:13.1: Adding to iommu group 56
[    2.264481] pci 0000:ff:13.4: Adding to iommu group 56
[    2.264530] pci 0000:ff:13.5: Adding to iommu group 56
[    2.264578] pci 0000:ff:13.6: Adding to iommu group 56
[   16.566400] ipmi_si: Adding SMBIOS-specified kcs state machine
[   16.566941] ipmi_si: Adding ACPI-specified kcs state machine
 
Did you ever figure this out? I'm looking to do the same thing, without passing individual disks. Guess I'll go back to google.

EDIT: Found this and it worked great! https://pve.proxmox.com/wiki/Pci_passthrough
Had to disable boot options on the HBA before VirtIO SCSI would load the os.
That's it!
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!