I'm trying to get these both to pass-through to a VM:
Proxmox is installed on a 32GB USB flash drive.
Added the Proxmox repo. Repo contents not included because my account is too new.
Did a full upgrade. All packages and the kernel.
END
Updated GRUB with:
Then ran 'update-grub'
Added the modules:
END
Then rebooted PVE.
Created the VM.
Modified the conf file to be the following:
END
Boot the VM and observe 1x of the LSI PCI controllers on the VM!
This specific controller doesn't have any disks.
I need to attach the second controller that has the disks.
Stopped the VM.
Modify the conf file to be the following *both* LSI controllers:
END
Starting the VM fails with this:
END
Revert the conf to only 'hostpci0: 26:00' and it boots.
Add the second card and it won't start.
Set the conf to only the second card with the disks 'hostpci0: 27:00' and get this same error:
END
I've also tried 'hostpci0: 27:00,pcie=1' and it would not boot due to the same error.
Configured it to 'hostpci0: 27:00,pcie=1,driver=vfio' and the VM boots!
Where is the card though? It's not showing during the VM boot process...
Tested the first card with the same config: hostpci0: 27:00,pcie=1,driver=vfio
The VM boots but the card is not showing during VM boot.
This message:
I can't find anything beyond developer email threads. Nothing concerning what should be done. Hence posting here.
How can I pass-through the LSI controller card with the disks?
How can I pass-through both LSI controller cards?
Please let me know if any additional data is necessary.
Code:
root@pve:~# lspci | grep LSI
26:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)
27:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)
root@pve:~#Proxmox is installed on a 32GB USB flash drive.
Added the Proxmox repo. Repo contents not included because my account is too new.
Did a full upgrade. All packages and the kernel.
Code:
root@pve:~# pveversion -v
proxmox-ve: 4.4-82 (running kernel: 4.4.40-1-pve)
pve-manager: 4.4-12 (running version: 4.4-12/e71b7a74)
pve-kernel-4.4.35-1-pve: 4.4.35-77
pve-kernel-4.4.40-1-pve: 4.4.40-82
lvm2: 2.02.116-pve3
corosync-pve: 2.4.2-1
libqb0: 1.0-1
pve-cluster: 4.0-48
qemu-server: 4.0-109
pve-firmware: 1.1-10
libpve-common-perl: 4.0-92
libpve-access-control: 4.0-23
libpve-storage-perl: 4.0-76
pve-libspice-server1: 0.12.8-2
vncterm: 1.3-1
pve-docs: 4.4-3
pve-qemu-kvm: 2.7.1-4
pve-container: 1.0-94
pve-firewall: 2.0-33
pve-ha-manager: 1.0-40
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u3
lxc-pve: 2.0.7-3
lxcfs: 2.0.6-pve1
criu: 1.6.0-1
novnc-pve: 0.5-8
smartmontools: 6.5+svn4324-1~pve80
zfsutils: 0.6.5.9-pve15~bpo80
root@pve:~#Updated GRUB with:
Code:
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"Then ran 'update-grub'
Added the modules:
Code:
root@pve:~# cat /etc/modules
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
root@pve:~#Then rebooted PVE.
Created the VM.
Modified the conf file to be the following:
Code:
root@pve:~# cat /etc/pve/qemu-server/101.conf
bootdisk: scsi0
cores: 1
ide2: none,media=cdrom
memory: 512
name: pci-passthru
numa: 0
ostype: l26
scsi0: local-zfs:vm-101-disk-1,size=2G
scsihw: virtio-scsi-pci
smbios1: uuid=e3edf130-dbbd-4a92-805e-ebad36b457f5
sockets: 1
machine: q35
hostpci0: 26:00
root@pve:~#Boot the VM and observe 1x of the LSI PCI controllers on the VM!
This specific controller doesn't have any disks.
I need to attach the second controller that has the disks.
Stopped the VM.
Modify the conf file to be the following *both* LSI controllers:
Code:
root@pve:~# cat /etc/pve/qemu-server/101.conf
bootdisk: scsi0
cores: 1
ide2: none,media=cdrom
memory: 512
name: pci-passthru
numa: 0
ostype: l26
scsi0: local-zfs:vm-101-disk-1,size=2G
scsihw: virtio-scsi-pci
smbios1: uuid=e3edf130-dbbd-4a92-805e-ebad36b457f5
sockets: 1
machine: q35
hostpci0: 26:00;27:00
root@pve:~#Starting the VM fails with this:
Code:
kvm: -device vfio-pci,host=27:00.0,id=hostpci0,bus=pci.0,addr=0x10: vfio: Hardware reports invalid configuration, MSIX PBA outside of specified BAR
kvm: -device vfio-pci,host=27:00.0,id=hostpci0,bus=pci.0,addr=0x10: Device initialization failed
TASK ERROR: start failed: command '/usr/bin/kvm -id 101 -chardev 'socket,id=qmp,path=/var/run/qemu-server/101.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -pidfile /var/run/qemu-server/101.pid -daemonize -smbios 'type=1,uuid=e3edf130-dbbd-4a92-805e-ebad36b457f5' -name pci-passthru -smp '1,sockets=1,cores=1,maxcpus=1' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vga cirrus -vnc unix:/var/run/qemu-server/101.vnc,x509,password -cpu kvm64,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce -m 512 -k en-us -readconfig /usr/share/qemu-server/pve-q35.cfg -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' -device 'vfio-pci,host=27:00.0,id=hostpci0,bus=pci.0,addr=0x10' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:302b94ebab' -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -device 'virtio-scsi-pci,id=scsihw0,bus=pci.0,addr=0x5' -drive 'file=/dev/zvol/rpool/data/vm-101-disk-1,if=none,id=drive-scsi0,format=raw,cache=none,aio=native,detect-zeroes=on' -device 'scsi-hd,bus=scsihw0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=100' -machine 'type=q35'' failed: exit code 1Revert the conf to only 'hostpci0: 26:00' and it boots.
Add the second card and it won't start.
Set the conf to only the second card with the disks 'hostpci0: 27:00' and get this same error:
Code:
kvm: -device vfio-pci,host=27:00.0,id=hostpci0,bus=pci.0,addr=0x10: vfio: Hardware reports invalid configuration, MSIX PBA outside of specified BAR
kvm: -device vfio-pci,host=27:00.0,id=hostpci0,bus=pci.0,addr=0x10: Device initialization failed
TASK ERROR: start failed: command '/usr/bin/kvm -id 101 -chardev 'socket,id=qmp,path=/var/run/qemu-server/101.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -pidfile /var/run/qemu-server/101.pid -daemonize -smbios 'type=1,uuid=e3edf130-dbbd-4a92-805e-ebad36b457f5' -name pci-passthru -smp '1,sockets=1,cores=1,maxcpus=1' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vga cirrus -vnc unix:/var/run/qemu-server/101.vnc,x509,password -cpu kvm64,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce -m 512 -k en-us -readconfig /usr/share/qemu-server/pve-q35.cfg -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' -device 'vfio-pci,host=27:00.0,id=hostpci0,bus=pci.0,addr=0x10' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:302b94ebab' -drive 'if=none,id=drive-ide2,media=cdrom,aio=threads' -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200' -device 'virtio-scsi-pci,id=scsihw0,bus=pci.0,addr=0x5' -drive 'file=/dev/zvol/rpool/data/vm-101-disk-1,if=none,id=drive-scsi0,format=raw,cache=none,aio=native,detect-zeroes=on' -device 'scsi-hd,bus=scsihw0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=100' -machine 'type=q35'' failed: exit code 1I've also tried 'hostpci0: 27:00,pcie=1' and it would not boot due to the same error.
Configured it to 'hostpci0: 27:00,pcie=1,driver=vfio' and the VM boots!
Where is the card though? It's not showing during the VM boot process...
Tested the first card with the same config: hostpci0: 27:00,pcie=1,driver=vfio
The VM boots but the card is not showing during VM boot.
This message:
Code:
vfio: Hardware reports invalid configuration, MSIX PBA outside of specified BARI can't find anything beyond developer email threads. Nothing concerning what should be done. Hence posting here.
How can I pass-through the LSI controller card with the disks?
How can I pass-through both LSI controller cards?
Please let me know if any additional data is necessary.