LSI SAS card passthrough error

F

flipper203

Member
Jun 24, 2019
7
0
21
42
Hello,

I am trying to passthrough my LSI SAS9211-4i PCIE card to a VM on my brand new proxmox installation. I followed the official guide https://pve.proxmox.com/wiki/Pci_passthrough
My setup is a HP microserver gen8 with a 1260l xeon, last proxmox instaleld on 2 SSD (in zfs mirror) connected to the internal HP controller. On the LSI card I have the four storage drives installed.

Everything is going well until I try to start the VM, regarding the parameters to change to set the passthrough. I get the foillowing error and the VM don't boot:
Code: 
Virtual Environment 5.4-6
    Virtual Machine 103 (2pFreenas) on node 'pve'
    Logs
    ()
    kvm: -device vfio-pci,host=07:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0: vfio error: 0000:07:00.0: failed to setup container for group 1: failed to set iommu for container: Operation not permitted
    TASK ERROR: start failed: command '/usr/bin/kvm -id 103 -name 2pFreenas -chardev 'socket,id=qmp,path=/var/run/qemu-server/103.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' -mon 'chardev=qmp-event,mode=control' -pidfile /var/run/qemu-server/103.pid -daemonize -smbios 'type=1,uuid=0c2366f3-9576-45ce-9f92-508441ea3ed5' -smp '2,sockets=1,cores=2,maxcpus=2' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vnc unix:/var/run/qemu-server/103.vnc,x509,password -cpu kvm64,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce -m 10192 -device 'vmgenid,guid=06ceb1f6-d7db-4cec-9025-d70335dbe1b2' -readconfig /usr/share/qemu-server/pve-q35.cfg -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' -device 'vfio-pci,host=07:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0' -device 'VGA,id=vga,bus=pcie.0,addr=0x1' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:a95eca963df1' -device 'virtio-scsi-pci,id=scsihw0,bus=pci.0,addr=0x5' -drive 'file=/dev/zvol/rpool/data/vm-103-disk-0,if=none,id=drive-scsi0,format=raw,cache=none,aio=native,detect-zeroes=on' -device 'scsi-hd,bus=scsihw0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap103i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=4E:77:B9:7A:E5:19,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' -machine 'type=q35'' failed: exit code 1

If I add the following option driver=vfio for the hostpci0, the VM starts but I get the followwing problem:
Code: 
vm 103 - unable to parse value of 'hostpci0' - format error
    driver: property is not defined in schema and the schema does not allow additional properties

I also had to put the options vfio_iommu_type1 allow_unsafe_interrupts=1

Any help on this?
 
So, I followed the above link and I still have an error:

Code: 
root@pve:~# dmesg | grep -e DMAR -e IOMMU -e AMD-Vi
[ 0.000000] ACPI: DMAR 0x00000000F1DE4A80 0003B4 (v01 HP ProLiant 00000001 \xffffffd2? 0000162E)
[ 0.000000] DMAR: IOMMU enabled
[ 0.000000] DMAR: Host address width 39
[ 0.000000] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
[ 0.000000] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap c9008020660262 ecap f010da
[ 0.000000] DMAR: RMRR base: 0x000000f1ffd000 end: 0x000000f1ffffff
[ 0.000000] DMAR: RMRR base: 0x000000f1ff6000 end: 0x000000f1ffcfff
[ 0.000000] DMAR: RMRR base: 0x000000f1f93000 end: 0x000000f1f94fff
[ 0.000000] DMAR: RMRR base: 0x000000f1f8f000 end: 0x000000f1f92fff
[ 0.000000] DMAR: RMRR base: 0x000000f1f7f000 end: 0x000000f1f8efff
[ 0.000000] DMAR: RMRR base: 0x000000f1f7e000 end: 0x000000f1f7efff
[ 0.000000] DMAR: RMRR base: 0x000000000f4000 end: 0x000000000f4fff
[ 0.000000] DMAR: RMRR base: 0x000000000e8000 end: 0x000000000e8fff
[ 0.000000] DMAR: RMRR base: 0x000000f1dee000 end: 0x000000f1deefff
[ 0.000000] DMAR-IR: IOAPIC id 8 under DRHD base 0xfed90000 IOMMU 0
[ 0.000000] DMAR-IR: HPET id 0 under DRHD base 0xfed90000
[ 0.000000] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit.
[ 0.000000] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting.
[ 0.000000] DMAR-IR: Enabled IRQ remapping in xapic mode
[ 0.720514] DMAR: No ATSR found
[ 0.720827] DMAR: dmar0: Using Queued invalidation
[ 0.720835] DMAR: Setting RMRR:
[ 0.721122] DMAR: Setting identity map for device 0000:01:00.0 [0xf1dee000 - 0xf1deefff]
[ 0.721440] DMAR: Setting identity map for device 0000:01:00.2 [0xf1dee000 - 0xf1deefff]
[ 0.721749] DMAR: Setting identity map for device 0000:01:00.4 [0xf1dee000 - 0xf1deefff]
[ 0.722076] DMAR: Setting identity map for device 0000:00:1f.2 [0xe8000 - 0xe8fff]
[ 0.722387] DMAR: Setting identity map for device 0000:07:00.0 [0xe8000 - 0xe8fff]
[ 0.722718] DMAR: Setting identity map for device 0000:03:00.0 [0xe8000 - 0xe8fff]
[ 0.723031] DMAR: Setting identity map for device 0000:03:00.1 [0xe8000 - 0xe8fff]
[ 0.723360] DMAR: Setting identity map for device 0000:04:00.0 [0xe8000 - 0xe8fff]
[ 0.723369] DMAR: Setting identity map for device 0000:01:00.0 [0xe8000 - 0xe8fff]
[ 0.723379] DMAR: Setting identity map for device 0000:01:00.2 [0xe8000 - 0xe8fff]
[ 0.723390] DMAR: Setting identity map for device 0000:00:1f.2 [0xf4000 - 0xf4fff]
[ 0.723392] DMAR: Setting identity map for device 0000:07:00.0 [0xf4000 - 0xf4fff]
[ 0.723394] DMAR: Setting identity map for device 0000:03:00.0 [0xf4000 - 0xf4fff]
[ 0.723395] DMAR: Setting identity map for device 0000:03:00.1 [0xf4000 - 0xf4fff]
[ 0.723396] DMAR: Setting identity map for device 0000:04:00.0 [0xf4000 - 0xf4fff]
[ 0.723398] DMAR: Setting identity map for device 0000:01:00.0 [0xf4000 - 0xf4fff]
[ 0.723399] DMAR: Setting identity map for device 0000:01:00.2 [0xf4000 - 0xf4fff]
[ 0.723401] DMAR: Setting identity map for device 0000:00:1f.2 [0xf1f7e000 - 0xf1f7efff]
[ 0.723410] DMAR: Setting identity map for device 0000:07:00.0 [0xf1f7e000 - 0xf1f7efff]
[ 0.723418] DMAR: Setting identity map for device 0000:03:00.0 [0xf1f7e000 - 0xf1f7efff]
[ 0.723427] DMAR: Setting identity map for device 0000:03:00.1 [0xf1f7e000 - 0xf1f7efff]
[ 0.723436] DMAR: Setting identity map for device 0000:04:00.0 [0xf1f7e000 - 0xf1f7efff]
[ 0.723445] DMAR: Setting identity map for device 0000:01:00.0 [0xf1f7e000 - 0xf1f7efff]
[ 0.723450] DMAR: Setting identity map for device 0000:01:00.2 [0xf1f7e000 - 0xf1f7efff]
[ 0.723455] DMAR: Setting identity map for device 0000:00:1f.2 [0xf1f7f000 - 0xf1f8efff]
[ 0.723457] DMAR: Setting identity map for device 0000:07:00.0 [0xf1f7f000 - 0xf1f8efff]
[ 0.723459] DMAR: Setting identity map for device 0000:03:00.0 [0xf1f7f000 - 0xf1f8efff]
[ 0.723460] DMAR: Setting identity map for device 0000:03:00.1 [0xf1f7f000 - 0xf1f8efff]
[ 0.723462] DMAR: Setting identity map for device 0000:04:00.0 [0xf1f7f000 - 0xf1f8efff]
[ 0.723464] DMAR: Setting identity map for device 0000:01:00.0 [0xf1f7f000 - 0xf1f8efff]
[ 0.723466] DMAR: Setting identity map for device 0000:01:00.2 [0xf1f7f000 - 0xf1f8efff]
[ 0.723468] DMAR: Setting identity map for device 0000:00:1f.2 [0xf1f8f000 - 0xf1f92fff]
[ 0.723470] DMAR: Setting identity map for device 0000:07:00.0 [0xf1f8f000 - 0xf1f92fff]
[ 0.723472] DMAR: Setting identity map for device 0000:03:00.0 [0xf1f8f000 - 0xf1f92fff]
[ 0.723473] DMAR: Setting identity map for device 0000:03:00.1 [0xf1f8f000 - 0xf1f92fff]
[ 0.723475] DMAR: Setting identity map for device 0000:04:00.0 [0xf1f8f000 - 0xf1f92fff]
[ 0.723476] DMAR: Setting identity map for device 0000:01:00.0 [0xf1f8f000 - 0xf1f92fff]
[ 0.723478] DMAR: Setting identity map for device 0000:01:00.2 [0xf1f8f000 - 0xf1f92fff]
[ 0.723480] DMAR: Setting identity map for device 0000:00:1f.2 [0xf1f93000 - 0xf1f94fff]
[ 0.723481] DMAR: Setting identity map for device 0000:07:00.0 [0xf1f93000 - 0xf1f94fff]
[ 0.723483] DMAR: Setting identity map for device 0000:03:00.0 [0xf1f93000 - 0xf1f94fff]
[ 0.723484] DMAR: Setting identity map for device 0000:03:00.1 [0xf1f93000 - 0xf1f94fff]
[ 0.723486] DMAR: Setting identity map for device 0000:04:00.0 [0xf1f93000 - 0xf1f94fff]
[ 0.723487] DMAR: Setting identity map for device 0000:01:00.0 [0xf1f93000 - 0xf1f94fff]
[ 0.723489] DMAR: Setting identity map for device 0000:01:00.2 [0xf1f93000 - 0xf1f94fff]
[ 0.723490] DMAR: Setting identity map for device 0000:01:00.0 [0xf1ff6000 - 0xf1ffcfff]
[ 0.723492] DMAR: Setting identity map for device 0000:01:00.2 [0xf1ff6000 - 0xf1ffcfff]
[ 0.723493] DMAR: Setting identity map for device 0000:01:00.4 [0xf1ff6000 - 0xf1ffcfff]
[ 0.723810] DMAR: Setting identity map for device 0000:00:1a.0 [0xf1ffd000 - 0xf1ffffff]
[ 0.724160] DMAR: Setting identity map for device 0000:00:1d.0 [0xf1ffd000 - 0xf1ffffff]
[ 0.724172] DMAR: Prepare 0-16MiB unity mapping for LPC
[ 0.724482] DMAR: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[ 0.724573] DMAR: Intel(R) Virtualization Technology for Directed I/O
[ 1734.255087] vfio-pci 0000:07:00.0: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor.

Code: 
root@pve:~# find /sys/kernel/iommu_groups/ -type l
/sys/kernel/iommu_groups/7/devices/0000:00:1c.7
/sys/kernel/iommu_groups/5/devices/0000:00:1c.4
/sys/kernel/iommu_groups/13/devices/0000:01:00.4
/sys/kernel/iommu_groups/13/devices/0000:01:00.2
/sys/kernel/iommu_groups/13/devices/0000:01:00.0
/sys/kernel/iommu_groups/13/devices/0000:01:00.1
/sys/kernel/iommu_groups/3/devices/0000:00:1a.0
/sys/kernel/iommu_groups/11/devices/0000:03:00.0
/sys/kernel/iommu_groups/11/devices/0000:03:00.1
/sys/kernel/iommu_groups/1/devices/0000:00:01.0
/sys/kernel/iommu_groups/1/devices/0000:07:00.0
/sys/kernel/iommu_groups/8/devices/0000:00:1d.0
/sys/kernel/iommu_groups/6/devices/0000:00:1c.6
/sys/kernel/iommu_groups/4/devices/0000:00:1c.0
/sys/kernel/iommu_groups/12/devices/0000:04:00.0
/sys/kernel/iommu_groups/2/devices/0000:00:06.0
/sys/kernel/iommu_groups/10/devices/0000:00:1f.2
/sys/kernel/iommu_groups/10/devices/0000:00:1f.0
/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/9/devices/0000:00:1e.0

Code: 
root@pve:~# cat /etc/pve/qemu-server/103.conf
balloon: 4096
bootdisk: scsi0
cores: 2
hostpci0: 07:00,pcie=on
machine: q35
memory: 10192
name: 2pFreenas
net0: virtio=4E:77:B9:7A:E5:19,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
parent: afterInitFreenas
scsi0: local-zfs:vm-103-disk-0,size=32G
scsihw: virtio-scsi-pci
smbios1: uuid=0c2366f3-9576-45ce-9f92-508441ea3ed5
sockets: 1
vmgenid: 06ceb1f6-d7db-4cec-9025-d70335dbe1b2

[afterInitFreenas]
balloon: 4096
bootdisk: scsi0
cores: 2
ide2: local:iso/FreeNAS-11.2-U4.1.iso,media=cdrom
memory: 10192
name: 2pFreenas
net0: virtio=4E:77:B9:7A:E5:19,bridge=vmbr0,firewall=1
numa: 0
onboot: 1
ostype: l26
runningmachine: pc-i440fx-3.0
scsi0: local-zfs:vm-103-disk-0,size=32G
scsihw: virtio-scsi-pci
smbios1: uuid=0c2366f3-9576-45ce-9f92-508441ea3ed5
snaptime: 1558993031
sockets: 1
vmgenid: 06ceb1f6-d7db-4cec-9025-d70335dbe1b2
vmstate: local-zfs:vm-103-state-afterInitFreenas

I also blacklisted the mpt3sas driver.
When I try to start the VM, I get the following error:

Code: 
kvm: -device vfio-pci,host=07:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0: vfio error: 0000:07:00.0: failed to setup container for group 1: failed to set iommu for container: Operation not permitted
TASK ERROR: start failed: command '/usr/bin/kvm -id 103 -name 2pFreenas -chardev 'socket,id=qmp,path=/var/run/qemu-server/103.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' -mon 'chardev=qmp-event,mode=control' -pidfile /var/run/qemu-server/103.pid -daemonize -smbios 'type=1,uuid=0c2366f3-9576-45ce-9f92-508441ea3ed5' -smp '2,sockets=1,cores=2,maxcpus=2' -nodefaults -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' -vnc unix:/var/run/qemu-server/103.vnc,x509,password -cpu kvm64,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce -m 10192 -device 'vmgenid,guid=06ceb1f6-d7db-4cec-9025-d70335dbe1b2' -readconfig /usr/share/qemu-server/pve-q35.cfg -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' -device 'vfio-pci,host=07:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0' -device 'VGA,id=vga,bus=pcie.0,addr=0x1' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:a95eca963df1' -device 'virtio-scsi-pci,id=scsihw0,bus=pci.0,addr=0x5' -drive 'file=/dev/zvol/rpool/data/vm-103-disk-0,if=none,id=drive-scsi0,format=raw,cache=none,aio=native,detect-zeroes=on' -device 'scsi-hd,bus=scsihw0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap103i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=4E:77:B9:7A:E5:19,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' -machine 'type=q35'' failed: exit code 1

Any help on this, I don't like the following message:
Code: 
vfio-pci 0000:07:00.0: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor

If I remember, before blacklisting the driver I didn't have this message....
 
I did again the update-initramfs and rebooted and I get the following dmesg without error now:
Code: 
root@pve:~# dmesg | grep -e DMAR -e IOMMU
[    0.000000] ACPI: DMAR 0x00000000F1DE4A80 0003B4 (v01 HP     ProLiant 00000001 \xffffffd2?   0000162E)
[    0.000000] DMAR: IOMMU enabled
[    0.000000] DMAR: Host address width 39
[    0.000000] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
[    0.000000] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap c9008020660262 ecap f010da
[    0.000000] DMAR: RMRR base: 0x000000f1ffd000 end: 0x000000f1ffffff
[    0.000000] DMAR: RMRR base: 0x000000f1ff6000 end: 0x000000f1ffcfff
[    0.000000] DMAR: RMRR base: 0x000000f1f93000 end: 0x000000f1f94fff
[    0.000000] DMAR: RMRR base: 0x000000f1f8f000 end: 0x000000f1f92fff
[    0.000000] DMAR: RMRR base: 0x000000f1f7f000 end: 0x000000f1f8efff
[    0.000000] DMAR: RMRR base: 0x000000f1f7e000 end: 0x000000f1f7efff
[    0.000000] DMAR: RMRR base: 0x000000000f4000 end: 0x000000000f4fff
[    0.000000] DMAR: RMRR base: 0x000000000e8000 end: 0x000000000e8fff
[    0.000000] DMAR: RMRR base: 0x000000f1dee000 end: 0x000000f1deefff
[    0.000000] DMAR-IR: IOAPIC id 8 under DRHD base  0xfed90000 IOMMU 0
[    0.000000] DMAR-IR: HPET id 0 under DRHD base 0xfed90000
[    0.000000] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit.
[    0.000000] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting.
[    0.000000] DMAR-IR: Enabled IRQ remapping in xapic mode
[    0.720582] DMAR: No ATSR found
[    0.720893] DMAR: dmar0: Using Queued invalidation
[    0.720902] DMAR: Setting RMRR:
[    0.721187] DMAR: Setting identity map for device 0000:01:00.0 [0xf1dee000 - 0xf1deefff]
[    0.721500] DMAR: Setting identity map for device 0000:01:00.2 [0xf1dee000 - 0xf1deefff]
[    0.721806] DMAR: Setting identity map for device 0000:01:00.4 [0xf1dee000 - 0xf1deefff]
[    0.722137] DMAR: Setting identity map for device 0000:00:1f.2 [0xe8000 - 0xe8fff]
[    0.722446] DMAR: Setting identity map for device 0000:07:00.0 [0xe8000 - 0xe8fff]
[    0.722772] DMAR: Setting identity map for device 0000:03:00.0 [0xe8000 - 0xe8fff]
[    0.723088] DMAR: Setting identity map for device 0000:03:00.1 [0xe8000 - 0xe8fff]
[    0.723415] DMAR: Setting identity map for device 0000:04:00.0 [0xe8000 - 0xe8fff]
[    0.723424] DMAR: Setting identity map for device 0000:01:00.0 [0xe8000 - 0xe8fff]
[    0.723434] DMAR: Setting identity map for device 0000:01:00.2 [0xe8000 - 0xe8fff]
[    0.723444] DMAR: Setting identity map for device 0000:00:1f.2 [0xf4000 - 0xf4fff]
[    0.723446] DMAR: Setting identity map for device 0000:07:00.0 [0xf4000 - 0xf4fff]
[    0.723448] DMAR: Setting identity map for device 0000:03:00.0 [0xf4000 - 0xf4fff]
[    0.723449] DMAR: Setting identity map for device 0000:03:00.1 [0xf4000 - 0xf4fff]
[    0.723451] DMAR: Setting identity map for device 0000:04:00.0 [0xf4000 - 0xf4fff]
[    0.723452] DMAR: Setting identity map for device 0000:01:00.0 [0xf4000 - 0xf4fff]
[    0.723454] DMAR: Setting identity map for device 0000:01:00.2 [0xf4000 - 0xf4fff]
[    0.723455] DMAR: Setting identity map for device 0000:00:1f.2 [0xf1f7e000 - 0xf1f7efff]
[    0.723464] DMAR: Setting identity map for device 0000:07:00.0 [0xf1f7e000 - 0xf1f7efff]
[    0.723473] DMAR: Setting identity map for device 0000:03:00.0 [0xf1f7e000 - 0xf1f7efff]
[    0.723481] DMAR: Setting identity map for device 0000:03:00.1 [0xf1f7e000 - 0xf1f7efff]
[    0.723490] DMAR: Setting identity map for device 0000:04:00.0 [0xf1f7e000 - 0xf1f7efff]
[    0.723499] DMAR: Setting identity map for device 0000:01:00.0 [0xf1f7e000 - 0xf1f7efff]
[    0.723504] DMAR: Setting identity map for device 0000:01:00.2 [0xf1f7e000 - 0xf1f7efff]
[    0.723509] DMAR: Setting identity map for device 0000:00:1f.2 [0xf1f7f000 - 0xf1f8efff]
[    0.723511] DMAR: Setting identity map for device 0000:07:00.0 [0xf1f7f000 - 0xf1f8efff]
[    0.723512] DMAR: Setting identity map for device 0000:03:00.0 [0xf1f7f000 - 0xf1f8efff]
[    0.723514] DMAR: Setting identity map for device 0000:03:00.1 [0xf1f7f000 - 0xf1f8efff]
[    0.723516] DMAR: Setting identity map for device 0000:04:00.0 [0xf1f7f000 - 0xf1f8efff]
[    0.723518] DMAR: Setting identity map for device 0000:01:00.0 [0xf1f7f000 - 0xf1f8efff]
[    0.723520] DMAR: Setting identity map for device 0000:01:00.2 [0xf1f7f000 - 0xf1f8efff]
[    0.723522] DMAR: Setting identity map for device 0000:00:1f.2 [0xf1f8f000 - 0xf1f92fff]
[    0.723523] DMAR: Setting identity map for device 0000:07:00.0 [0xf1f8f000 - 0xf1f92fff]
[    0.723525] DMAR: Setting identity map for device 0000:03:00.0 [0xf1f8f000 - 0xf1f92fff]
[    0.723527] DMAR: Setting identity map for device 0000:03:00.1 [0xf1f8f000 - 0xf1f92fff]
[    0.723528] DMAR: Setting identity map for device 0000:04:00.0 [0xf1f8f000 - 0xf1f92fff]
[    0.723530] DMAR: Setting identity map for device 0000:01:00.0 [0xf1f8f000 - 0xf1f92fff]
[    0.723532] DMAR: Setting identity map for device 0000:01:00.2 [0xf1f8f000 - 0xf1f92fff]
[    0.723533] DMAR: Setting identity map for device 0000:00:1f.2 [0xf1f93000 - 0xf1f94fff]
[    0.723535] DMAR: Setting identity map for device 0000:07:00.0 [0xf1f93000 - 0xf1f94fff]
[    0.723536] DMAR: Setting identity map for device 0000:03:00.0 [0xf1f93000 - 0xf1f94fff]
[    0.723537] DMAR: Setting identity map for device 0000:03:00.1 [0xf1f93000 - 0xf1f94fff]
[    0.723539] DMAR: Setting identity map for device 0000:04:00.0 [0xf1f93000 - 0xf1f94fff]
[    0.723541] DMAR: Setting identity map for device 0000:01:00.0 [0xf1f93000 - 0xf1f94fff]
[    0.723542] DMAR: Setting identity map for device 0000:01:00.2 [0xf1f93000 - 0xf1f94fff]
[    0.723544] DMAR: Setting identity map for device 0000:01:00.0 [0xf1ff6000 - 0xf1ffcfff]
[    0.723545] DMAR: Setting identity map for device 0000:01:00.2 [0xf1ff6000 - 0xf1ffcfff]
[    0.723547] DMAR: Setting identity map for device 0000:01:00.4 [0xf1ff6000 - 0xf1ffcfff]
[    0.723863] DMAR: Setting identity map for device 0000:00:1a.0 [0xf1ffd000 - 0xf1ffffff]
[    0.724211] DMAR: Setting identity map for device 0000:00:1d.0 [0xf1ffd000 - 0xf1ffffff]
[    0.724221] DMAR: Prepare 0-16MiB unity mapping for LPC
[    0.724531] DMAR: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[    0.724622] DMAR: Intel(R) Virtualization Technology for Directed I/O

but when I attach it to a VM, I get the same message as in the previous post. What am I missing ?
 
flipper203 said:
vfio-pci 0000:07:00.0: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor
Click to expand...
this means that either your mainboard or the mainboard/device combo has some platform restriction for passthrough, sometimes a bios/firmware upgrade can help, but if not, there is not much you can do
 
dcsapak said:
this means that either your mainboard or the mainboard/device combo has some platform restriction for passthrough, sometimes a bios/firmware upgrade can help, but if not, there is not much you can do
Click to expand...
the thing is when I reboot, before I attatch the device to a VM and try to boot it, I don't get this message in the dmesg command
 
flipper203 said:
the thing is when I reboot, before I attatch the device to a VM and try to boot it, I don't get this message in the dmesg command
Click to expand...
yes, when the vm tries to passthrough it, the error occurs ... i am not sure what you are saying?
 
If I reboot the proxmox host, without any vm configured as passthrough, I get the following result:
Code: 
root@pve:~# dmesg | grep -e DMAR -e IOMMU
[    0.000000] ACPI: DMAR 0x00000000F1DE4A80 0003B4 (v01 HP     ProLiant 00000001 \xffffffd2?   0000162E)
[    0.000000] DMAR: IOMMU enabled
[    0.000000] DMAR: Host address width 39
[    0.000000] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
[    0.000000] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap c9008020660262 ecap f010da
[    0.000000] DMAR: RMRR base: 0x000000f1ffd000 end: 0x000000f1ffffff
[    0.000000] DMAR: RMRR base: 0x000000f1ff6000 end: 0x000000f1ffcfff
[    0.000000] DMAR: RMRR base: 0x000000f1f93000 end: 0x000000f1f94fff
[    0.000000] DMAR: RMRR base: 0x000000f1f8f000 end: 0x000000f1f92fff
[    0.000000] DMAR: RMRR base: 0x000000f1f7f000 end: 0x000000f1f8efff
[    0.000000] DMAR: RMRR base: 0x000000f1f7e000 end: 0x000000f1f7efff
[    0.000000] DMAR: RMRR base: 0x000000000f4000 end: 0x000000000f4fff
[    0.000000] DMAR: RMRR base: 0x000000000e8000 end: 0x000000000e8fff
[    0.000000] DMAR: RMRR base: 0x000000f1dee000 end: 0x000000f1deefff
[    0.000000] DMAR-IR: IOAPIC id 8 under DRHD base  0xfed90000 IOMMU 0
[    0.000000] DMAR-IR: HPET id 0 under DRHD base 0xfed90000
[    0.000000] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit.
[    0.000000] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting.
[    0.000000] DMAR-IR: Enabled IRQ remapping in xapic mode
[    0.720453] DMAR: No ATSR found
[    0.720759] DMAR: dmar0: Using Queued invalidation
[    0.720767] DMAR: Setting RMRR:
[    0.721055] DMAR: Setting identity map for device 0000:01:00.0 [0xf1dee000 - 0xf1deefff]
[    0.721369] DMAR: Setting identity map for device 0000:01:00.2 [0xf1dee000 - 0xf1deefff]
[    0.721674] DMAR: Setting identity map for device 0000:01:00.4 [0xf1dee000 - 0xf1deefff]
[    0.722001] DMAR: Setting identity map for device 0000:00:1f.2 [0xe8000 - 0xe8fff]
[    0.722312] DMAR: Setting identity map for device 0000:07:00.0 [0xe8000 - 0xe8fff]
[    0.722641] DMAR: Setting identity map for device 0000:03:00.0 [0xe8000 - 0xe8fff]
[    0.722953] DMAR: Setting identity map for device 0000:03:00.1 [0xe8000 - 0xe8fff]
[    0.723282] DMAR: Setting identity map for device 0000:04:00.0 [0xe8000 - 0xe8fff]
[    0.723292] DMAR: Setting identity map for device 0000:01:00.0 [0xe8000 - 0xe8fff]
[    0.723302] DMAR: Setting identity map for device 0000:01:00.2 [0xe8000 - 0xe8fff]
[    0.723313] DMAR: Setting identity map for device 0000:00:1f.2 [0xf4000 - 0xf4fff]
[    0.723315] DMAR: Setting identity map for device 0000:07:00.0 [0xf4000 - 0xf4fff]
[    0.723316] DMAR: Setting identity map for device 0000:03:00.0 [0xf4000 - 0xf4fff]
[    0.723318] DMAR: Setting identity map for device 0000:03:00.1 [0xf4000 - 0xf4fff]
[    0.723319] DMAR: Setting identity map for device 0000:04:00.0 [0xf4000 - 0xf4fff]
[    0.723320] DMAR: Setting identity map for device 0000:01:00.0 [0xf4000 - 0xf4fff]
[    0.723322] DMAR: Setting identity map for device 0000:01:00.2 [0xf4000 - 0xf4fff]
[    0.723323] DMAR: Setting identity map for device 0000:00:1f.2 [0xf1f7e000 - 0xf1f7efff]
[    0.723332] DMAR: Setting identity map for device 0000:07:00.0 [0xf1f7e000 - 0xf1f7efff]
[    0.723341] DMAR: Setting identity map for device 0000:03:00.0 [0xf1f7e000 - 0xf1f7efff]
[    0.723350] DMAR: Setting identity map for device 0000:03:00.1 [0xf1f7e000 - 0xf1f7efff]
[    0.723358] DMAR: Setting identity map for device 0000:04:00.0 [0xf1f7e000 - 0xf1f7efff]
[    0.723367] DMAR: Setting identity map for device 0000:01:00.0 [0xf1f7e000 - 0xf1f7efff]
[    0.723372] DMAR: Setting identity map for device 0000:01:00.2 [0xf1f7e000 - 0xf1f7efff]
[    0.723377] DMAR: Setting identity map for device 0000:00:1f.2 [0xf1f7f000 - 0xf1f8efff]
[    0.723379] DMAR: Setting identity map for device 0000:07:00.0 [0xf1f7f000 - 0xf1f8efff]
[    0.723381] DMAR: Setting identity map for device 0000:03:00.0 [0xf1f7f000 - 0xf1f8efff]
[    0.723382] DMAR: Setting identity map for device 0000:03:00.1 [0xf1f7f000 - 0xf1f8efff]
[    0.723384] DMAR: Setting identity map for device 0000:04:00.0 [0xf1f7f000 - 0xf1f8efff]
[    0.723386] DMAR: Setting identity map for device 0000:01:00.0 [0xf1f7f000 - 0xf1f8efff]
[    0.723388] DMAR: Setting identity map for device 0000:01:00.2 [0xf1f7f000 - 0xf1f8efff]
[    0.723390] DMAR: Setting identity map for device 0000:00:1f.2 [0xf1f8f000 - 0xf1f92fff]
[    0.723391] DMAR: Setting identity map for device 0000:07:00.0 [0xf1f8f000 - 0xf1f92fff]
[    0.723393] DMAR: Setting identity map for device 0000:03:00.0 [0xf1f8f000 - 0xf1f92fff]
[    0.723395] DMAR: Setting identity map for device 0000:03:00.1 [0xf1f8f000 - 0xf1f92fff]
[    0.723396] DMAR: Setting identity map for device 0000:04:00.0 [0xf1f8f000 - 0xf1f92fff]
[    0.723398] DMAR: Setting identity map for device 0000:01:00.0 [0xf1f8f000 - 0xf1f92fff]
[    0.723400] DMAR: Setting identity map for device 0000:01:00.2 [0xf1f8f000 - 0xf1f92fff]
[    0.723401] DMAR: Setting identity map for device 0000:00:1f.2 [0xf1f93000 - 0xf1f94fff]
[    0.723403] DMAR: Setting identity map for device 0000:07:00.0 [0xf1f93000 - 0xf1f94fff]
[    0.723404] DMAR: Setting identity map for device 0000:03:00.0 [0xf1f93000 - 0xf1f94fff]
[    0.723406] DMAR: Setting identity map for device 0000:03:00.1 [0xf1f93000 - 0xf1f94fff]
[    0.723407] DMAR: Setting identity map for device 0000:04:00.0 [0xf1f93000 - 0xf1f94fff]
[    0.723409] DMAR: Setting identity map for device 0000:01:00.0 [0xf1f93000 - 0xf1f94fff]
[    0.723410] DMAR: Setting identity map for device 0000:01:00.2 [0xf1f93000 - 0xf1f94fff]
[    0.723412] DMAR: Setting identity map for device 0000:01:00.0 [0xf1ff6000 - 0xf1ffcfff]
[    0.723413] DMAR: Setting identity map for device 0000:01:00.2 [0xf1ff6000 - 0xf1ffcfff]
[    0.723415] DMAR: Setting identity map for device 0000:01:00.4 [0xf1ff6000 - 0xf1ffcfff]
[    0.723731] DMAR: Setting identity map for device 0000:00:1a.0 [0xf1ffd000 - 0xf1ffffff]
[    0.724105] DMAR: Setting identity map for device 0000:00:1d.0 [0xf1ffd000 - 0xf1ffffff]
[    0.724116] DMAR: Prepare 0-16MiB unity mapping for LPC
[    0.724426] DMAR: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[    0.724516] DMAR: Intel(R) Virtualization Technology for Directed I/O

So, no error. If I attach the PCIe device to the VM, and try to start it, I get the error above when trying to start the VM, and the dmesg command outputs the error again.
 
flipper203 said:
So, no error. If I attach the PCIe device to the VM, and try to start it, I get the error above when trying to start the VM, and the dmesg command outputs the error again.
Click to expand...
yes the error occurs only when you try to passthrough... this is normal...
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom