PCI Passthrough - No interrupt remapping support

maxprox

maxprox

Renowned Member
Proxmox Subscriber
Aug 23, 2011
423
57
93
Germany - Nordhessen
fair-comp.de
Hello,

As in the other threads I've PCI passthrough configured:
On the Proxmox host I do nothing with the PCI-card I want to use for "pci-passthrough". For example, I have not disabled it in the host,
also I have not disable a module, because for this PCI-card there is no module in the host. I'm not sure if that caused the fault.
The VM who should get the pci-card is Ubuntu Server 11.04

the VM config:

Code: 
net0: virtio=0A:22:B1:6B:00:E9,bridge=vmbr0
ide2: local:iso/ubuntu-11.04-server-amd64.iso,media=cdrom
name: vdr
bootdisk: virtio0
virtio0: local:113/vm-113-disk-1.raw
cores: 2
ostype: l26
memory: 2048
sockets: 1
hostpci0: 03:00.0

In the BIOS of the Supermicro X8SIL-F Bord I activate the Vt-d an the output of dmesg | grep -e DMAR -e IOMMU is:

Code: 
root@fcprox01:# dmesg | grep -e DMAR -e IOMMU
ACPI: DMAR 00000000bf7b00f0 00090 (v01    AMI  OEMDMAR 00000001 MSFT 00000097)
Intel-IOMMU: enabled
DMAR: Host address width 36
DMAR: DRHD base: 0x000000fed90000 flags: 0x1
IOMMU fed90000: ver 1:0 cap c90780106f0462 ecap f020e3
DMAR: RMRR base: 0x000000000ed000 end: 0x000000000effff
DMAR: RMRR base: 0x000000bf7ed000 end: 0x000000bf7fffff
DMAR: No ATSR found
IOMMU 0xfed90000: using Queued invalidation
IOMMU: Setting RMRR:
IOMMU: Setting identity map for device 0000:00:1d.0 [0xbf7ed000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1a.0 [0xbf7ed000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1d.0 [0xed000 - 0xf0000]
IOMMU: Setting identity map for device 0000:00:1a.0 [0xed000 - 0xf0000]
IOMMU: Prepare 0-16MiB unity mapping for LPC
IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0x1000000]

The Kernel parameter is:
Code: 
root@fcprox01:/boot# cat /etc/default/grub | grep iommu
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"



The PCI-Card in the Proxmox host I mean is:
Code: 
root@fcprox01:# lspci

03:00.0 Multimedia controller: Philips Semiconductors SAA7160 (rev 02)

The error in the syslog I think is the line "kvm_iommu_map_guest: No interrupt remapping support, disallowing device assignment. Re-enble with "allow_unsafe_assigned_interrupts=1" module option.

Code: 
Nov 18 22:22:20 fcprox01 pvedaemon[17882]: VM 113 start
Nov 18 22:22:20 fcprox01 kernel: pci-stub 0000:03:00.0: claimed by stub
Nov 18 22:22:20 fcprox01 kernel: pci-stub 0000:03:00.0: claimed by stub
Nov 18 22:22:20 fcprox01 kernel: pci-stub 0000:03:00.0: restoring config space at offset 0x1 (was 0x100400, writing 0x100007)
Nov 18 22:22:21 fcprox01 kernel: device tap113i0 entered promiscuous mode
Nov 18 22:22:21 fcprox01 kernel: vmbr0: port 3(tap113i0) entering forwarding state
Nov 18 22:22:21 fcprox01 kernel: pci-stub 0000:03:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
Nov 18 22:22:21 fcprox01 kernel: pci-stub 0000:03:00.0: restoring config space at offset 0x1 (was 0x100400, writing 0x100007)
Nov 18 22:22:21 fcprox01 kernel: kvm_iommu_map_guest: No interrupt remapping support, disallowing device assignment. Re-enble with "allow_unsafe_assigned_interrupts=1" module option.
Nov 18 22:22:21 fcprox01 kernel: pci-stub 0000:03:00.0: PCI INT A disabled
Nov 18 22:22:21 fcprox01 pvedaemon[17882]: VM 113 start failed: command '/usr/bin/kvm -id 113 -chardev socket,id=monitor,path=/var/run/qemu-server/113.mon,server,nowait -mon chardev=monitor,mode=readline -vnc unix:/var/run/qemu-server/113.vnc,x509,password -pidfile /var/run/qemu-server/113.pid -daemonize -readconfig /usr/share/qemu-server/pve-usb.cfg -device usb-tablet,bus=ehci.0,port=6 -device pci-assign,host=03:00.0,id=hostpci0,bus=pci.0,addr=0x10 -name vdr -smp sockets=1,cores=2 -nodefaults -boot menu=on,order=cdn -vga cirrus -tdf -k en-us -drive file=/var/lib/vz/template/iso/ubuntu-11.04-server-amd64.iso,if=none,id=drive-ide2,media=cdrom,aio=native -device ide-drive,bus=ide.1,unit=0,drive=drive-ide2,id=device-ide2 -drive file=/var/lib/vz/images/113/vm-113-disk-1.raw,if=none,id=drive-virtio0,aio=native,boot=on -device virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa -m 2048 -netdev type=tap,id=net0,ifname=tap113i0,script=/var/lib/qemu-server/pve-bridge -device virtio-net-pci,romfile=,mac=0A:22:B1:6B:00:E9,netdev=net0,bus=pci.0,addr=0x12' failed: exit code 1
Nov 18 22:22:21 fcprox01 pvedaemon[17882]: start failed: command '/usr/bin/kvm -id 113 -chardev socket,id=monitor,path=/var/run/qemu-server/113.mon,server,nowait -mon chardev=monitor,mode=readline -vnc unix:/var/run/qemu-server/113.vnc,x509,password -pidfile /var/run/qemu-server/113.pid -daemonize -readconfig /usr/share/qemu-server/pve-usb.cfg -device usb-tablet,bus=ehci.0,port=6 -device pci-assign,host=03:00.0,id=hostpci0,bus=pci.0,addr=0x10 -name vdr -smp sockets=1,cores=2 -nodefaults -boot menu=on,order=cdn -vga cirrus -tdf -k en-us -drive file=/var/lib/vz/template/iso/ubuntu-11.04-server-amd64.iso,if=none,id=drive-ide2,media=cdrom,aio=native -device ide-drive,bus=ide.1,unit=0,drive=drive-ide2,id=device-ide2 -drive file=/var/lib/vz/images/113/vm-113-disk-1.raw,if=none,id=drive-virtio0,aio=native,boot=on -device virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa -m 2048 -netdev type=tap,id=net0,ifname=tap113i0,script=/var/lib/qemu-server/pve-bridge -device virtio-net-pci,romfile=,mac=0A:22:B1:6B:00:E9,netdev=net0,bus=pci.0,addr=0x12' failed: exit code 1
Nov 18 22:22:21 fcprox01 pvedaemon[17611]: <root@pam> end task UPID:fcprox01:000045DA:001928D6:4EC6CC8C:qmstart:113:root@pam: unexpected status
Nov 18 22:22:21 fcprox01 kernel: vmbr0: port 3(tap113i0) entering disabled state
Nov 18 22:22:21 fcprox01 kernel: vmbr0: port 3(tap113i0) entering disabled state
Nov 18

what can I do?

regards,
proxmox
 
Last edited:
Hello, can someone tell me whether the following kernel options are already turned on?
Support for DMA Remapping Devices" to "*"
"Enable DMA Remapping Devices" to "*"
Click to expand...
Or do I have to recompile the kernel? Just as in this thread ...

I found another post in this context: Disable device assignment without interrupt But I do not understand what I can do with it.

my proxmox version is:
Code: 
root@fcprox01:# pveversion -v
pve-manager: 2.0-10 (pve-manager/2.0/7a10f3e6)
running kernel: 2.6.32-6-pve
proxmox-ve-2.6.32: 2.0-52
pve-kernel-2.6.32-6-pve: 2.6.32-52
lvm2: 2.02.86-1pve1
clvm: 2.02.86-1pve1
corosync-pve: 1.4.1-1
openais-pve: 1.1.4-1
libqb: 0.6.0-1
redhat-cluster-pve: 3.1.7-1
pve-cluster: 1.0-11
qemu-server: 2.0-3
pve-firmware: 1.0-13
libpve-common-perl: 1.0-7
libpve-access-control: 1.0-2
libpve-storage-perl: 2.0-6
vncterm: 1.0-2
vzctl: 3.0.29-3pve3
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 0.15.0-1
ksm-control-daemon: 1.1-1
regards, proxmox
 
Last edited:
dietmar said:
That should be enabled (see (/boot/config-*))
Click to expand...

But I can not find a line with the word remapping:
Code: 
root@fcprox01:/boot# cat  config-2.6.32-6-pve | grep -i remapping

And the other question is what to do with the line:
Code: 
"kvm_iommu_map_guest: No interrupt remapping support, disallowing device assignment. Re-enble with "allow_unsafe_assigned_interrupts=1" module option.
there are somewhere a possibility to switch it on?
EDIT:
I found some different Parameter for the VMID.conf and pci-passthrough:
Code: 
## hostpci0: 03:00.0
## args: -pcidevice host=03:00.0
## args: -device pci-assign,host=03:00.0,id=hostpci0
which parameters are correct for proxmox 2.0 and my solution?

maxprox
 
Last edited:
  • Like
Reactions: ONE FOTON
I found some different Parameter for the VMID.conf and pci-passthrough:
Code: 
## hostpci0: 03:00.0
## args: -pcidevice host=03:00.0
## args: -device pci-assign,host=03:00.0,id=hostpci0
which parameters are correct for proxmox 2.0 and my solution?
Click to expand...

Please I would also like to know the answer to this question. Anyone?
 
Yes,

Code: 
  hostpci0: xx:xx.x

For example here my VMID.conf:

Code: 
root@fcprox01:/etc/pve/nodes/fcprox01/qemu-server# cat 113.conf 
net0: virtio=0A:22:B1:6B:00:E9,bridge=vmbr0
ide2: local:iso/ubuntu-11.04-server-amd64.iso,media=cdrom
name: vdr
bootdisk: virtio0
virtio0: local:113/vm-113-disk-1.raw
virtio1: vdr2TB:vm-113-disk-1,backup=no
cores: 2
sockets: 1
ostype: l26
memory: 2048
keyboard: de
hostpci0: 03:00.0
root@fcprox01:/etc/pve/nodes/fcprox01/qemu-server#

But have also a look at here :confused:
 
Hello,
Now I had three months of a stable system based on PCI passthrough ...
since the last kernel update, it's over.

Has anything changed with respect to PCI passthrough?
Or in relation to IRQ remapping or whatever?

some syslog messages from the proxmox host, if I start the VM 112 in question
Code: 
Mar 24 01:23:31 fcprox01 ntpd_intres[1953]: DNS zentyal-vm -> 172.1.1.3
Mar 24 01:23:31 fcprox01 ntpd_intres[1953]: DNS zentyal-vm.peter.lan -> 172.1.1.3
Mar 24 01:25:17 fcprox01 qm[3121]: start VM 112: UPID:fcprox01:00000C31:00007FC2:4F6D146D:qmstart:112:root@pam:
Mar 24 01:25:17 fcprox01 qm[3118]: <root@pam> starting task UPID:fcprox01:00000C31:00007FC2:4F6D146D:qmstart:112:root@pam:
Mar 24 01:25:17 fcprox01 kernel: pci-stub 0000:03:00.0: claimed by stub
Mar 24 01:25:17 fcprox01 kernel: pci-stub 0000:03:00.0: claimed by stub
Mar 24 01:25:17 fcprox01 kernel: pci-stub 0000:03:00.0: restoring config space at offset 0x1 (was 0x100400, writing 0x100007)
Mar 24 01:25:18 fcprox01 kernel: device tap112i0 entered promiscuous mode
Mar 24 01:25:18 fcprox01 kernel: vmbr0: port 3(tap112i0) entering forwarding state
Mar 24 01:25:18 fcprox01 kernel: pci-stub 0000:03:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
Mar 24 01:25:18 fcprox01 kernel: pci-stub 0000:03:00.0: restoring config space at offset 0x1 (was 0x100400, writing 0x100007)
Mar 24 01:25:18 fcprox01 kernel: assign device: host bdf = 3:0:0
Mar 24 01:25:18 fcprox01 kernel:  alloc irq_desc for 42 on node -1
Mar 24 01:25:18 fcprox01 kernel:  alloc kstat_irqs on node -1
Mar 24 01:25:18 fcprox01 kernel: pci-stub 0000:03:00.0: irq 42 for MSI/MSI-X
Mar 24 01:25:18 fcprox01 kernel: pci-stub 0000:03:00.0: restoring config space at offset 0x1 (was 0x100400, writing 0x100407)
Mar 24 01:25:18 fcprox01 qm[3118]: <root@pam> end task UPID:fcprox01:00000C31:00007FC2:4F6D146D:qmstart:112:root@pam: OK
Mar 24 01:25

These lines relate to the PCI TV card
Code: 
Mar 24 01:25:17 fcprox01 kernel: pci-stub 0000:03:00.0: claimed by stub

Some messages of the VM 112 (easyVDR) dmesg:
If I understand correctly, the TV card (SAA716x / a TechnoTrend S2 6400) is first installed and then discarded (?)

Code: 
[    5.685539] SAA716x FF 0000:00:10.0: PCI INT A -> Link[LNKD] -> GSI 11
(level, high) -> IRQ 11
[    5.685565] SAA716x FF 0000:00:10.0: setting latency timer to 64
[    5.756392] psmouse serio1: ID: 10 00 64
[    5.774123] init: easyvdr-create-ramdisk main process (514) killed by TERM
signal
[    5.797103] type=1400 audit(1332549254.538:2): apparmor="STATUS"
operation="profile_load" name="/sbin/dhclient" pid=576 comm="apparmor_parser"
....
[    7.452016] SAA716x FF FPGA version 1.09
[    7.501850] init: ssh main process (526) terminated with status 255
[    7.509216] SAA716x FF loader version 1.03
[    7.783967] type=1400 audit(1332549256.522:5): apparmor="STATUS"
operation="profile_replace" name="/sbin/dhclient" pid=924 comm="apparmor_parser"
....
[    7.892044] init: failsafe main process (704) killed by TERM signal
[    8.174267] DVB: registering new adapter (SAA716x dvb adapter)
[    8.289877] stv6110x_attach: Attaching STV6110x
[    8.292017] DVB: registering adapter 0 frontend 0 (STV090x Multistandard)...
[    8.292124] DVB: registering new adapter (SAA716x dvb adapter)
[    8.292356] stv6110x_attach: Attaching STV6110x
[    8.326433] DVB: registering adapter 1 frontend 0 (STV090x Multistandard)...
[    8.392265] RPC: Registered named UNIX socket transport module.
.....

[   18.325389] SAA716x FF 0000:00:10.0: PCI INT A disabled
[   18.325396] SAA716x FF: probe of 0000:00:10.0 failed with error -1
[   18.405082] init: udev-fallback-graphics main process (1522) terminated with
status 1
[   18.430679] init: easyvdr-vdr post-start process (1563) terminated with
status 1
.....




pveversion -v
pve-manager: 2.0-42 (pve-manager/2.0/3d6d8258)
running kernel: 2.6.32-10-pve
proxmox-ve-2.6.32: 2.0-63
pve-kernel-2.6.32-10-pve: 2.6.32-63
pve-kernel-2.6.32-6-pve: 2.6.32-55
pve-kernel-2.6.32-7-pve: 2.6.32-60
lvm2: 2.02.88-2pve2
clvm: 2.02.88-2pve2
corosync-pve: 1.4.1-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.8-3
resource-agents-pve: 3.9.2-3

fence-agents-pve: 3.1.7-1
pve-cluster: 1.0-25
qemu-server: 2.0-28
pve-firmware: 1.0-15
libpve-common-perl: 1.0-21
libpve-access-control: 1.0-17
libpve-storage-perl: 2.0-14
vncterm: 1.0-2
vzctl: 3.0.30-2pve2
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.0-7
ksm-control-daemon: 1.1-1

regards
maxprox
 
Last edited:
Re: PCI Passthrough - No interrupt remapping support [solved]

Indeed
the reason was the new version of pve-qemu-kvm: 1.0-7
after a downgrade to the version 1.0-3 it works as before

what I have done is:

wget http://download.proxmox.com/debian/d....0-3_amd64.deb
dpkg -i pve-qemu-kvm_1.0-3_amd64.deb

all machines restart

t@fcprox01:~# pveversion -v
pve-manager: 2.0-42 (pve-manager/2.0/3d6d8258)
running kernel: 2.6.32-10-pve
proxmox-ve-2.6.32: 2.0-63
pve-kernel-2.6.32-10-pve: 2.6.32-63
pve-kernel-2.6.32-6-pve: 2.6.32-55
pve-kernel-2.6.32-7-pve: 2.6.32-60
lvm2: 2.02.88-2pve2
clvm: 2.02.88-2pve2
corosync-pve: 1.4.1-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.8-3
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.7-1
pve-cluster: 1.0-25
qemu-server: 2.0-28
pve-firmware: 1.0-15
libpve-common-perl: 1.0-21
libpve-access-control: 1.0-17
libpve-storage-perl: 2.0-14
vncterm: 1.0-2
vzctl: 3.0.30-2pve2
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.0-3 <== <==
ksm-control-daemon: 1.1-1
root@fcprox01:

regards
maxprox
 
Last edited:
Re: PCI Passthrough - No interrupt remapping support [solved]

Hi,
since more than a half year last weekend I try
Code: 
aptitude update && aptitude full-upgrade

an now with the new versions there are no problems with pci passthrough - like bevor
with pve-qemu-kvm version <= 1.0-5
This problem for me is solved -

thank you


Code: 
root@fcprox01:~# pveversion -v
pve-manager: 2.2-32 (pve-manager/2.2/3089a616)
running kernel: 2.6.32-17-pved
proxmox-ve-2.6.32: 2.2-83
pve-kernel-2.6.32-12-pve: 2.6.32-68
pve-kernel-2.6.32-17-pve: 2.6.32-83
lvm2: 2.02.95-1pve2
clvm: 2.02.95-1pve2
corosync-pve: 1.4.4-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.93-2
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.9-1
pve-cluster: 1.0-34
qemu-server: 2.0-71
pve-firmware: 1.0-21
libpve-common-perl: 1.0-41
libpve-access-control: 1.0-25
libpve-storage-perl: 2.0-36
vncterm: 1.0-3
vzctl: 4.0-1pve2
vzprocps: 2.0.11-2
vzquota: 3.1-1
pve-qemu-kvm: 1.3-10
ksm-control-daemon: 1.1-1
root@fcprox01:~#
 
  • Like
Reactions: ONE FOTON
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back