PDA

View Full Version : PCI Passthrough - No interrupt remapping support



maxprox
11-18-2011, 11:10 PM
Hello,

As in the other threads I've PCI passthrough configured:
On the Proxmox host I do nothing with the PCI-card I want to use for "pci-passthrough". For example, I have not disabled it in the host,
also I have not disable a module, because for this PCI-card there is no module in the host. I'm not sure if that caused the fault.
The VM who should get the pci-card is Ubuntu Server 11.04

the VM config:


net0: virtio=0A:22:B1:6B:00:E9,bridge=vmbr0
ide2: local:iso/ubuntu-11.04-server-amd64.iso,media=cdrom
name: vdr
bootdisk: virtio0
virtio0: local:113/vm-113-disk-1.raw
cores: 2
ostype: l26
memory: 2048
sockets: 1
hostpci0: 03:00.0

In the BIOS of the Supermicro X8SIL-F Bord I activate the Vt-d an the output of dmesg | grep -e DMAR -e IOMMU is:


root@fcprox01:# dmesg | grep -e DMAR -e IOMMU
ACPI: DMAR 00000000bf7b00f0 00090 (v01 AMI OEMDMAR 00000001 MSFT 00000097)
Intel-IOMMU: enabled
DMAR: Host address width 36
DMAR: DRHD base: 0x000000fed90000 flags: 0x1
IOMMU fed90000: ver 1:0 cap c90780106f0462 ecap f020e3
DMAR: RMRR base: 0x000000000ed000 end: 0x000000000effff
DMAR: RMRR base: 0x000000bf7ed000 end: 0x000000bf7fffff
DMAR: No ATSR found
IOMMU 0xfed90000: using Queued invalidation
IOMMU: Setting RMRR:
IOMMU: Setting identity map for device 0000:00:1d.0 [0xbf7ed000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1a.0 [0xbf7ed000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1d.0 [0xed000 - 0xf0000]
IOMMU: Setting identity map for device 0000:00:1a.0 [0xed000 - 0xf0000]
IOMMU: Prepare 0-16MiB unity mapping for LPC
IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0x1000000]


The Kernel parameter is:

root@fcprox01:/boot# cat /etc/default/grub | grep iommu
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"



The PCI-Card in the Proxmox host I mean is:

root@fcprox01:# lspci

03:00.0 Multimedia controller: Philips Semiconductors SAA7160 (rev 02)


The error in the syslog I think is the line "kvm_iommu_map_guest: No interrupt remapping support, disallowing device assignment. Re-enble with "allow_unsafe_assigned_interrupts=1" module option.


Nov 18 22:22:20 fcprox01 pvedaemon[17882]: VM 113 start
Nov 18 22:22:20 fcprox01 kernel: pci-stub 0000:03:00.0: claimed by stub
Nov 18 22:22:20 fcprox01 kernel: pci-stub 0000:03:00.0: claimed by stub
Nov 18 22:22:20 fcprox01 kernel: pci-stub 0000:03:00.0: restoring config space at offset 0x1 (was 0x100400, writing 0x100007)
Nov 18 22:22:21 fcprox01 kernel: device tap113i0 entered promiscuous mode
Nov 18 22:22:21 fcprox01 kernel: vmbr0: port 3(tap113i0) entering forwarding state
Nov 18 22:22:21 fcprox01 kernel: pci-stub 0000:03:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
Nov 18 22:22:21 fcprox01 kernel: pci-stub 0000:03:00.0: restoring config space at offset 0x1 (was 0x100400, writing 0x100007)
Nov 18 22:22:21 fcprox01 kernel: kvm_iommu_map_guest: No interrupt remapping support, disallowing device assignment. Re-enble with "allow_unsafe_assigned_interrupts=1" module option.
Nov 18 22:22:21 fcprox01 kernel: pci-stub 0000:03:00.0: PCI INT A disabled
Nov 18 22:22:21 fcprox01 pvedaemon[17882]: VM 113 start failed: command '/usr/bin/kvm -id 113 -chardev socket,id=monitor,path=/var/run/qemu-server/113.mon,server,nowait -mon chardev=monitor,mode=readline -vnc unix:/var/run/qemu-server/113.vnc,x509,password -pidfile /var/run/qemu-server/113.pid -daemonize -readconfig /usr/share/qemu-server/pve-usb.cfg -device usb-tablet,bus=ehci.0,port=6 -device pci-assign,host=03:00.0,id=hostpci0,bus=pci.0,addr=0x1 0 -name vdr -smp sockets=1,cores=2 -nodefaults -boot menu=on,order=cdn -vga cirrus -tdf -k en-us -drive file=/var/lib/vz/template/iso/ubuntu-11.04-server-amd64.iso,if=none,id=drive-ide2,media=cdrom,aio=native -device ide-drive,bus=ide.1,unit=0,drive=drive-ide2,id=device-ide2 -drive file=/var/lib/vz/images/113/vm-113-disk-1.raw,if=none,id=drive-virtio0,aio=native,boot=on -device virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa -m 2048 -netdev type=tap,id=net0,ifname=tap113i0,script=/var/lib/qemu-server/pve-bridge -device virtio-net-pci,romfile=,mac=0A:22:B1:6B:00:E9,netdev=net0,bus =pci.0,addr=0x12' failed: exit code 1
Nov 18 22:22:21 fcprox01 pvedaemon[17882]: start failed: command '/usr/bin/kvm -id 113 -chardev socket,id=monitor,path=/var/run/qemu-server/113.mon,server,nowait -mon chardev=monitor,mode=readline -vnc unix:/var/run/qemu-server/113.vnc,x509,password -pidfile /var/run/qemu-server/113.pid -daemonize -readconfig /usr/share/qemu-server/pve-usb.cfg -device usb-tablet,bus=ehci.0,port=6 -device pci-assign,host=03:00.0,id=hostpci0,bus=pci.0,addr=0x1 0 -name vdr -smp sockets=1,cores=2 -nodefaults -boot menu=on,order=cdn -vga cirrus -tdf -k en-us -drive file=/var/lib/vz/template/iso/ubuntu-11.04-server-amd64.iso,if=none,id=drive-ide2,media=cdrom,aio=native -device ide-drive,bus=ide.1,unit=0,drive=drive-ide2,id=device-ide2 -drive file=/var/lib/vz/images/113/vm-113-disk-1.raw,if=none,id=drive-virtio0,aio=native,boot=on -device virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa -m 2048 -netdev type=tap,id=net0,ifname=tap113i0,script=/var/lib/qemu-server/pve-bridge -device virtio-net-pci,romfile=,mac=0A:22:B1:6B:00:E9,netdev=net0,bus =pci.0,addr=0x12' failed: exit code 1
Nov 18 22:22:21 fcprox01 pvedaemon[17611]: <root@pam> end task UPID:fcprox01:000045DA:001928D6:4EC6CC8C:qmstart:1 13:root@pam: unexpected status
Nov 18 22:22:21 fcprox01 kernel: vmbr0: port 3(tap113i0) entering disabled state
Nov 18 22:22:21 fcprox01 kernel: vmbr0: port 3(tap113i0) entering disabled state
Nov 18

what can I do?

regards,
proxmox

maxprox
11-18-2011, 11:44 PM
Hello, can someone tell me whether the following kernel options are already turned on?

Support for DMA Remapping Devices" to "*"
"Enable DMA Remapping Devices" to "*"
Or do I have to recompile the kernel? Just as in this thread ... (http://wiki.linuxmce.org/index.php/Installing_LinuxMCE_on_Virtual_Machine_via_KVM)

I found another post in this context: Disable device assignment without interrupt (http://www.serverphorums.com/read.php?12,349528) But I do not understand what I can do with it.

my proxmox version is:

root@fcprox01:# pveversion -v
pve-manager: 2.0-10 (pve-manager/2.0/7a10f3e6)
running kernel: 2.6.32-6-pve
proxmox-ve-2.6.32: 2.0-52
pve-kernel-2.6.32-6-pve: 2.6.32-52
lvm2: 2.02.86-1pve1
clvm: 2.02.86-1pve1
corosync-pve: 1.4.1-1
openais-pve: 1.1.4-1
libqb: 0.6.0-1
redhat-cluster-pve: 3.1.7-1
pve-cluster: 1.0-11
qemu-server: 2.0-3
pve-firmware: 1.0-13
libpve-common-perl: 1.0-7
libpve-access-control: 1.0-2
libpve-storage-perl: 2.0-6
vncterm: 1.0-2
vzctl: 3.0.29-3pve3
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 0.15.0-1
ksm-control-daemon: 1.1-1

regards, proxmox

dietmar
11-19-2011, 08:50 AM
Or do I have to recompile the kernel? Just as in this thread ... (http://wiki.linuxmce.org/index.php/Installing_LinuxMCE_on_Virtual_Machine_via_KVM)


That should be enabled (see (/boot/config-*))

maxprox
11-19-2011, 09:38 AM
That should be enabled (see (/boot/config-*))

But I can not find a line with the word remapping:

root@fcprox01:/boot# cat config-2.6.32-6-pve | grep -i remapping



And the other question is what to do with the line:

"kvm_iommu_map_guest: No interrupt remapping support, disallowing device assignment. Re-enble with "allow_unsafe_assigned_interrupts=1" module option.

there are somewhere a possibility to switch it on?
EDIT:
I found some different Parameter for the VMID.conf and pci-passthrough:

## hostpci0: 03:00.0
## args: -pcidevice host=03:00.0
## args: -device pci-assign,host=03:00.0,id=hostpci0
which parameters are correct for proxmox 2.0 and my solution?

maxprox

maxprox
11-21-2011, 02:56 PM
The work is done:
with the help of tin in this thread (http://forum.proxmox.com/threads/6952-PCI-PCIe-passthrough?p=43180#post43180)
I have create the file:

root@fcprox01:/etc/modprobe.d# cat kvm_iommu_map_guest.conf
options kvm allow_unsafe_assigned_interrupts=1
root@fcprox01:/etc/modprobe.d#

And now my VDR (TV-Video-Recorder) works as it should!

mbello
11-23-2011, 07:26 AM
I found some different Parameter for the VMID.conf and pci-passthrough:

## hostpci0: 03:00.0
## args: -pcidevice host=03:00.0
## args: -device pci-assign,host=03:00.0,id=hostpci0
which parameters are correct for proxmox 2.0 and my solution?


Please I would also like to know the answer to this question. Anyone?

dietmar
11-23-2011, 09:24 AM
the hostpci0 config should work

maxprox
11-23-2011, 10:48 PM
Yes,


hostpci0: xx:xx.x

For example here my VMID.conf:


root@fcprox01:/etc/pve/nodes/fcprox01/qemu-server# cat 113.conf
net0: virtio=0A:22:B1:6B:00:E9,bridge=vmbr0
ide2: local:iso/ubuntu-11.04-server-amd64.iso,media=cdrom
name: vdr
bootdisk: virtio0
virtio0: local:113/vm-113-disk-1.raw
virtio1: vdr2TB:vm-113-disk-1,backup=no
cores: 2
sockets: 1
ostype: l26
memory: 2048
keyboard: de
hostpci0: 03:00.0
root@fcprox01:/etc/pve/nodes/fcprox01/qemu-server#

But have also a look at here (http://forum.proxmox.com/threads/7255-PCI-pass-through?p=43288#post43288):confused:

maxprox
03-24-2012, 01:24 PM
Hello,
Now I had three months of a stable system based on PCI passthrough ...
since the last kernel update, it's over.

Has anything changed with respect to PCI passthrough?
Or in relation to IRQ remapping or whatever?

some syslog messages from the proxmox host, if I start the VM 112 in question

Mar 24 01:23:31 fcprox01 ntpd_intres[1953]: DNS zentyal-vm -> 172.1.1.3
Mar 24 01:23:31 fcprox01 ntpd_intres[1953]: DNS zentyal-vm.peter.lan -> 172.1.1.3
Mar 24 01:25:17 fcprox01 qm[3121]: start VM 112: UPID:fcprox01:00000C31:00007FC2:4F6D146D:qmstart:1 12:root@pam:
Mar 24 01:25:17 fcprox01 qm[3118]: <root@pam> starting task UPID:fcprox01:00000C31:00007FC2:4F6D146D:qmstart:1 12:root@pam:
Mar 24 01:25:17 fcprox01 kernel: pci-stub 0000:03:00.0: claimed by stub
Mar 24 01:25:17 fcprox01 kernel: pci-stub 0000:03:00.0: claimed by stub
Mar 24 01:25:17 fcprox01 kernel: pci-stub 0000:03:00.0: restoring config space at offset 0x1 (was 0x100400, writing 0x100007)
Mar 24 01:25:18 fcprox01 kernel: device tap112i0 entered promiscuous mode
Mar 24 01:25:18 fcprox01 kernel: vmbr0: port 3(tap112i0) entering forwarding state
Mar 24 01:25:18 fcprox01 kernel: pci-stub 0000:03:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
Mar 24 01:25:18 fcprox01 kernel: pci-stub 0000:03:00.0: restoring config space at offset 0x1 (was 0x100400, writing 0x100007)
Mar 24 01:25:18 fcprox01 kernel: assign device: host bdf = 3:0:0
Mar 24 01:25:18 fcprox01 kernel: alloc irq_desc for 42 on node -1
Mar 24 01:25:18 fcprox01 kernel: alloc kstat_irqs on node -1
Mar 24 01:25:18 fcprox01 kernel: pci-stub 0000:03:00.0: irq 42 for MSI/MSI-X
Mar 24 01:25:18 fcprox01 kernel: pci-stub 0000:03:00.0: restoring config space at offset 0x1 (was 0x100400, writing 0x100407)
Mar 24 01:25:18 fcprox01 qm[3118]: <root@pam> end task UPID:fcprox01:00000C31:00007FC2:4F6D146D:qmstart:1 12:root@pam: OK
Mar 24 01:25

These lines relate to the PCI TV card

Mar 24 01:25:17 fcprox01 kernel: pci-stub 0000:03:00.0: claimed by stub

Some messages of the VM 112 (easyVDR) dmesg:
If I understand correctly, the TV card (SAA716x / a TechnoTrend S2 6400) is first installed and then discarded (?)


[ 5.685539] SAA716x FF 0000:00:10.0: PCI INT A -> Link[LNKD] -> GSI 11
(level, high) -> IRQ 11
[ 5.685565] SAA716x FF 0000:00:10.0: setting latency timer to 64
[ 5.756392] psmouse serio1: ID: 10 00 64
[ 5.774123] init: easyvdr-create-ramdisk main process (514) killed by TERM
signal
[ 5.797103] type=1400 audit(1332549254.538:2): apparmor="STATUS"
operation="profile_load" name="/sbin/dhclient" pid=576 comm="apparmor_parser"
....
[ 7.452016] SAA716x FF FPGA version 1.09
[ 7.501850] init: ssh main process (526) terminated with status 255
[ 7.509216] SAA716x FF loader version 1.03
[ 7.783967] type=1400 audit(1332549256.522:5): apparmor="STATUS"
operation="profile_replace" name="/sbin/dhclient" pid=924 comm="apparmor_parser"
....
[ 7.892044] init: failsafe main process (704) killed by TERM signal
[ 8.174267] DVB: registering new adapter (SAA716x dvb adapter)
[ 8.289877] stv6110x_attach: Attaching STV6110x
[ 8.292017] DVB: registering adapter 0 frontend 0 (STV090x Multistandard)...
[ 8.292124] DVB: registering new adapter (SAA716x dvb adapter)
[ 8.292356] stv6110x_attach: Attaching STV6110x
[ 8.326433] DVB: registering adapter 1 frontend 0 (STV090x Multistandard)...
[ 8.392265] RPC: Registered named UNIX socket transport module.
.....

[ 18.325389] SAA716x FF 0000:00:10.0: PCI INT A disabled
[ 18.325396] SAA716x FF: probe of 0000:00:10.0 failed with error -1
[ 18.405082] init: udev-fallback-graphics main process (1522) terminated with
status 1
[ 18.430679] init: easyvdr-vdr post-start process (1563) terminated with
status 1
.....





pveversion -v
pve-manager: 2.0-42 (pve-manager/2.0/3d6d8258)
running kernel: 2.6.32-10-pve
proxmox-ve-2.6.32: 2.0-63
pve-kernel-2.6.32-10-pve: 2.6.32-63
pve-kernel-2.6.32-6-pve: 2.6.32-55
pve-kernel-2.6.32-7-pve: 2.6.32-60
lvm2: 2.02.88-2pve2
clvm: 2.02.88-2pve2
corosync-pve: 1.4.1-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.8-3
resource-agents-pve: 3.9.2-3

fence-agents-pve: 3.1.7-1
pve-cluster: 1.0-25
qemu-server: 2.0-28
pve-firmware: 1.0-15
libpve-common-perl: 1.0-21
libpve-access-control: 1.0-17
libpve-storage-perl: 2.0-14
vncterm: 1.0-2
vzctl: 3.0.30-2pve2
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.0-7
ksm-control-daemon: 1.1-1

regards
maxprox

maxprox
03-24-2012, 07:40 PM
Indeed
the reason was the new version of pve-qemu-kvm: 1.0-7
after a downgrade to the version 1.0-3 it works as before

what I have done is:

wget http://download.proxmox.com/debian/d....0-3_amd64.deb (http://download.proxmox.com/debian/dists/squeeze/pve/binary-amd64/pve-qemu-kvm_1.0-3_amd64.deb)
dpkg -i pve-qemu-kvm_1.0-3_amd64.deb

all machines restart

t@fcprox01:~# pveversion -v
pve-manager: 2.0-42 (pve-manager/2.0/3d6d8258)
running kernel: 2.6.32-10-pve
proxmox-ve-2.6.32: 2.0-63
pve-kernel-2.6.32-10-pve: 2.6.32-63
pve-kernel-2.6.32-6-pve: 2.6.32-55
pve-kernel-2.6.32-7-pve: 2.6.32-60
lvm2: 2.02.88-2pve2
clvm: 2.02.88-2pve2
corosync-pve: 1.4.1-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.8-3
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.7-1
pve-cluster: 1.0-25
qemu-server: 2.0-28
pve-firmware: 1.0-15
libpve-common-perl: 1.0-21
libpve-access-control: 1.0-17
libpve-storage-perl: 2.0-14
vncterm: 1.0-2
vzctl: 3.0.30-2pve2
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.0-3 <== <==
ksm-control-daemon: 1.1-1
root@fcprox01:

regards
maxprox

maxprox
01-08-2013, 05:31 PM
Hi,
since more than a half year last weekend I try

aptitude update && aptitude full-upgrade

an now with the new versions there are no problems with pci passthrough - like bevor
with pve-qemu-kvm version <= 1.0-5
This problem for me is solved -

thank you



root@fcprox01:~# pveversion -v
pve-manager: 2.2-32 (pve-manager/2.2/3089a616)
running kernel: 2.6.32-17-pved
proxmox-ve-2.6.32: 2.2-83
pve-kernel-2.6.32-12-pve: 2.6.32-68
pve-kernel-2.6.32-17-pve: 2.6.32-83
lvm2: 2.02.95-1pve2
clvm: 2.02.95-1pve2
corosync-pve: 1.4.4-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.93-2
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.9-1
pve-cluster: 1.0-34
qemu-server: 2.0-71
pve-firmware: 1.0-21
libpve-common-perl: 1.0-41
libpve-access-control: 1.0-25
libpve-storage-perl: 2.0-36
vncterm: 1.0-3
vzctl: 4.0-1pve2
vzprocps: 2.0.11-2
vzquota: 3.1-1
pve-qemu-kvm: 1.3-10
ksm-control-daemon: 1.1-1
root@fcprox01:~#

BlackBurn
03-11-2013, 08:27 PM
Maybe a strange question, but did you already updated to the latest version 2.3 ?
pve-qemu-kvm: 1.4-6

I got some errors, posted in this post: http://forum.proxmox.com/threads/13134-Problems-PCI-Passthrough-(TeVii-S471)