1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

PCI Passthrough - No interrupt remapping support

Discussion in 'Proxmox VE: Installation and configuration' started by maxprox, Nov 18, 2011.

  1. maxprox

    maxprox Member

    Joined:
    Aug 23, 2011
    Messages:
    231
    Likes Received:
    5
    Hello,

    As in the other threads I've PCI passthrough configured:
    On the Proxmox host I do nothing with the PCI-card I want to use for "pci-passthrough". For example, I have not disabled it in the host,
    also I have not disable a module, because for this PCI-card there is no module in the host. I'm not sure if that caused the fault.
    The VM who should get the pci-card is Ubuntu Server 11.04

    the VM config:

    Code:
    net0: virtio=0A:22:B1:6B:00:E9,bridge=vmbr0
    ide2: local:iso/ubuntu-11.04-server-amd64.iso,media=cdrom
    name: vdr
    bootdisk: virtio0
    virtio0: local:113/vm-113-disk-1.raw
    cores: 2
    ostype: l26
    memory: 2048
    sockets: 1
    hostpci0: 03:00.0
    In the BIOS of the Supermicro X8SIL-F Bord I activate the Vt-d an the output of dmesg | grep -e DMAR -e IOMMU is:

    Code:
    root@fcprox01:# dmesg | grep -e DMAR -e IOMMU
    ACPI: DMAR 00000000bf7b00f0 00090 (v01    AMI  OEMDMAR 00000001 MSFT 00000097)
    Intel-IOMMU: enabled
    DMAR: Host address width 36
    DMAR: DRHD base: 0x000000fed90000 flags: 0x1
    IOMMU fed90000: ver 1:0 cap c90780106f0462 ecap f020e3
    DMAR: RMRR base: 0x000000000ed000 end: 0x000000000effff
    DMAR: RMRR base: 0x000000bf7ed000 end: 0x000000bf7fffff
    DMAR: No ATSR found
    IOMMU 0xfed90000: using Queued invalidation
    IOMMU: Setting RMRR:
    IOMMU: Setting identity map for device 0000:00:1d.0 [0xbf7ed000 - 0xbf800000]
    IOMMU: Setting identity map for device 0000:00:1a.0 [0xbf7ed000 - 0xbf800000]
    IOMMU: Setting identity map for device 0000:00:1d.0 [0xed000 - 0xf0000]
    IOMMU: Setting identity map for device 0000:00:1a.0 [0xed000 - 0xf0000]
    IOMMU: Prepare 0-16MiB unity mapping for LPC
    IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0x1000000]
    
    The Kernel parameter is:
    Code:
    root@fcprox01:/boot# cat /etc/default/grub | grep iommu
    GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"


    The PCI-Card in the Proxmox host I mean is:
    Code:
    root@fcprox01:# lspci
    
    03:00.0 Multimedia controller: Philips Semiconductors SAA7160 (rev 02)
    
    The error in the syslog I think is the line "kvm_iommu_map_guest: No interrupt remapping support, disallowing device assignment. Re-enble with "allow_unsafe_assigned_interrupts=1" module option.

    Code:
    Nov 18 22:22:20 fcprox01 pvedaemon[17882]: VM 113 start
    Nov 18 22:22:20 fcprox01 kernel: pci-stub 0000:03:00.0: claimed by stub
    Nov 18 22:22:20 fcprox01 kernel: pci-stub 0000:03:00.0: claimed by stub
    Nov 18 22:22:20 fcprox01 kernel: pci-stub 0000:03:00.0: restoring config space at offset 0x1 (was 0x100400, writing 0x100007)
    Nov 18 22:22:21 fcprox01 kernel: device tap113i0 entered promiscuous mode
    Nov 18 22:22:21 fcprox01 kernel: vmbr0: port 3(tap113i0) entering forwarding state
    Nov 18 22:22:21 fcprox01 kernel: pci-stub 0000:03:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
    Nov 18 22:22:21 fcprox01 kernel: pci-stub 0000:03:00.0: restoring config space at offset 0x1 (was 0x100400, writing 0x100007)
    Nov 18 22:22:21 fcprox01 kernel: kvm_iommu_map_guest: No interrupt remapping support, disallowing device assignment. Re-enble with "allow_unsafe_assigned_interrupts=1" module option.
    Nov 18 22:22:21 fcprox01 kernel: pci-stub 0000:03:00.0: PCI INT A disabled
    Nov 18 22:22:21 fcprox01 pvedaemon[17882]: VM 113 start failed: command '/usr/bin/kvm -id 113 -chardev socket,id=monitor,path=/var/run/qemu-server/113.mon,server,nowait -mon chardev=monitor,mode=readline -vnc unix:/var/run/qemu-server/113.vnc,x509,password -pidfile /var/run/qemu-server/113.pid -daemonize -readconfig /usr/share/qemu-server/pve-usb.cfg -device usb-tablet,bus=ehci.0,port=6 -device pci-assign,host=03:00.0,id=hostpci0,bus=pci.0,addr=0x10 -name vdr -smp sockets=1,cores=2 -nodefaults -boot menu=on,order=cdn -vga cirrus -tdf -k en-us -drive file=/var/lib/vz/template/iso/ubuntu-11.04-server-amd64.iso,if=none,id=drive-ide2,media=cdrom,aio=native -device ide-drive,bus=ide.1,unit=0,drive=drive-ide2,id=device-ide2 -drive file=/var/lib/vz/images/113/vm-113-disk-1.raw,if=none,id=drive-virtio0,aio=native,boot=on -device virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa -m 2048 -netdev type=tap,id=net0,ifname=tap113i0,script=/var/lib/qemu-server/pve-bridge -device virtio-net-pci,romfile=,mac=0A:22:B1:6B:00:E9,netdev=net0,bus=pci.0,addr=0x12' failed: exit code 1
    Nov 18 22:22:21 fcprox01 pvedaemon[17882]: start failed: command '/usr/bin/kvm -id 113 -chardev socket,id=monitor,path=/var/run/qemu-server/113.mon,server,nowait -mon chardev=monitor,mode=readline -vnc unix:/var/run/qemu-server/113.vnc,x509,password -pidfile /var/run/qemu-server/113.pid -daemonize -readconfig /usr/share/qemu-server/pve-usb.cfg -device usb-tablet,bus=ehci.0,port=6 -device pci-assign,host=03:00.0,id=hostpci0,bus=pci.0,addr=0x10 -name vdr -smp sockets=1,cores=2 -nodefaults -boot menu=on,order=cdn -vga cirrus -tdf -k en-us -drive file=/var/lib/vz/template/iso/ubuntu-11.04-server-amd64.iso,if=none,id=drive-ide2,media=cdrom,aio=native -device ide-drive,bus=ide.1,unit=0,drive=drive-ide2,id=device-ide2 -drive file=/var/lib/vz/images/113/vm-113-disk-1.raw,if=none,id=drive-virtio0,aio=native,boot=on -device virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa -m 2048 -netdev type=tap,id=net0,ifname=tap113i0,script=/var/lib/qemu-server/pve-bridge -device virtio-net-pci,romfile=,mac=0A:22:B1:6B:00:E9,netdev=net0,bus=pci.0,addr=0x12' failed: exit code 1
    Nov 18 22:22:21 fcprox01 pvedaemon[17611]: <root@pam> end task UPID:fcprox01:000045DA:001928D6:4EC6CC8C:qmstart:113:root@pam: unexpected status
    Nov 18 22:22:21 fcprox01 kernel: vmbr0: port 3(tap113i0) entering disabled state
    Nov 18 22:22:21 fcprox01 kernel: vmbr0: port 3(tap113i0) entering disabled state
    Nov 18
    what can I do?

    regards,
    proxmox
     
    #1 maxprox, Nov 18, 2011
    Last edited: Nov 19, 2011
  2. maxprox

    maxprox Member

    Joined:
    Aug 23, 2011
    Messages:
    231
    Likes Received:
    5
    Hello, can someone tell me whether the following kernel options are already turned on?
    Or do I have to recompile the kernel? Just as in this thread ...

    I found another post in this context: Disable device assignment without interrupt But I do not understand what I can do with it.

    my proxmox version is:
    Code:
    root@fcprox01:# pveversion -v
    pve-manager: 2.0-10 (pve-manager/2.0/7a10f3e6)
    running kernel: 2.6.32-6-pve
    proxmox-ve-2.6.32: 2.0-52
    pve-kernel-2.6.32-6-pve: 2.6.32-52
    lvm2: 2.02.86-1pve1
    clvm: 2.02.86-1pve1
    corosync-pve: 1.4.1-1
    openais-pve: 1.1.4-1
    libqb: 0.6.0-1
    redhat-cluster-pve: 3.1.7-1
    pve-cluster: 1.0-11
    qemu-server: 2.0-3
    pve-firmware: 1.0-13
    libpve-common-perl: 1.0-7
    libpve-access-control: 1.0-2
    libpve-storage-perl: 2.0-6
    vncterm: 1.0-2
    vzctl: 3.0.29-3pve3
    vzprocps: 2.0.11-2
    vzquota: 3.0.12-3
    pve-qemu-kvm: 0.15.0-1
    ksm-control-daemon: 1.1-1
    
    regards, proxmox
     
    #2 maxprox, Nov 18, 2011
    Last edited: Nov 19, 2011
  3. dietmar

    dietmar Proxmox Staff Member
    Staff Member

    Joined:
    Apr 28, 2005
    Messages:
    15,375
    Likes Received:
    176
    That should be enabled (see (/boot/config-*))
     
  4. maxprox

    maxprox Member

    Joined:
    Aug 23, 2011
    Messages:
    231
    Likes Received:
    5
    But I can not find a line with the word remapping:
    Code:
    root@fcprox01:/boot# cat  config-2.6.32-6-pve | grep -i remapping
    
    
    And the other question is what to do with the line:
    Code:
    "kvm_iommu_map_guest: No interrupt remapping support, disallowing device assignment. Re-enble with "allow_unsafe_assigned_interrupts=1" module option.
    
    there are somewhere a possibility to switch it on?
    EDIT:
    I found some different Parameter for the VMID.conf and pci-passthrough:
    Code:
    ## hostpci0: 03:00.0
    ## args: -pcidevice host=03:00.0
    ## args: -device pci-assign,host=03:00.0,id=hostpci0 
    which parameters are correct for proxmox 2.0 and my solution?

    maxprox
     
    #4 maxprox, Nov 19, 2011
    Last edited: Nov 19, 2011
  5. maxprox

    maxprox Member

    Joined:
    Aug 23, 2011
    Messages:
    231
    Likes Received:
    5
    The work is done:
    with the help of tin in this thread
    I have create the file:
    Code:
    root@fcprox01:/etc/modprobe.d# cat kvm_iommu_map_guest.conf
    options kvm allow_unsafe_assigned_interrupts=1
    root@fcprox01:/etc/modprobe.d#
    
    And now my VDR (TV-Video-Recorder) works as it should!
     
  6. mbello

    mbello New Member

    Joined:
    Nov 23, 2011
    Messages:
    10
    Likes Received:
    0
    Please I would also like to know the answer to this question. Anyone?
     
  7. dietmar

    dietmar Proxmox Staff Member
    Staff Member

    Joined:
    Apr 28, 2005
    Messages:
    15,375
    Likes Received:
    176
    the hostpci0 config should work
     
  8. maxprox

    maxprox Member

    Joined:
    Aug 23, 2011
    Messages:
    231
    Likes Received:
    5
    Yes,

    Code:
      hostpci0: xx:xx.x 
    For example here my VMID.conf:

    Code:
    root@fcprox01:/etc/pve/nodes/fcprox01/qemu-server# cat 113.conf 
    net0: virtio=0A:22:B1:6B:00:E9,bridge=vmbr0
    ide2: local:iso/ubuntu-11.04-server-amd64.iso,media=cdrom
    name: vdr
    bootdisk: virtio0
    virtio0: local:113/vm-113-disk-1.raw
    virtio1: vdr2TB:vm-113-disk-1,backup=no
    cores: 2
    sockets: 1
    ostype: l26
    memory: 2048
    keyboard: de
    hostpci0: 03:00.0
    root@fcprox01:/etc/pve/nodes/fcprox01/qemu-server# 
    But have also a look at here :confused:
     
  9. maxprox

    maxprox Member

    Joined:
    Aug 23, 2011
    Messages:
    231
    Likes Received:
    5
    Hello,
    Now I had three months of a stable system based on PCI passthrough ...
    since the last kernel update, it's over.

    Has anything changed with respect to PCI passthrough?
    Or in relation to IRQ remapping or whatever?

    some syslog messages from the proxmox host, if I start the VM 112 in question
    Code:
    Mar 24 01:23:31 fcprox01 ntpd_intres[1953]: DNS zentyal-vm -> 172.1.1.3
    Mar 24 01:23:31 fcprox01 ntpd_intres[1953]: DNS zentyal-vm.peter.lan -> 172.1.1.3
    Mar 24 01:25:17 fcprox01 qm[3121]: start VM 112: UPID:fcprox01:00000C31:00007FC2:4F6D146D:qmstart:112:root@pam:
    Mar 24 01:25:17 fcprox01 qm[3118]: <root@pam> starting task UPID:fcprox01:00000C31:00007FC2:4F6D146D:qmstart:112:root@pam:
    Mar 24 01:25:17 fcprox01 kernel: pci-stub 0000:03:00.0: claimed by stub
    Mar 24 01:25:17 fcprox01 kernel: pci-stub 0000:03:00.0: claimed by stub
    Mar 24 01:25:17 fcprox01 kernel: pci-stub 0000:03:00.0: restoring config space at offset 0x1 (was 0x100400, writing 0x100007)
    Mar 24 01:25:18 fcprox01 kernel: device tap112i0 entered promiscuous mode
    Mar 24 01:25:18 fcprox01 kernel: vmbr0: port 3(tap112i0) entering forwarding state
    Mar 24 01:25:18 fcprox01 kernel: pci-stub 0000:03:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
    Mar 24 01:25:18 fcprox01 kernel: pci-stub 0000:03:00.0: restoring config space at offset 0x1 (was 0x100400, writing 0x100007)
    Mar 24 01:25:18 fcprox01 kernel: assign device: host bdf = 3:0:0
    Mar 24 01:25:18 fcprox01 kernel:  alloc irq_desc for 42 on node -1
    Mar 24 01:25:18 fcprox01 kernel:  alloc kstat_irqs on node -1
    Mar 24 01:25:18 fcprox01 kernel: pci-stub 0000:03:00.0: irq 42 for MSI/MSI-X
    Mar 24 01:25:18 fcprox01 kernel: pci-stub 0000:03:00.0: restoring config space at offset 0x1 (was 0x100400, writing 0x100407)
    Mar 24 01:25:18 fcprox01 qm[3118]: <root@pam> end task UPID:fcprox01:00000C31:00007FC2:4F6D146D:qmstart:112:root@pam: OK
    Mar 24 01:25
    These lines relate to the PCI TV card
    Code:
    Mar 24 01:25:17 fcprox01 kernel: pci-stub 0000:03:00.0: claimed by stub
    Some messages of the VM 112 (easyVDR) dmesg:
    If I understand correctly, the TV card (SAA716x / a TechnoTrend S2 6400) is first installed and then discarded (?)

    Code:
    [    5.685539] SAA716x FF 0000:00:10.0: PCI INT A -> Link[LNKD] -> GSI 11
    (level, high) -> IRQ 11
    [    5.685565] SAA716x FF 0000:00:10.0: setting latency timer to 64
    [    5.756392] psmouse serio1: ID: 10 00 64
    [    5.774123] init: easyvdr-create-ramdisk main process (514) killed by TERM
    signal
    [    5.797103] type=1400 audit(1332549254.538:2): apparmor="STATUS"
    operation="profile_load" name="/sbin/dhclient" pid=576 comm="apparmor_parser"
    ....
    [    7.452016] SAA716x FF FPGA version 1.09
    [    7.501850] init: ssh main process (526) terminated with status 255
    [    7.509216] SAA716x FF loader version 1.03
    [    7.783967] type=1400 audit(1332549256.522:5): apparmor="STATUS"
    operation="profile_replace" name="/sbin/dhclient" pid=924 comm="apparmor_parser"
    ....
    [    7.892044] init: failsafe main process (704) killed by TERM signal
    [    8.174267] DVB: registering new adapter (SAA716x dvb adapter)
    [    8.289877] stv6110x_attach: Attaching STV6110x
    [    8.292017] DVB: registering adapter 0 frontend 0 (STV090x Multistandard)...
    [    8.292124] DVB: registering new adapter (SAA716x dvb adapter)
    [    8.292356] stv6110x_attach: Attaching STV6110x
    [    8.326433] DVB: registering adapter 1 frontend 0 (STV090x Multistandard)...
    [    8.392265] RPC: Registered named UNIX socket transport module.
    .....
    
    [   18.325389] SAA716x FF 0000:00:10.0: PCI INT A disabled
    [   18.325396] SAA716x FF: probe of 0000:00:10.0 failed with error -1
    [   18.405082] init: udev-fallback-graphics main process (1522) terminated with
    status 1
    [   18.430679] init: easyvdr-vdr post-start process (1563) terminated with
    status 1
    .....
    



    pveversion -v
    pve-manager: 2.0-42 (pve-manager/2.0/3d6d8258)
    running kernel: 2.6.32-10-pve
    proxmox-ve-2.6.32: 2.0-63
    pve-kernel-2.6.32-10-pve: 2.6.32-63
    pve-kernel-2.6.32-6-pve: 2.6.32-55
    pve-kernel-2.6.32-7-pve: 2.6.32-60
    lvm2: 2.02.88-2pve2
    clvm: 2.02.88-2pve2
    corosync-pve: 1.4.1-1
    openais-pve: 1.1.4-2
    libqb: 0.10.1-2
    redhat-cluster-pve: 3.1.8-3
    resource-agents-pve: 3.9.2-3

    fence-agents-pve: 3.1.7-1
    pve-cluster: 1.0-25
    qemu-server: 2.0-28
    pve-firmware: 1.0-15
    libpve-common-perl: 1.0-21
    libpve-access-control: 1.0-17
    libpve-storage-perl: 2.0-14
    vncterm: 1.0-2
    vzctl: 3.0.30-2pve2
    vzprocps: 2.0.11-2
    vzquota: 3.0.12-3
    pve-qemu-kvm: 1.0-7
    ksm-control-daemon: 1.1-1

    regards
    maxprox
     
    #9 maxprox, Mar 24, 2012
    Last edited: Mar 24, 2012
  10. maxprox

    maxprox Member

    Joined:
    Aug 23, 2011
    Messages:
    231
    Likes Received:
    5
    Re: PCI Passthrough - No interrupt remapping support [solved]

    Indeed
    the reason was the new version of pve-qemu-kvm: 1.0-7
    after a downgrade to the version 1.0-3 it works as before

    what I have done is:

    wget http://download.proxmox.com/debian/d....0-3_amd64.deb
    dpkg -i pve-qemu-kvm_1.0-3_amd64.deb

    all machines restart

    t@fcprox01:~# pveversion -v
    pve-manager: 2.0-42 (pve-manager/2.0/3d6d8258)
    running kernel: 2.6.32-10-pve
    proxmox-ve-2.6.32: 2.0-63
    pve-kernel-2.6.32-10-pve: 2.6.32-63
    pve-kernel-2.6.32-6-pve: 2.6.32-55
    pve-kernel-2.6.32-7-pve: 2.6.32-60
    lvm2: 2.02.88-2pve2
    clvm: 2.02.88-2pve2
    corosync-pve: 1.4.1-1
    openais-pve: 1.1.4-2
    libqb: 0.10.1-2
    redhat-cluster-pve: 3.1.8-3
    resource-agents-pve: 3.9.2-3
    fence-agents-pve: 3.1.7-1
    pve-cluster: 1.0-25
    qemu-server: 2.0-28
    pve-firmware: 1.0-15
    libpve-common-perl: 1.0-21
    libpve-access-control: 1.0-17
    libpve-storage-perl: 2.0-14
    vncterm: 1.0-2
    vzctl: 3.0.30-2pve2
    vzprocps: 2.0.11-2
    vzquota: 3.0.12-3
    pve-qemu-kvm: 1.0-3 <== <==
    ksm-control-daemon: 1.1-1
    root@fcprox01:

    regards
    maxprox
     
    #10 maxprox, Mar 24, 2012
    Last edited: Jan 8, 2013
  11. maxprox

    maxprox Member

    Joined:
    Aug 23, 2011
    Messages:
    231
    Likes Received:
    5
    Re: PCI Passthrough - No interrupt remapping support [solved]

    Hi,
    since more than a half year last weekend I try
    Code:
    aptitude update && aptitude full-upgrade
    an now with the new versions there are no problems with pci passthrough - like bevor
    with pve-qemu-kvm version <= 1.0-5
    This problem for me is solved -

    thank you


    Code:
    root@fcprox01:~# pveversion -v
    pve-manager: 2.2-32 (pve-manager/2.2/3089a616)
    running kernel: 2.6.32-17-pved
    proxmox-ve-2.6.32: 2.2-83
    pve-kernel-2.6.32-12-pve: 2.6.32-68
    pve-kernel-2.6.32-17-pve: 2.6.32-83
    lvm2: 2.02.95-1pve2
    clvm: 2.02.95-1pve2
    corosync-pve: 1.4.4-1
    openais-pve: 1.1.4-2
    libqb: 0.10.1-2
    redhat-cluster-pve: 3.1.93-2
    resource-agents-pve: 3.9.2-3
    fence-agents-pve: 3.1.9-1
    pve-cluster: 1.0-34
    qemu-server: 2.0-71
    pve-firmware: 1.0-21
    libpve-common-perl: 1.0-41
    libpve-access-control: 1.0-25
    libpve-storage-perl: 2.0-36
    vncterm: 1.0-3
    vzctl: 4.0-1pve2
    vzprocps: 2.0.11-2
    vzquota: 3.1-1
    pve-qemu-kvm: 1.3-10
    ksm-control-daemon: 1.1-1
    root@fcprox01:~# 
    
    
     
  12. BlackBurn

    BlackBurn New Member

    Joined:
    Mar 11, 2013
    Messages:
    2
    Likes Received:
    0

Share This Page