After Upgrade (RC -> final) -> PCI Passthrough Problems

Hi Dietmar,

yes it works for me!
Thank's a lot!

First i upgraded again to "qemu-server_2.0-33_amd64.deb" by "aptitude update" & "aptitude full-upgrade".
Then, i installed the new Package "pve-qemu-kvm_1.0-9_amd64.deb".
And now, it works as before the upgrade to final, without the additional argument "prefer_msi=on".


pve-manager: 2.0-54 (pve-manager/2.0/4b59ea39)
running kernel: 2.6.32-10-pve
proxmox-ve-2.6.32: 2.0-63
pve-kernel-2.6.32-10-pve: 2.6.32-63
pve-kernel-2.6.32-7-pve: 2.6.32-60
lvm2: 2.02.88-2pve2
clvm: 2.02.88-2pve2
corosync-pve: 1.4.1-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.8-3
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.7-2
pve-cluster: 1.0-26
qemu-server: 2.0-33
pve-firmware: 1.0-15
libpve-common-perl: 1.0-23
libpve-access-control: 1.0-17
libpve-storage-perl: 2.0-16
vncterm: 1.0-2
vzctl: 3.0.30-2pve2
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.0-9
ksm-control-daemon: 1.1-1
Click to expand...


Alex
 
trendco said:
Hi Dietmar,

yes it works for me!
Thank's a lot!

First i upgraded again to "qemu-server_2.0-33_amd64.deb" by "aptitude update" & "aptitude full-upgrade".
Then, i installed the new Package "pve-qemu-kvm_1.0-9_amd64.deb".
And now, it works as before the upgrade to final, without the additional argument "prefer_msi=on".
Alex
Click to expand...

Hello,
today I also update by "aptitude update" & "aptitude full-upgrade".
=> PCI Passthrough doesn't work - as after the last updates, since > pve-qemu-kvm_1.0-7
I get:
Code: 
dmesg

[   18.325389] SAA716x FF 0000:00:10.0: PCI INT A disabled
[   18.325396] SAA716x FF: probe of 0000:00:10.0 failed with error -1

Also Ditmars fix "pve-qemu-kvm_1.0-9_amd64.deb" doesn't work for me

the only solution for my PCI passthrough is to downgrade to version pve-qemu-kvm_1.0-5
Code: 
root@fcprox01:~# pveversion -v
pve-manager: 2.1-1 (pve-manager/2.1/f9b0f63a)
running kernel: 2.6.32-11-pve
proxmox-ve-2.6.32: 2.0-66
pve-kernel-2.6.32-10-pve: 2.6.32-63
pve-kernel-2.6.32-11-pve: 2.6.32-66
lvm2: 2.02.95-1pve2
clvm: 2.02.95-1pve2
corosync-pve: 1.4.3-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.8-3
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.7-2
pve-cluster: 1.0-26
qemu-server: 2.0-39
pve-firmware: 1.0-15
libpve-common-perl: 1.0-27
libpve-access-control: 1.0-21
libpve-storage-perl: 2.0-18
vncterm: 1.0-2
vzctl: 3.0.30-2pve5
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.0-5     [B]<<==[/B]
ksm-control-daemon: 1.1-1
root@fcprox01:~#

maxprox
 
maxprox,

Maybe it has been updated since your last post, but I just updated and it works for me. I use "hostpci0: XX:XX.X" in the .conf file (no msi or intx options). However, I found that passing through my PCIe device works, while PCI does not. Maybe it's due to:
- The VT-d specification states that all conventional PCI devices behind a PCIe-to-PCI bridge have to be assigned to the same domain.
- PCIe devices do not have this restriction.
Click to expand...
(from http://wiki.xensource.com/xenwiki/VTdHowTo)
This may be specific to my motherboard, which probably uses a PCIe-to-PCI bridge.

Also, on the Proxmox wiki page about pci passthrough (http://pve.proxmox.com/wiki/Pci_passthrough), I had to use different options, as what's provided there resulted in a kernel panic for me. Here is what works for my AMD Opteron 4130:
Code: 
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=on,pt"
Anyone else experience the same? If so, maybe we should update the wiki.
 
jram said:
maxprox,

Maybe it has been updated since your last post, but I just updated and it works for me. I use "hostpci0: XX:XX.X" in the .conf file (no msi or intx options). However, I found that passing through my PCIe device works, while PCI does not. Maybe it's due to:
(from http://wiki.xensource.com/xenwiki/VTdHowTo)
This may be specific to my motherboard, which probably uses a PCIe-to-PCI bridge.

Also, on the Proxmox wiki page about pci passthrough (http://pve.proxmox.com/wiki/Pci_passthrough), I had to use different options, as what's provided there resulted in a kernel panic for me. Here is what works for my AMD Opteron 4130:
Code: 
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=on,pt"
Anyone else experience the same? If so, maybe we should update the wiki.
Click to expand...

Hello jram,

thank you for your answer. Unfortunately it does not work for me.
I do a new full-upgrade with exact the same result I wrote above.
I work with a supermicro X8SIL-F board - Intel 3420 Chipset with a Xeon X3450 QC CPU
My problem child is a Technotrend S2-6400 full-HD PCIe x1 TV-card.
The only solution for me is
1:
downgrade to pve-qemu-kvm 1.0-5
2:
Kernel option in the /etc/default/grub file =>
Code: 
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
3:
cat /etc/pve/nodes/fcprox01/qemu-server/109.conf =>
Code: 
hostpci0: 03:00.0
4:
a new file with only one row:
Code: 
:~# cat /etc/modprobe.d/kvm_iommu_map_guest.conf 
options kvm allow_unsafe_assigned_interrupts=1

what is the ",pt" in your kernel options for?
EDIT: I see it in the wiki, it's for AMD systems

I will have a look with the next version pve-qemu-kvm 1.0-10

regards,
maxprox
 
Last edited:
jram said:
maxprox,

Maybe it has been updated since your last post, but I just updated and it works for me. I use "hostpci0: XX:XX.X" in the .conf file (no msi or intx options). However, I found that passing through my PCIe device works, while PCI does not. Maybe it's due to:
(from http://wiki.xensource.com/xenwiki/VTdHowTo)
This may be specific to my motherboard, which probably uses a PCIe-to-PCI bridge.
Click to expand...

Hi,

Can you c/p your pveversion -v? I have been trying for days now to get this working but I can't. Maybe I can find a difference with your setup.

Also, on the Proxmox wiki page about pci passthrough (http://pve.proxmox.com/wiki/Pci_passthrough), I had to use different options, as what's provided there resulted in a kernel panic for me. Here is what works for my AMD Opteron 4130:
Code: 
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=on,pt"
Anyone else experience the same? If so, maybe we should update the wiki.
Click to expand...

This is correct, I also get the kernel panic if I use what is on the wiki. I already found out that amd_iommu is what you need, I didn't use the on with the iommu option but that doesn't seem to make a difference.

I know this should work because I previously had esXi on the same hardware and there I could easily pass-through my PCIe DVB-S2 card to my virtual machine. Now I'm trying to do the same in Proxmox but it fails no matter what I do. Currently I'm at the state where I reverted everything back to a "clean" install, other then the boot options. My motherboard is from ASRock with an 890FX chipset

Code: 
root@barabas:~# pveversion -v
pve-manager: 2.1-1 (pve-manager/2.1/f9b0f63a)
running kernel: 2.6.32-11-pve
proxmox-ve-2.6.32: 2.0-66
pve-kernel-2.6.32-11-pve: 2.6.32-66
pve-kernel-2.6.35-2-pve: 2.6.35-13
lvm2: 2.02.95-1pve2
clvm: 2.02.95-1pve2
corosync-pve: 1.4.3-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.8-3
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.7-2
pve-cluster: 1.0-26
qemu-server: 2.0-39
pve-firmware: 1.0-15
libpve-common-perl: 1.0-27
libpve-access-control: 1.0-21
libpve-storage-perl: 2.0-18
vncterm: 1.0-2
vzctl: 3.0.30-2pve5
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.0-9
ksm-control-daemon: 1.1-1

The VM does not start from the web interface. This is what I get when I try to start the VM from the commandline:

Code: 
root@barabas:~# /usr/bin/kvm -id 103 -chardev socket,id=monitor,path=/var/run/qemu-server/103.mon,server,nowait -mon chardev=monitor,mode=readline -vnc unix:/var/run/qemu-server/103.vnc,x509,password -pidfile /var/run/qemu-server/103.pid -daemonize -usbdevice tablet -device pci-assign,host=06:00.0,id=hostpci0 -name Streamer -smp sockets=2,cores=2 -nodefaults -boot menu=on -vga cirrus -k fr-be -drive if=none,id=drive-ide2,media=cdrom,aio=native -device ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200 -drive file=/var/lib/vz/images/103/vm-103-disk-1.qcow2,if=none,id=drive-ide0,aio=native -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0,id=ide0,bootindex=100 -m 4096 -netdev type=tap,id=net0,ifname=tap103i0,script=/var/lib/qemu-server/pve-bridge -device e1000,mac=62:F9:C4:84:6B:6F,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300 -netdev type=tap,id=net1,ifname=tap103i1,script=/var/lib/qemu-server/pve-bridge -device e1000,mac=CA:E8:4F:A6:59:C3,netdev=net1,bus=pci.0,addr=0x13,id=net1,bootindex=301 -cpuunits 1000
Failed to assign device "hostpci0" : Invalid argument
kvm: -device pci-assign,host=06:00.0,id=hostpci0: Device 'pci-assign' could not be initialized

Googling on "invalid argument" didn't bring me much luck. The following appears in my dmesg:

Code: 
pci 0000:06:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
pci 0000:06:00.0: restoring config space at offset 0x1 (was 0x100400, writing 0x100003)
assign device 6:0.0 failed
pci 0000:06:00.0: PCI INT A disabled

So it seems assigning the PCI card doesn't seem to work. Hints I have found are:

A lot of devices (including my USB controllers) are sharing the same IRQ:

Code: 
root@barabas:~# lspci -vv | grep IRQ
        Interrupt: pin A routed to IRQ 56
        Interrupt: pin A routed to IRQ 19
        Interrupt: pin A routed to IRQ 18
        Interrupt: pin B routed to IRQ 17
        Interrupt: pin A routed to IRQ 18
        Interrupt: pin B routed to IRQ 17
        Interrupt: pin C routed to IRQ 18
        Interrupt: pin A routed to IRQ 18
        Interrupt: pin B routed to IRQ 17
        Interrupt: pin A routed to IRQ 60
        Interrupt: pin B routed to IRQ 59
        Interrupt: pin A routed to IRQ 57
        Interrupt: pin A routed to IRQ 58
        Interrupt: pin A routed to IRQ 20
        Interrupt: pin B routed to IRQ 21
        Interrupt: pin A routed to IRQ 17

From dmesg :
Code: 
AMD-Vi: Enabling IOMMU at 0000:00:00.2 cap 0x40
AMD-Vi: Initialized for Passthrough Mode
AMD-Vi: Enabling IOMMU at 0000:00:00.2 cap 0x40

I also found this :
Code: 
Your BIOS doesn't leave a aperture memory hole
Please enable the IOMMU option in the BIOS setup
This costs you 64 MB of RAM
Mapping aperture over 65536 KB of RAM @ 20000000

Which I found strange. It is enabled in the BIOS and I know it works since I haven't changed it since esXi.
 
If I try pve-qemu-kvm_1.0-5_amd64.deb like max suggests I get this error:
Code: 
Failed to assign device "hostpci0" : Invalid argument
**
ERROR:qom/object.c:375:object_delete: assertion failed: (obj->ref == 0)

EDIT: BTW, I boot the normal .32 kernel, the .35 was also one I installed during my efforts, but don't use anymore
 
Last edited:
Mathy said:
If I try pve-qemu-kvm_1.0-5_amd64.deb like max suggests I get this error:
Code: 
Failed to assign device "hostpci0" : Invalid argument
**
ERROR:qom/object.c:375:object_delete: assertion failed: (obj->ref == 0)

EDIT: BTW, I boot the normal .32 kernel, the .35 was also one I installed during my efforts, but don't use anymore
Click to expand...

Hello,

Your ASRock Mainboard BIOS has the Option AMD IOMMU (like Intel: Vt-d) on?
Then try all my four steps at the same time with AMD and your Hardware Options - "It's worth a try"

maxprox
 
Hey maxprox,

The BIOS setting is on. All your 4 steps are the same, I have the hostpci0 statement, the boot line adaptations and the modprobe file in place. I tried with the 1.0-5, it didn't work. Only thing I can still think of is that I pulled it of a mirror server since I can't find it on the official and perhaps it's a different version. Can you perhaps upload your .deb somewhere so I can try that?

Thx,
Mathy
 
I'm a fan too, that's why I want it to work ;-) If I don't get a reply here from the dev's maybe I'll try an OpenVZ container and just passing through the device node, it's just that I have this VM already set up completely, and I don't want to the conversion work if it maybe is as simple as some config parameter.
 
Mathy,
Here is my pveversion -v:
Code: 
pve-manager: 2.1-1 (pve-manager/2.1/f9b0f63a)
running kernel: 2.6.32-11-pve
proxmox-ve-2.6.32: 2.0-66
pve-kernel-2.6.32-11-pve: 2.6.32-66
lvm2: 2.02.95-1pve2
clvm: 2.02.95-1pve2
corosync-pve: 1.4.3-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.8-3
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.7-2
pve-cluster: 1.0-26
qemu-server: 2.0-39
pve-firmware: 1.0-15
libpve-common-perl: 1.0-27
libpve-access-control: 1.0-21
libpve-storage-perl: 2.0-18
vncterm: 1.0-2
vzctl: 3.0.30-2pve5
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.0-9
ksm-control-daemon: 1.1-1
Just for testing, have you tried passthrough on the machine with a different PCIe device, like an extra NIC? Or, if you have extra servers laying around, maybe try that TV tuner card in a different proxmox server (with a different type of motherboard/processor). Could be worth seeing if you can get anything to pass through, to narrow the problem down to the tv tuner card.
 
I did try it before with a PCI card and failed but apparently the PCI card was faulty. Haven't tried anything else but that is a good tip, I'll see if I can find something.

Our versions are exactly the same BTW.
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom