DVB PCIe card in VM

G

gusto

Well-Known Member
Feb 10, 2018
85
2
48
25
Proxmox 7.1
HW:
MotherBoard (IOMMU enabled)
CPU
TBS6909 DVB-S2 8 Tuner PCIe Card
TBS 6925 PCI-E DVB-S2 TV Tuner Card

Code: 
lspci
Code: 
07:00.0 Multimedia controller: TBS Technologies DVB Tuner PCIe Card
08:00.0 Multimedia controller: Philips Semiconductors SAA7160 (rev 03)
I want to use DVB cards in VM (debian 11 netinst). I followed the instructions.

Code: 
nano /etc/default/grub
I edited the line at
Code: 
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on"
Code: 
update-grub
reebot
Code: 
dmesg | grep -e DMAR -e IOMMU
the output was this

Code: 
[    0.530315] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[    0.534304] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[    0.534822] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).

Code: 
nano /etc/modules
I ad this lines

Code: 
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

Code: 
dmesg | grep 'remapping'
[    0.534312] AMD-Vi: Interrupt remapping enabled

Code: 
find /sys/kernel/iommu_groups/ -type l
/sys/kernel/iommu_groups/17/devices/0000:0b:00.0
/sys/kernel/iommu_groups/7/devices/0000:00:07.1
/sys/kernel/iommu_groups/15/devices/0000:0a:00.2
/sys/kernel/iommu_groups/5/devices/0000:00:04.0
/sys/kernel/iommu_groups/13/devices/0000:09:00.0
/sys/kernel/iommu_groups/13/devices/0000:09:00.1
/sys/kernel/iommu_groups/3/devices/0000:00:03.0
/sys/kernel/iommu_groups/11/devices/0000:00:18.3
/sys/kernel/iommu_groups/11/devices/0000:00:18.1
/sys/kernel/iommu_groups/11/devices/0000:00:18.6
/sys/kernel/iommu_groups/11/devices/0000:00:18.4
/sys/kernel/iommu_groups/11/devices/0000:00:18.2
/sys/kernel/iommu_groups/11/devices/0000:00:18.0
/sys/kernel/iommu_groups/11/devices/0000:00:18.7
/sys/kernel/iommu_groups/11/devices/0000:00:18.5
/sys/kernel/iommu_groups/1/devices/0000:00:01.3
/sys/kernel/iommu_groups/18/devices/0000:0b:00.2
/sys/kernel/iommu_groups/8/devices/0000:00:08.0
/sys/kernel/iommu_groups/16/devices/0000:0a:00.3
/sys/kernel/iommu_groups/6/devices/0000:00:07.0
/sys/kernel/iommu_groups/14/devices/0000:0a:00.0
/sys/kernel/iommu_groups/4/devices/0000:00:03.1
/sys/kernel/iommu_groups/12/devices/0000:03:00.0
/sys/kernel/iommu_groups/12/devices/0000:02:07.0
/sys/kernel/iommu_groups/12/devices/0000:02:00.0
/sys/kernel/iommu_groups/12/devices/0000:08:00.0
/sys/kernel/iommu_groups/12/devices/0000:01:00.2
/sys/kernel/iommu_groups/12/devices/0000:02:03.0
/sys/kernel/iommu_groups/12/devices/0000:01:00.0
/sys/kernel/iommu_groups/12/devices/0000:02:06.0
/sys/kernel/iommu_groups/12/devices/0000:07:00.0
/sys/kernel/iommu_groups/12/devices/0000:02:02.0
/sys/kernel/iommu_groups/12/devices/0000:01:00.1
/sys/kernel/iommu_groups/12/devices/0000:04:00.0
/sys/kernel/iommu_groups/12/devices/0000:02:04.0
/sys/kernel/iommu_groups/2/devices/0000:00:02.0
/sys/kernel/iommu_groups/10/devices/0000:00:14.3
/sys/kernel/iommu_groups/10/devices/0000:00:14.0
/sys/kernel/iommu_groups/0/devices/0000:00:01.0
/sys/kernel/iommu_groups/19/devices/0000:0b:00.3
/sys/kernel/iommu_groups/9/devices/0000:00:08.1

Now I will log in to the VM as root and display the device pci (lspci).
lspci-kvm.png
I don't see DVB devices there (07.00.0, 08.00.0).
When I add a device via the GUI and reboot the VM, proxmox freezes.
device.png
VM config
Code: 
boot: order=scsi0;ide2;net0
cores: 2
ide2: local:iso/debian11.iso,media=cdrom
machine: q35
memory: 4096
meta: creation-qemu=6.1.0,ctime=1646137846
name: debian-test
net0: virtio=02:86:56:3C:60:1E,bridge=vmbr0
numa: 0
ostype: l26
scsi0: local-lvm:vm-100-disk-0,size=16G
scsihw: virtio-scsi-pci
smbios1: uuid=93e555d1-51ae-4745-b3b3-a2b9f3d55581
sockets: 1
vmgenid: 56500fbf-2481-49c6-a391-9e0f83589b7a
Where am I making a mistake?
 
Last edited:
You need to add the 07:00.0 and 08:00.0 devices to the VM, which will show a hostpci0 and hostpci1 line in the VM configuration file.

However, both devices are part of IOMMU group 12, which also contains several other devices like the SATA, a few USB and a network controller. When you start the VM, the Proxmox host will lose those devices because you cannot share devices from the same IOMMU group between VMs or between the Proxmox host and a VM. That's why your Proxmox freezes: it cannot read/write to disk and cannot access the network anymore!

The IOMMU groups are determined by the motherboard (and BIOS version). On the Ryzen platform, unless you have a X570, you can only passthrough one or two of the x16 PCIe slots and the M.2 slot closest to the CPU and the audio and USB devices from the CPU. Try putting one of those tuners in the middle x16 PCIe slot and see if it is alone in a IOMMU group (bridges don't count).

If you don't care about VM isolation and security, you can try the pcie_acs_override that is included in the Proxmox kernel.

EDIT: You don't need amd_iommu=on, which is actually invalid because it is on by default. dmesg | grep -e DMAR -e IOMMU is only useful for Intel systens; check if there are multiple IOMMU groups to verify that IOMMU is enabled instead.
 
Last edited:
You need to add the 07:00.0 and 08:00.0 devices to the VM, which will show a hostpci0 and hostpci1 line in the VM configuration file.
Click to expand...
I added devices via GUI. My config in the previous post does not contain these lines. Then when proxmox froze I removed the rows.

The IOMMU groups are determined by the motherboard (and BIOS version). On the Ryzen platform, unless you have a X570, you can only passthrough one or two of the x16 PCIe slots and the M.2 slot closest to the CPU and the audio and USB devices from the CPU. Try putting one of those tuners in the middle x16 PCIe slot and see if it is alone in a IOMMU group (bridges don't count).
Click to expand...
Does this mean that IOMMU groups are given BIOS? Can't I change it? DVB devices are designed for PCIe 1x. I tried to put the device in another slot, but nothing has changed (only from 07:00.0 to 05.00.0

EDIT: You don't need amd_iommu=on, which is actually invalid because it is on by default. dmesg | grep -e DMAR -e IOMMU is only useful for Intel systens; check if there are multiple IOMMU groups to verify that IOMMU is enabled instead.
Click to expand...
I found a post where you advised.
Code: 
for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done;
Code: 
IOMMU group 0 00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU group 10 00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 59)
IOMMU group 10 00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
IOMMU group 11 00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1022:1460]
IOMMU group 11 00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1022:1461]
IOMMU group 11 00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1022:1462]
IOMMU group 11 00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1022:1463]
IOMMU group 11 00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1022:1464]
IOMMU group 11 00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1022:1465]
IOMMU group 11 00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6 [1022:1466]
IOMMU group 11 00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 [1022:1467]
IOMMU group 12 01:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] X370 Series Chipset USB 3.1 xHCI Controller [1022:43b9] (rev 02)
IOMMU group 12 01:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] X370 Series Chipset SATA Controller [1022:43b5] (rev 02)
IOMMU group 12 01:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] X370 Series Chipset PCIe Upstream Port [1022:43b0] (rev 02)
IOMMU group 12 02:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU group 12 02:02.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU group 12 02:03.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU group 12 02:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU group 12 02:06.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU group 12 02:07.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU group 12 03:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1143 USB 3.1 Host Controller [1b21:1343]
IOMMU group 12 04:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit Network Connection [8086:1539] (rev 03)
IOMMU group 12 05:00.0 Multimedia controller [0480]: TBS Technologies DVB Tuner PCIe Card [544d:6178]
IOMMU group 12 08:00.0 Multimedia controller [0480]: Philips Semiconductors SAA7160 [1131:7160] (rev 03)
IOMMU group 13 09:00.0 VGA compatible controller [0300]: NVIDIA Corporation GT218 [GeForce 210] [10de:0a65] (rev a2)
IOMMU group 13 09:00.1 Audio device [0403]: NVIDIA Corporation High Definition Audio Controller [10de:0be3] (rev a1)
IOMMU group 14 0a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Raven/Raven2 PCIe Dummy Function [1022:145a]
IOMMU group 15 0a:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor [1022:1456]
IOMMU group 16 0a:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller [1022:145c]
IOMMU group 17 0b:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Zeppelin/Renoir PCIe Dummy Function [1022:1455]
IOMMU group 18 0b:00.2 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
IOMMU group 19 0b:00.3 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) HD Audio Controller [1022:1457]
IOMMU group 1 00:01.3 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge [1022:1453]
IOMMU group 2 00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU group 3 00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU group 4 00:03.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge [1022:1453]
IOMMU group 5 00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU group 6 00:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU group 7 00:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B [1022:1454]
IOMMU group 8 00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU group 9 00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B [1022:1454]
 
You cannot change the IOMMU groups, you can only break the groups with the pcie_acs_override I already mentioned.
Trying any other PCIe slot except for the first two PCIe x16 (one of which is only wired for x8 and the other will only work at x8 when both are use) or the first M.2 slot does not change anything. It is always the same for Ryzen (except X570) and sometime less slots work because of BIOS issues.
You can put a x1 PCIe device in a x16 slot and it will work fine. You can do the other way around even and it will work, but slower.
 
I placed DVB cards in PCIe 16x and now there are separate IOMMU groups.
Big thanks avw
 
You must log in or register to reply here.
Top Bottom