[SOLVED] TBS6909 driver compilation error

[Helio Mendonça]

Hi

After setting up my Proxmox server I was trying now to install the driver for a PCIe DVB-S2 card (TBS6909 to be more precise).
The idea is to use this card in a VM using PCI passthrough but from what I saw I need to install the driver in the Proxmox Host machine first, right?
This was already tried with similar HW as you can check here: https://www.tbsdtv.com/forum/viewtopic.php?f=176&t=25087

Some info about my system:

CPU: Ryzen 7 1700
Motherboard: GIGABYTE AX370-GAMING K7
proxmox-ve: 5.4-1 (running kernel: 4.15.18-14-pve)

The installation process of the TBS6909 driver is described here: https://github.com/tbsdtv/linux_media/wiki and is composed by the following steps:

git clone https://github.com/tbsdtv/media_build.git
git clone --depth=1 https://github.com/tbsdtv/linux_media.git -b latest ./media
cd media_build
make dir DIR=../media
make allyesconfig
make -j4
sudo make install
sudo reboot

But when I do the 'make -j4' I get the error "You are building kernel with non-retpoline compiler, please update your compiler":

root@pve:~/media_build# make -j4
make -C /root/media_build/v4l
make[1]: Entering directory '/root/media_build/v4l'
make -C firmware prep
creating symbolic links...
make[2]: Entering directory '/root/media_build/v4l/firmware'
make[2]: Nothing to be done for 'prep'.
make[2]: Leaving directory '/root/media_build/v4l/firmware'
make -C firmware
make[2]: Entering directory '/root/media_build/v4l/firmware'
make[2]: Nothing to be done for 'default'.
make[2]: Leaving directory '/root/media_build/v4l/firmware'
Kernel build directory is /lib/modules/4.15.18-14-pve/build
make -C ../linux apply_patches
make[2]: Entering directory '/root/media_build/linux'
Syncing with dir ../../media
Patches for 4.15.18-14-pve already applied.
make[2]: Leaving directory '/root/media_build/linux'
make -C /lib/modules/4.15.18-14-pve/build SUBDIRS=/root/media_build/v4l  modules
make[2]: Entering directory '/usr/src/linux-headers-4.15.18-14-pve'
arch/x86/Makefile:245: *** You are building kernel with non-retpoline compiler, please update your compiler..  Stop.
make[2]: Leaving directory '/usr/src/linux-headers-4.15.18-14-pve'
Makefile:51: recipe for target 'default' failed
make[1]: *** [default] Error 2
make[1]: Leaving directory '/root/media_build/v4l'
Makefile:26: recipe for target 'all' failed
make: *** [all] Error 2

As far as I understand "retpoline" is a security measure against "branch target injection" (due to Spectre security vulnerability) implemented in newer kernels and to avoid this error I need to use a more recent gcc compiler, since the one that comes with pve is just version 6:

root@pve:~/media_build# gcc --version
gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516

How can I install gcc version 7 or even 8 in pve?
Have you any other alternative to install the driver?

Regards

PS - A workaround that I tried without success was to disable the "retpoline" security measures editing the following line in the /etc/default/grub file:

GRUB_CMDLINE_LINUX_DEFAULT="quiet spectre_v2=off"

But after a reboot the security measure seems to be always active:

root@pve:~/media_build# dmesg | grep 'retpoline'
[    0.047118] Spectre V2 : Mitigation: Full AMD retpoline

 

[lhorace]

Hello there,

Depending on how you plan to passthrough the device (e.g. with vfio, the one I'm mostly aware of), it can't be in use by the host. When the VM takes over the device, it's invisible to the host. You'll need, only then, compile and install the drivers in the guest. See https://pve.proxmox.com/wiki/Pci_passthrough for more info.

Cheers

 

[Helio Mendonça]

Thanks @lhorace for the tip!
In reality, I initially tried to do the compilation process inside the VM, but the error was the same!
I only tried then to do it in the host machine because in the post I saw the mentioned kernel was 4.15.18-9-pve which implies a compilation done in the host and not in the VM.

Regarding the gcc, Proxmox is based on Debian Stretch and it seems that the higher version of gcc possible is 6.
So the only solution seems to pass by disabling the "retpoline" security measures of kernel 4.15.18-14-pve (the PS part of my last post).
But, as you can check, my attempts on that were unsuccessful (during boot the "Spectre V2 : Mitigation: Full AMD retpoline" keeps appearing)!
Any help on this, please?

 

[lhorace]

By any chance, where did you get the drivers from? That error looks to me: You are building kernel, implying you are compiling a kernel from scratch? I could be wrong here. Closest thing I could find is this mailing list post here https://lkml.org/lkml/2018/12/8/92.

 

[Helio Mendonça]

I simply followed the WiKi page of the vendor of the TBS6909: https://github.com/tbsdtv/linux_media/wiki
That same procedure is described in the User Guide: https://www.tbsiptv.com/download/tbs6909/tbs6909_user_guide.pdf
Regards

 

[lhorace]

You could disable retopline in the guest, though I don't know what affect that would have if you later re-enable it. Nevertheless, you could have a chat with the folks upstream or try their https://github.com/tbsdtv/media_build/blob/master/install.sh script. BTW, it's pointless building the module on the host, since you can't use the module on any other kernel but that kernel version.

 

[Helio Mendonça]

OK! Let's see first the PCI Passthrough process.
When I try to add the PCI card to the hardware of the VM I get a warning:
The selected Device is not in a seperateIOMMU group, make sure this is intended.


So I tried to follow the steps in the link dedicated to this subject: https://pve.proxmox.com/wiki/Pci_passthrough

Since my CPU is a Ryzen 7 1700 I followed the AMD procedure:

Edit /etc/default/grub,
Changing: GRUB_CMDLINE_LINUX_DEFAULT="quiet"
To: GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on"

Doubts:

• This is done in the Proxmox host or in the VM?
• After these steps I need to execute "update-grub" as indicated in the case of Intel CPUs? Yes (see * below)

Then I added to /etc/modules the following lines:
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

But after this, when I execute the "dmesg | grep ecap" command, I do not get any reply!!!

What can I do now?

* Note that I am not a Linux expert, but I confirmed that only after executing the "update_grub" command, the changes done in the /etc/default/grub are implemented!

 

[Helio Mendonça]

Just to inform that I gave up of following the procedure described in the TBSDTV Github WiKi page and instead used the TBS Open Source Linux Driver Offline Package V190415, which follows these steps:

#wget http://www.tbsdtv.com/download/document/linux/media_build-2019-04-15.tar.bz2
#sudo rm -rf /lib/modules/`uname -r`/kernel/drivers/media/
#tar jxvf media_build-2019-04-15.tar.bz2
#cd media_build
#./install.sh
#reboot

And after doing that inside the pve host I can finally get the 8 DVB adapters of the TBS6909:

root@pve:~# dmesg | grep frontend
[   32.696074] TBSECP3 driver 0000:09:00.0: DVB: registering adapter 0 frontend 0 (TurboSight TBS 6909 DVB-S/S2 )...
[   32.819287] TBSECP3 driver 0000:09:00.0: DVB: registering adapter 1 frontend 0 (TurboSight TBS 6909 DVB-S/S2 )...
[   32.939498] TBSECP3 driver 0000:09:00.0: DVB: registering adapter 2 frontend 0 (TurboSight TBS 6909 DVB-S/S2 )...
[   33.060803] TBSECP3 driver 0000:09:00.0: DVB: registering adapter 3 frontend 0 (TurboSight TBS 6909 DVB-S/S2 )...
[   33.184883] TBSECP3 driver 0000:09:00.0: DVB: registering adapter 4 frontend 0 (TurboSight TBS 6909 DVB-S/S2 )...
[   33.308216] TBSECP3 driver 0000:09:00.0: DVB: registering adapter 5 frontend 0 (TurboSight TBS 6909 DVB-S/S2 )...
[   33.432215] TBSECP3 driver 0000:09:00.0: DVB: registering adapter 6 frontend 0 (TurboSight TBS 6909 DVB-S/S2 )...
[   33.557174] TBSECP3 driver 0000:09:00.0: DVB: registering adapter 7 frontend 0 (TurboSight TBS 6909 DVB-S/S2 )...

Now I need to:

• confirm that indeed I must do this inside the VM (and not in the host pve as @lhorace suggested)
• learn how to do a proper PCI passthrough of the TBS6909 to the VM

Any help on this please?

 

[lhorace]

What's tricky with IOMMU Groups and PCI Passthrough, often than not, you have to pass the entire group to the VM. Which means your Intel, Qualcom, AsMedia and TBS device. Otherwise it may not work properly. Though alternatively, they're two options you can try: Either get the ACS patch https://wiki.archlinux.org/index.ph...passing_the_IOMMU_groups_(ACS_override_patch) and pray that this works (Which requires a patched kernel) OR move the card to a different PCI slot so the TBS device can be in it's own IOMMU group.

On a positive note, great that you got the driver working on the host but as I said previously, once the device is passthrough to the guest VM, the host kernel can no longer use it.

Cheers

 

[Helio Mendonça]

Since the compilation problem is solved I will create now a new thread related with the use of the TBS6909 in a VM/CT running on Proxmox.