PCIE Google Coral Install Instructions

FoodFighter said:
Oh, thank you!!!
I searched high and low for a working repo/package.
But now you have to be careful that you don't accidentally install the faulty Google package again with
apt update && apt upgrade -y
and mess everything up.

To prevent this, you can exclude the package from the apt-upgrade.
apt-mark hold gasket-dkms

However, this solution is not ideal. Is there a better solution?
(Perhaps one should fork the GitHub repository and increase the version number. I will test this.)


[EDIT]:
found this: https://github.com/markfrancisonly/frigate_debian_scripts
there is already a PullRequest #50 in the official repo, that works great!
someone made a beautifil script for installation and rebuild and so on.

I tried to adapt that script for a proxmox-Version (not fully tested):
https://github.com/Bodengriller/frigate_debian_scripts/tree/Proxmox-Edition
feel free to test.

basicly it works this way for a fresh installation:

Code: 
### Install the TPU driver and library ###
1. Installing prerequisites...
    apt update && apt upgrade -y
    apt install -y git devscripts dh-dkms dkms proxmox-headers-$(uname -r)


2. Adding Coral Edge TPU repository...
    curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | gpg --dearmor -o /etc/apt/keyrings/coral-edgetpu.gpg
    echo "deb [signed-by=/etc/apt/keyrings/coral-edgetpu.gpg] https://packages.cloud.google.com/apt coral-edgetpu-stable main" | tee /etc/apt/sources.list.d/coral-edgetpu.list
    apt update
    apt install -y libedgetpu1-std


3. Installing driver from source...
# Download Git-repo and switch to PullRequest #50 [https://github.com/google/gasket-driver/pull/50]
    mkdir -p /home/coral-build
    git clone https://github.com/google/gasket-driver.git /home/coral-build/gasket-driver
    cd /home/coral-build/gasket-driver
    git fetch origin pull/50/head:pr-50
    git checkout pr-50
    debuild -us -uc -tc -b
    dpkg -i ../gasket-dkms_*_all.deb
    apt update
    if no error => reboot !


# Verify the drivers are functioning properly
    lspci -nn | grep 089a
    ls /dev/apex_0


# For unprivileged LXC containers
    nano /etc/udev/rules.d/99-chmod777.rules
        SUBSYSTEM=="apex", MODE="0777", GROUP="apex"
        KERNEL=="renderD128", MODE="0777"
    udevadm control --reload-rules && udevadm trigger
Click to expand...
Unfortunately I'm still getting the ls: cannot access '/dev/apex_0' : No such file or directory after following your script & instructions
 
smithab96 said:
Unfortunately I'm still getting the ls: cannot access '/dev/apex_0' : No such file or directory after following your script & instructions
Click to expand...
Can you please show me the complete installation log? It seems that a dependency is missing or something.
(Did you reboot after a successful --install?)
 
smithab96 said:
Unfortunately I'm still getting the ls: cannot access '/dev/apex_0' : No such file or directory after following your script & instructions
Click to expand...
I think I've found the problem.
It's not enough to just install proxmox-headers-$(uname -r); there are other dependencies that are missing.
I added the pve-headers package and tested everything in a fresh Proxmox VM. It's working for me now.
[pve-manager/9.0.5/9c5600b249dbfd2f (running kernel: 6.14.8-2-pve)]

Since there are sometimes problems with the headers not being updated to the current kernel version, I left proxmox-headers-$(uname -r) in there as well.

Please test it again with the corrected script.
 
Last edited:
  • Like
Reactions: walber
FoodFighter said:
I think I've found the problem.
It's not enough to just install proxmox-headers-$(uname -r); there are other dependencies that are missing.
I added the pve-headers package and tested everything in a fresh Proxmox VM. It's working for me now.
[pve-manager/9.0.5/9c5600b249dbfd2f (running kernel: 6.14.8-2-pve)]

Since there are sometimes problems with the headers not being updated to the current kernel version, I left proxmox-headers-$(uname -r) in there as well.

Please test it again with the corrected script.
Click to expand...
@FoodFighter - Thx, this script worked on my recently upgraded system, I'm currently on PVE 9.0.5. I can finally see /dev/apex_0 and /dev/apex_1
 
FoodFighter said:
I think I've found the problem.
It's not enough to just install proxmox-headers-$(uname -r); there are other dependencies that are missing.
I added the pve-headers package and tested everything in a fresh Proxmox VM. It's working for me now.
[pve-manager/9.0.5/9c5600b249dbfd2f (running kernel: 6.14.8-2-pve)]

Since there are sometimes problems with the headers not being updated to the current kernel version, I left proxmox-headers-$(uname -r) in there as well.

Please test it again with the corrected script.
Click to expand...

I ran the latest script on 6.14.11-1-pve and the system cannot find apex devices

root@pve4:~# ./install_coral_tpu.sh --status
Coral TPU PCIe hardware detected:
01:00.0 System peripheral [0880]: Global Unichip Corp. Coral Edge TPU [1ac1:089a]

Apex devices not found in /dev.
DKMS Gasket module: gasket/1.0, 6.12.41+deb13-amd64, x86_64: installed
gasket/1.0, 6.12.43+deb13-amd64, x86_64: installed
gasket/1.0, 6.14.11-1-pve, x86_64: installed
gasket/1.0, 6.14.8-2-pve, x86_64: installed
Apex module version: 1.2
libedgetpu1-std version: 16.0

Non-root access:
Udev rule for non-root access: not present.
Apex group for non-root access: not present.

Unprivileged LXC access:
Udev rule for unprivileged LXC access: not present.
Driver installation validation failed.

Usage: bash ./install_coral_tpu.sh [OPTION]
Options:
--status : Display installation status.
--install : Install the TPU driver and library.
--uninstall : Uninstall the TPU driver and library.
--reinstall : Reinstall the TPU driver from source.
--rebuild : Rebuild the TPU driver for the current kernel.
--setup-non-root [username] : Set up non-root access for the specified user.
--setup-unprivileged : Set up chmod777 for unprivileged LXC containers
Click to expand...
 
Last edited:
Also:

root@pve4:/home/coral-build/gasket-driver# dmesg |grep gasket
[ 3.669543] gasket: module verification failed: signature and/or required key missing - tainting kernel
 
root@pve4:~# ./install_coral_tpu.sh --install
Installing prerequisites...
Hit:1 http://deb.debian.org/debian trixie InRelease
Get:2 http://security.debian.org/debian-security trixie-security InRelease [43.4 kB]
Hit:3 http://deb.debian.org/debian trixie-updates InRelease
Hit:4 https://packages.cloud.google.com/apt coral-edgetpu-stable InRelease
Hit:5 http://download.proxmox.com/debian/pve trixie InRelease
Fetched 43.4 kB in 1s (33.6 kB/s)
All packages are up to date.
git is already the newest version (1:2.47.3-0+deb13u1).
devscripts is already the newest version (2.25.15+deb13u1).
dh-dkms is already the newest version (3.2.0-1).
dkms is already the newest version (3.2.0-1).
pve-headers is already the newest version (9.0.0).
proxmox-headers-6.14.11-1-pve is already the newest version (6.14.11-1).
Summary:
Upgrading: 0, Installing: 0, Removing: 0, Not Upgrading: 0
Adding Coral Edge TPU repository...
File '/etc/apt/keyrings/coral-edgetpu.gpg' exists. Overwrite? (y/N) y
deb [signed-by=/etc/apt/keyrings/coral-edgetpu.gpg] https://packages.cloud.google.com/apt coral-edgetpu-stable main
Hit:1 http://security.debian.org/debian-security trixie-security InRelease
Hit:2 http://deb.debian.org/debian trixie InRelease
Hit:3 http://deb.debian.org/debian trixie-updates InRelease
Hit:4 https://packages.cloud.google.com/apt coral-edgetpu-stable InRelease
Hit:5 http://download.proxmox.com/debian/pve trixie InRelease
All packages are up to date.
libedgetpu1-std is already the newest version (16.0).
Summary:
Upgrading: 0, Installing: 0, Removing: 0, Not Upgrading: 0
Installing driver from source...
Cloning into '/tmp/coral-build/gasket-driver'...
remote: Enumerating objects: 850, done.
remote: Counting objects: 100% (230/230), done.
remote: Compressing objects: 100% (21/21), done.
remote: Total 850 (delta 214), reused 209 (delta 209), pack-reused 620 (from 1)
Receiving objects: 100% (850/850), 254.81 KiB | 2.47 MiB/s, done.
Resolving deltas: 100% (589/589), done.
remote: Enumerating objects: 26, done.
remote: Counting objects: 100% (25/25), done.
remote: Compressing objects: 100% (11/11), done.
remote: Total 26 (delta 14), reused 22 (delta 14), pack-reused 1 (from 1)
Unpacking objects: 100% (26/26), 28.11 KiB | 899.00 KiB/s, done.
From https://github.com/google/gasket-driver
* [new ref] refs/pull/50/head -> pr-50
Switched to branch 'pr-50'
dpkg-buildpackage -us -uc -ui -tc -b
dpkg-buildpackage: info: source package gasket-dkms
dpkg-buildpackage: info: source version 1.0-18.4
dpkg-buildpackage: info: source distribution noble
dpkg-buildpackage: info: source changed by Nicola Ferralis <feranick@hotmail.com>
dpkg-source --before-build .
dpkg-buildpackage: info: host architecture amd64
debian/rules clean
dh clean --with dkms
dh_clean
debian/rules binary
dh binary --with dkms
dh_update_autotools_config
dh_autoreconf
create-stamp debian/debhelper-build-stamp
dh_prep
debian/rules override_dh_install
make[1]: Entering directory '/tmp/coral-build/gasket-driver'
dh_install src/* usr/src/gasket-1.0/
make[1]: Leaving directory '/tmp/coral-build/gasket-driver'
dh_installdocs
dh_installchangelogs
debian/rules override_dh_dkms
make[1]: Entering directory '/tmp/coral-build/gasket-driver'
dh_dkms -V 1.0
make[1]: Leaving directory '/tmp/coral-build/gasket-driver'
dh_installudev
dh_perl
dh_link
dh_strip_nondeterminism
dh_compress
dh_fixperms
dh_missing
dh_installdeb
dh_gencontrol
dh_md5sums
dh_builddeb
dpkg-deb: building package 'gasket-dkms' in '../gasket-dkms_1.0-18.4_all.deb'.
dpkg-genbuildinfo --build=binary -O../gasket-dkms_1.0-18.4_amd64.buildinfo
dpkg-genchanges --build=binary -O../gasket-dkms_1.0-18.4_amd64.changes
dpkg-genchanges: info: binary-only upload (no source code included)
debian/rules clean
dh clean --with dkms
dh_clean
dpkg-source --after-build .
dpkg-buildpackage: info: binary-only upload (no source included)
Now running lintian gasket-dkms_1.0-18.4_amd64.changes ...
running with root privileges is not recommended!
E: gasket-dkms changes: bad-distribution-in-changes-file noble
W: gasket-dkms: old-fsf-address-in-copyright-file
Finished running lintian.
(Reading database ... 159432 files and directories currently installed.)
Preparing to unpack .../gasket-dkms_1.0-18.4_all.deb ...
Module gasket/1.0 for kernel 6.14.8-2-pve (x86_64):
Before uninstall, this module version was ACTIVE on this kernel.
Deleting /lib/modules/6.14.8-2-pve/updates/dkms/gasket.ko
Deleting /lib/modules/6.14.8-2-pve/updates/dkms/apex.ko
Running depmod.... done.

Module gasket/1.0 for kernel 6.14.11-1-pve (x86_64):
Before uninstall, this module version was ACTIVE on this kernel.
Deleting /lib/modules/6.14.11-1-pve/updates/dkms/gasket.ko
Deleting /lib/modules/6.14.11-1-pve/updates/dkms/apex.ko
Running depmod.... done.

Deleting module gasket/1.0 completely from the DKMS tree.
Unpacking gasket-dkms (1.0-18.4) over (1.0-18.4) ...
Setting up gasket-dkms (1.0-18.4) ...
Loading new gasket/1.0 DKMS files...
Building for 6.12.43+deb13-amd64, 6.14.8-2-pve and 6.14.11-1-pve

Building initial module gasket/1.0 for 6.12.43+deb13-amd64
Sign command: /lib/modules/6.12.43+deb13-amd64/build/scripts/sign-file
Signing key: /var/lib/dkms/mok.key
Public certificate (MOK): /var/lib/dkms/mok.pub

Building module(s)... done.
Signing module /var/lib/dkms/gasket/1.0/build/gasket.ko
Signing module /var/lib/dkms/gasket/1.0/build/apex.ko
Installing /lib/modules/6.12.43+deb13-amd64/updates/dkms/gasket.ko.xz
Installing /lib/modules/6.12.43+deb13-amd64/updates/dkms/apex.ko.xz
Running depmod.... done.

Building initial module gasket/1.0 for 6.14.8-2-pve
Sign command: /lib/modules/6.14.8-2-pve/build/scripts/sign-file
Signing key: /var/lib/dkms/mok.key
Public certificate (MOK): /var/lib/dkms/mok.pub

Building module(s)... done.
Signing module /var/lib/dkms/gasket/1.0/build/gasket.ko
Signing module /var/lib/dkms/gasket/1.0/build/apex.ko
Installing /lib/modules/6.14.8-2-pve/updates/dkms/gasket.ko
Installing /lib/modules/6.14.8-2-pve/updates/dkms/apex.ko
Running depmod... done.

Building initial module gasket/1.0 for 6.14.11-1-pve
Sign command: /lib/modules/6.14.11-1-pve/build/scripts/sign-file
Signing key: /var/lib/dkms/mok.key
Public certificate (MOK): /var/lib/dkms/mok.pub

Building module(s)... done.
Signing module /var/lib/dkms/gasket/1.0/build/gasket.ko
Signing module /var/lib/dkms/gasket/1.0/build/apex.ko
Installing /lib/modules/6.14.11-1-pve/updates/dkms/gasket.ko
Installing /lib/modules/6.14.11-1-pve/updates/dkms/apex.ko
Running depmod... done.
Installation complete.
Reboot is required to apply changes. Do you want to reboot now? (y/n): n
Please reboot manually later to apply changes.
Click to expand...

After reboot

root@pve4:~# lsmod|grep apex
apex 28672 0
gasket 126976 1 apex

root@pve4:~# ls /dev/apex_0
ls: cannot access '/dev/apex_0': No such file or directory
Click to expand...

One interesting tidbit is that everything worked in PVE 8 and all these problems emerged after upgrading to PVE 9
 
Last edited:
unmesh59 said:
After reboot



One interesting tidbit is that everything worked in PVE 8 and all these problems emerged after upgrading to PVE 9
Click to expand...
Same problem, CORAL doesn't work on PVE 9.

Code: 
root@pve-s1:~# ./install_coral_tpu.sh --install
Installing prerequisites...
Hit:1 http://ftp.ru.debian.org/debian trixie InRelease
Hit:2 http://security.debian.org trixie-security InRelease
Hit:3 http://deb.debian.org/debian trixie InRelease
Get:4 http://ftp.ru.debian.org/debian trixie-updates InRelease [47.3 kB]
Hit:5 http://download.proxmox.com/debian/ceph-squid trixie InRelease
Hit:6 http://download.proxmox.com/debian/pve trixie InRelease
Fetched 47.3 kB in 1s (56.1 kB/s)
All packages are up to date.
git is already the newest version (1:2.47.3-0+deb13u1).
devscripts is already the newest version (2.25.15+deb13u1).
dh-dkms is already the newest version (3.2.0-1).
dkms is already the newest version (3.2.0-1).
pve-headers is already the newest version (9.0.0).
proxmox-headers-6.14.11-2-pve is already the newest version (6.14.11-2).
The following packages were automatically installed and are no longer required:
  linux-headers-amd64  proxmox-kernel-6.14.8-2-pve-signed
Use 'apt autoremove' to remove them.

Summary:
  Upgrading: 0, Installing: 0, Removing: 0, Not Upgrading: 0
Adding Coral Edge TPU repository...
deb [signed-by=/etc/apt/keyrings/coral-edgetpu.gpg] https://packages.cloud.google.com/apt coral-edgetpu-stable main
Hit:1 http://download.proxmox.com/debian/ceph-squid trixie InRelease
Hit:2 http://security.debian.org trixie-security InRelease
Hit:3 http://deb.debian.org/debian trixie InRelease
Hit:4 http://download.proxmox.com/debian/pve trixie InRelease
Hit:5 http://ftp.ru.debian.org/debian trixie InRelease
Hit:6 http://ftp.ru.debian.org/debian trixie-updates InRelease
Get:7 https://packages.cloud.google.com/apt coral-edgetpu-stable InRelease [1,423 B]
Get:8 https://packages.cloud.google.com/apt coral-edgetpu-stable/main all Packages [1,865 B]
Get:9 https://packages.cloud.google.com/apt coral-edgetpu-stable/main amd64 Packages [6,888 B]
Fetched 10.2 kB in 1s (8,658 B/s)
All packages are up to date.
The following packages were automatically installed and are no longer required:
  linux-headers-amd64  proxmox-kernel-6.14.8-2-pve-signed
Use 'apt autoremove' to remove them.

Installing:
  libedgetpu1-std

Summary:
  Upgrading: 0, Installing: 1, Removing: 0, Not Upgrading: 0
  Download size: 0 B / 387 kB
  Space needed: 1,206 kB / 82.7 GB available

Selecting previously unselected package libedgetpu1-std:amd64.
(Reading database ... 213993 files and directories currently installed.)
Preparing to unpack .../libedgetpu1-std_16.0_amd64.deb ...
Unpacking libedgetpu1-std:amd64 (16.0) ...
Setting up libedgetpu1-std:amd64 (16.0) ...
Processing triggers for libc-bin (2.41-12) ...
Installing driver from source...
Cloning into '/tmp/coral-build/gasket-driver'...
remote: Enumerating objects: 850, done.
remote: Counting objects: 100% (230/230), done.
remote: Compressing objects: 100% (24/24), done.
remote: Total 850 (delta 214), reused 206 (delta 206), pack-reused 620 (from 1)
Receiving objects: 100% (850/850), 254.86 KiB | 578.00 KiB/s, done.
Resolving deltas: 100% (589/589), done.
remote: Enumerating objects: 26, done.
remote: Counting objects: 100% (25/25), done.
remote: Compressing objects: 100% (11/11), done.
remote: Total 26 (delta 14), reused 22 (delta 14), pack-reused 1 (from 1)
Unpacking objects: 100% (26/26), 28.11 KiB | 293.00 KiB/s, done.
From https://github.com/google/gasket-driver
 * [new ref]         refs/pull/50/head -> pr-50
Switched to branch 'pr-50'
 dpkg-buildpackage -us -uc -ui -tc -b
dpkg-buildpackage: info: source package gasket-dkms
dpkg-buildpackage: info: source version 1.0-18.4
dpkg-buildpackage: info: source distribution noble
dpkg-buildpackage: info: source changed by Nicola Ferralis <feranick@hotmail.com>
 dpkg-source --before-build .
dpkg-buildpackage: info: host architecture amd64
 debian/rules clean
dh clean --with dkms
   dh_clean
 debian/rules binary
dh binary --with dkms
   dh_update_autotools_config
   dh_autoreconf
   create-stamp debian/debhelper-build-stamp
   dh_prep
   debian/rules override_dh_install
make[1]: Entering directory '/tmp/coral-build/gasket-driver'
dh_install src/* usr/src/gasket-1.0/
make[1]: Leaving directory '/tmp/coral-build/gasket-driver'
   dh_installdocs
   dh_installchangelogs
   debian/rules override_dh_dkms
make[1]: Entering directory '/tmp/coral-build/gasket-driver'
dh_dkms -V 1.0
make[1]: Leaving directory '/tmp/coral-build/gasket-driver'
   dh_installudev
   dh_perl
   dh_link
   dh_strip_nondeterminism
   dh_compress
   dh_fixperms
   dh_missing
   dh_installdeb
   dh_gencontrol
   dh_md5sums
   dh_builddeb
dpkg-deb: building package 'gasket-dkms' in '../gasket-dkms_1.0-18.4_all.deb'.
 dpkg-genbuildinfo --build=binary -O../gasket-dkms_1.0-18.4_amd64.buildinfo
 dpkg-genchanges --build=binary -O../gasket-dkms_1.0-18.4_amd64.changes
dpkg-genchanges: info: binary-only upload (no source code included)
 debian/rules clean
dh clean --with dkms
   dh_clean
 dpkg-source --after-build .
dpkg-buildpackage: info: binary-only upload (no source included)
Now running lintian gasket-dkms_1.0-18.4_amd64.changes ...
running with root privileges is not recommended!
E: gasket-dkms changes: bad-distribution-in-changes-file noble
W: gasket-dkms: old-fsf-address-in-copyright-file
Finished running lintian.
Selecting previously unselected package gasket-dkms.
(Reading database ... 214000 files and directories currently installed.)
Preparing to unpack .../gasket-dkms_1.0-18.4_all.deb ...
Unpacking gasket-dkms (1.0-18.4) ...
Setting up gasket-dkms (1.0-18.4) ...
Loading new gasket/1.0 DKMS files...
Building for 6.12.43+deb13-amd64, 6.14.11-1-pve and 6.14.11-2-pve

Building initial module gasket/1.0 for 6.12.43+deb13-amd64
Sign command: /lib/modules/6.12.43+deb13-amd64/build/scripts/sign-file
Signing key: /var/lib/dkms/mok.key
Public certificate (MOK): /var/lib/dkms/mok.pub

Building module(s).... done.
Signing module /var/lib/dkms/gasket/1.0/build/gasket.ko
Signing module /var/lib/dkms/gasket/1.0/build/apex.ko
Installing /lib/modules/6.12.43+deb13-amd64/updates/dkms/gasket.ko.xz
Installing /lib/modules/6.12.43+deb13-amd64/updates/dkms/apex.ko.xz
Running depmod..... done.

Building initial module gasket/1.0 for 6.14.11-1-pve
Sign command: /lib/modules/6.14.11-1-pve/build/scripts/sign-file
Signing key: /var/lib/dkms/mok.key
Public certificate (MOK): /var/lib/dkms/mok.pub

Building module(s).... done.
Signing module /var/lib/dkms/gasket/1.0/build/gasket.ko
Signing module /var/lib/dkms/gasket/1.0/build/apex.ko
Installing /lib/modules/6.14.11-1-pve/updates/dkms/gasket.ko
Installing /lib/modules/6.14.11-1-pve/updates/dkms/apex.ko
Running depmod.... done.

Building initial module gasket/1.0 for 6.14.11-2-pve
Sign command: /lib/modules/6.14.11-2-pve/build/scripts/sign-file
Signing key: /var/lib/dkms/mok.key
Public certificate (MOK): /var/lib/dkms/mok.pub

Building module(s).... done.
Signing module /var/lib/dkms/gasket/1.0/build/gasket.ko
Signing module /var/lib/dkms/gasket/1.0/build/apex.ko
Installing /lib/modules/6.14.11-2-pve/updates/dkms/gasket.ko
Installing /lib/modules/6.14.11-2-pve/updates/dkms/apex.ko
Running depmod.... done.
Installation complete.
Reboot is required to apply changes. Do you want to reboot now? (y/n): y

root@pve-s1:~# ./install_coral_tpu.sh --status

Coral TPU PCIe hardware not detected (lspci | grep 1ac1:089a).
Apex devices not found in /dev.
DKMS Gasket module: gasket/1.0, 6.12.43+deb13-amd64, x86_64: installed
gasket/1.0, 6.14.11-1-pve, x86_64: installed
gasket/1.0, 6.14.11-2-pve, x86_64: installed
Apex module not loaded.
libedgetpu1-std version: 16.0

Non-root access:
Udev rule for non-root access: not present.
Apex group for non-root access: not present.

Unprivileged LXC access:
Udev rule for unprivileged LXC access: not present.
Driver installation validation failed.
 
Last edited:
ShaMAD said:
Same problem, CORAL doesn't work on PVE 9.
Click to expand...

maybe a problem with gasket/1.0, 6.14.11?
on my site everything works fine:
PVE 9.0.5

Code: 
root@proxmox:/home#  ./install_coral_tpu.sh --status
Coral TPU PCIe hardware detected:
01:00.0 System peripheral [0880]: Global Unichip Corp. Coral Edge TPU [1ac1:089a]

Apex devices installed:
/dev/apex_0

DKMS Gasket module: gasket/1.0, 6.12.41+deb13-amd64, x86_64: installed
gasket/1.0, 6.14.8-2-pve, x86_64: installed
Apex module version: 1.2
libedgetpu1-std version: 16.0

Non-root access:
Udev rule for non-root access: not present.
Apex group for non-root access: not present.

Unprivileged LXC access:
Udev rule for unprivileged LXC access: present.
 
Since I have 3 versions of gasket but am running 6.14.11 kernel, is it safe to boot to kernel 6.14.8 through grub?

root@pve4:~# dkms status
gasket/1.0, 6.12.43+deb13-amd64, x86_64: installed
gasket/1.0, 6.14.11-1-pve, x86_64: installed
gasket/1.0, 6.14.8-2-pve, x86_64: installed
 
Booting 6.14.8-2-pve did not produce any changes.

Bash: 
dpkg -l | grep 6.14.8-2-pve
update-grub
proxmox-boot-tool kernel pin 6.14.8-2-pve
proxmox-boot-tool refresh
proxmox-boot-tool status
reboot

Bash: 
uname -r
6.14.8-2-pve

./install_coral_tpu.sh --install
reboot

Bash: 
 ./install_coral_tpu.sh --status

Coral TPU PCIe hardware not detected (lspci | grep 1ac1:089a).
Apex devices not found in /dev.

DKMS Gasket module: gasket/1.0, 6.12.43+deb13-amd64, x86_64: installed
gasket/1.0, 6.14.11-1-pve, x86_64: installed
gasket/1.0, 6.14.11-2-pve, x86_64: installed
gasket/1.0, 6.14.8-2-pve, x86_64: installed

Apex module not loaded.
libedgetpu1-std version: 16.0

Non-root access:
Udev rule for non-root access: not present.
Apex group for non-root access: not present.

Unprivileged LXC access:
Udev rule for unprivileged LXC access: not present.
Driver installation validation failed.
 
ShaMAD said:
Booting 6.14.8-2-pve did not produce any changes.

Bash: 
dpkg -l | grep 6.14.8-2-pve
update-grub
proxmox-boot-tool kernel pin 6.14.8-2-pve
proxmox-boot-tool refresh
proxmox-boot-tool status
reboot

Bash: 
uname -r
6.14.8-2-pve

./install_coral_tpu.sh --install
reboot

Bash: 
 ./install_coral_tpu.sh --status

Coral TPU PCIe hardware not detected (lspci | grep 1ac1:089a).
Apex devices not found in /dev.

DKMS Gasket module: gasket/1.0, 6.12.43+deb13-amd64, x86_64: installed
gasket/1.0, 6.14.11-1-pve, x86_64: installed
gasket/1.0, 6.14.11-2-pve, x86_64: installed
gasket/1.0, 6.14.8-2-pve, x86_64: installed

Apex module not loaded.
libedgetpu1-std version: 16.0

Non-root access:
Udev rule for non-root access: not present.
Apex group for non-root access: not present.

Unprivileged LXC access:
Udev rule for unprivileged LXC access: not present.
Driver installation validation failed.
Click to expand...
I figured out what the reason is. I bought an m.2 adapter for Dual TPU, but it doesn't have a Coral chip.o_O
https://www.makerfabs.com/dual-edge-tpu-adapter-m2-2280-b-m-key.html
 
Hi everyone,

My journey started with trying direct PCI passthrough of the Coral M.2 card to my Home Assistant OS VM. This consistently failed with a pci_irq_handler: Assertion ... failed error on my Intel N200 (Alder Lake-N) based Mini-PC.

Complicating things further, I discovered that my HAOS VM is fundamentally unstable when using the modern q35 machine type. It only boots reliably with the older i440fx default. This created a dilemma, as q35 is often required for PCI Express passthrough to work correctly.

I've since switched to the strategy of installing the drivers on the Proxmox host itself, and it was successful!

My Proxmox host now correctly loads the gasket and apex modules, and the /dev/apex_0 device is present and stable on the host.

The New Challenge: Getting /dev/apex_0 into the Home Assistant OS VM

My goal is to get this /dev/apex_0 host device into my Home Assistant OS (HAOS) VM. This is where I'm hoping for some new ideas.

Here is what I've tried so far to map the device:

  1. Using -args: I tried qm set 100 -args '-device gasket', which failed with 'gasket' is not a valid device model name.
  2. Using LXC-style rules: I tried adding these lines to my 100.conf, attempting to translate the successful LXC method to a VM:


    Code: 
    lxc.cgroup2.devices.allow: c 241:* rwm
lxc.mount.entry: /dev/apex_0 dev/apex_0 none bind,optional,create=file
My Question:
Has anyone successfully passed a host character device like /dev/apex_0 into a HAOS VM?

I'm wondering if there's a specific qm set command, a virtio device type (vdev), or another method I am overlooking that can achieve this device mapping into a hardened, container-based OS like Home Assistant.

Any suggestions on how to bridge this final gap would be greatly appreciated!

System Info:

  • Server: MOREFINE M6 Mini-PC (Intel N200)
  • Proxmox Version: 9.0.5
  • Guest OS: Home Assistant OS (VM)
 
koosoli said:
Hi everyone,

My journey started with trying direct PCI passthrough of the Coral M.2 card to my Home Assistant OS VM. This consistently failed with a pci_irq_handler: Assertion ... failed error on my Intel N200 (Alder Lake-N) based Mini-PC.

Complicating things further, I discovered that my HAOS VM is fundamentally unstable when using the modern q35 machine type. It only boots reliably with the older i440fx default. This created a dilemma, as q35 is often required for PCI Express passthrough to work correctly.

I've since switched to the strategy of installing the drivers on the Proxmox host itself, and it was successful!

My Proxmox host now correctly loads the gasket and apex modules, and the /dev/apex_0 device is present and stable on the host.

The New Challenge: Getting /dev/apex_0 into the Home Assistant OS VM

My goal is to get this /dev/apex_0 host device into my Home Assistant OS (HAOS) VM. This is where I'm hoping for some new ideas.

Here is what I've tried so far to map the device:

  1. Using -args: I tried qm set 100 -args '-device gasket', which failed with 'gasket' is not a valid device model name.
  2. Using LXC-style rules: I tried adding these lines to my 100.conf, attempting to translate the successful LXC method to a VM:


    Code: 
    lxc.cgroup2.devices.allow: c 241:* rwm
lxc.mount.entry: /dev/apex_0 dev/apex_0 none bind,optional,create=file
My Question:
Has anyone successfully passed a host character device like /dev/apex_0 into a HAOS VM?

I'm wondering if there's a specific qm set command, a virtio device type (vdev), or another method I am overlooking that can achieve this device mapping into a hardened, container-based OS like Home Assistant.

Any suggestions on how to bridge this final gap would be greatly appreciated!

System Info:

  • Server: MOREFINE M6 Mini-PC (Intel N200)
  • Proxmox Version: 9.0.5
  • Guest OS: Home Assistant OS (VM)
Click to expand...
 
Hi everyone

Thought I would chip in what I done so far.

Host is an Intel i5 with 16Gb ram. Installed proxmox 9.0.9. I have a Coral TPU PCIe installed in one of the slots.

Created a vm with cpu type host, 2 cores, 8Gb ram, 64Gb partition for the OS, 3Tb for frigate, OS Type Linux 2.4. Did a all functions pass thru of the PCI Device to the VM

Booted the VM and went into the UEFI bios and turned off Secure Boot

Booted the VM and installed Debian 13

Gave the VM a static IP and installed docker for frigate.

I then Enable IOMMU in grub and also disabled module_sig_check=false in grub.

Snippet of my /etc/default/grub below

Code: 
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on module_sig_check=false"
GRUB_CMDLINE_LINUX="quiet intel_iommu=on module_sig_check=false"

Then I did a update-grub and reboot.

Check the hardware is showing in the VM

Code: 
lspci | grep Coral
   
    01:00.0 System peripheral: Global Unichip Corp. Coral Edge TPU

Add the Coral repo and install EdgeTPU

Code: 
curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo gpg --dearmor -o /etc/apt/keyrings/coral-edgetpu.gpg

echo "deb [signed-by=/etc/apt/keyrings/coral-edgetpu.gpg] https://packages.cloud.google.com/apt coral-edgetpu-stable main" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list

apt update
apt install libedgetpu1-std

Add the gasket driver from the source as I couldn't find a Debian 13 package

Code: 
apt -y install make
apt -y install build-essential linux-headers-$(uname -r) git
apt -y install cmake devscripts debhelper dkms dh-dkms
   
cd ~
git clone https://github.com/google/gasket-driver.git
cd gasket-driver/src
cp gasket_core.c gasket_core.c.backup
sed -i 's/no_llseek/noop_llseek/g' gasket_core.c
make

If the make works continue

Install the modules

Code: 
mkdir -p /lib/modules/$(uname -r)/extra/
cp gasket.ko /lib/modules/$(uname -r)/extra/
cp apex.ko /lib/modules/$(uname -r)/extra/

Update module dependencies

Code: 
depmod -a

Create our own signing keys

Code: 
openssl req -new -x509 -newkey rsa:2048 -keyout coral_signing_key.priv -outform DER -out coral_signing_key.der -nodes -days 36500 -subj "/CN=Coral Module Signing Key"

Sign the modules

Code: 
/usr/src/linux-headers-$(uname -r)/scripts/sign-file sha256 coral_signing_key.priv coral_signing_key.der /lib/modules/$(uname -r)/extra/gasket.ko
/usr/src/linux-headers-$(uname -r)/scripts/sign-file sha256 coral_signing_key.priv coral_signing_key.der /lib/modules/$(uname -r)/extra/apex.ko

Update module dependencies

Code: 
depmod -a

Try loading the signed modules

Code: 
modprobe gasket
modprobe apex

Check if they loaded successfully

Code: 
lsmod | grep gasket
   
        gasket                135168  1 apex
   
lsmod | grep apex
   
        apex                   32768  0
        gasket                135168  1 apex

Check for device nodes

Code: 
ls -la /dev/apex*
   
    crw------- 1 root root 120, 0 Sep 15 15:19 /dev/apex_0

Set persistent permissions on the device

Code: 
sh -c "echo 'SUBSYSTEM==\"apex\", MODE=\"0660\", GROUP=\"apex\"' >> /etc/udev/rules.d/65-apex.rules"

Add group permissions

Code: 
groupadd apex
adduser $USER apex

Then rebooted the VM

I then checked to see if the hardware was visble.

Code: 
 lspci -v | grep -A 10 -i coral
 
        01:00.0 System peripheral: Global Unichip Corp. Coral Edge TPU (prog-if ff)
        Subsystem: Global Unichip Corp. Coral Edge TPU
        Physical Slot: 0
        Flags: bus master, fast devsel, latency 0, IRQ 16
        Memory at 7000100000 (64-bit, prefetchable) [size=16K]
        Memory at 7000000000 (64-bit, prefetchable) [size=1M]
        Capabilities: [80] Express Endpoint, MSI 00
        Capabilities: [d0] MSI-X: Enable+ Count=128 Masked-
        Capabilities: [e0] MSI: Enable- Count=1/32 Maskable- 64bit+
        Capabilities: [f8] Power Management version 3
        Capabilities: [100] Vendor Specific Information: ID=1556 Rev=1 Len=008 <?>
        Capabilities: [108] Latency Tolerance Reporting

At this point I've noticed the console is showing the below message

Code: 
apex 0000:01:00.0: RAM did not enable within timeout (12000 ms)

So I did dmesg | grep apex and got the below

Code: 
[ 1796.588309] apex 0000:01:00.0: RAM did not enable within timeout (12000 ms)
[ 1796.588553] apex 0000:01:00.0: Error in device open cb: -110
[ 1796.589191] apex 0000:01:00.0: Apex performance not throttled due to temperature
[ 1801.716215] apex 0000:01:00.0: Apex performance not throttled due to temperature

My Frigate docker logs are also showing the below

Code: 
docker logs frigate 2>&1 | grep -i -E "(coral|tpu|edgetpu|detector)"

    detector.coral                 INFO    : Starting detection process: 321
    frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as pci
    frigate.app                    INFO    : Output process started: 346
    frigate.detectors.plugins.edgetpu_tfl ERROR   : No EdgeTPU was detected. If you do not have a Coral device yet, you must configure CPU detectors.

docker logs frigate 2>&1 | grep -i error | grep -i tpu

    frigate.detectors.plugins.edgetpu_tfl ERROR   : No EdgeTPU was detected. If you do not have a Coral device yet, you must configure CPU detectors.
    ValueError: Failed to load delegate from libedgetpu.so.1.0

I then decided to try a second VM but this time running Debian 12.

The process was similar to the above, except I did an apt install gasket-dkms rather than building from source

The result is the same.

What am I missing?
 
You must log in or register to reply here.
Top Bottom