Update Error with Coral TPU Drivers

It seems like I'm still unable to use the google coral in an LXC. I have done the steps as above without any issues.

lsusb gives the expected output from within the LXC:
Bus 002 Device 002: ID 18d1:9302 Google Inc.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub


LXC config is unchanged:
lxc.mount.entry: /dev/serial/by-id dev/serial/by-id none bind,optional,create=dir
lxc.mount.entry: /dev/ttyUSB0 dev/ttyUSB0 none bind,optional,create=file
lxc.mount.entry: /dev/ttyUSB1 dev/ttyUSB1 none bind,optional,create=file
lxc.mount.entry: /dev/ttyACM0 dev/ttyACM0 none bind,optional,create=file
lxc.mount.entry: /dev/ttyACM1 dev/ttyACM1 none bind,optional,create=file
lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.cgroup2.devices.allow: c 29:0 rwm
lxc.mount.entry: /dev/fb0 dev/fb0 none bind,optional,create=file
lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file


But I still get the following error in frigate:

2023-12-09 17:47:22.078721637 [2023-12-09 17:47:22] frigate.detectors.plugins.edgetpu_tfl ERROR : No EdgeTPU was detected. If you do not have a Coral device yet, you must configure CPU detectors.

any way to check what could still be wrong?

Looking back, I'm using a USB TPU, most likely these steps are only for PCI TPU is that correct?
 
Last edited:
robko said:
It seems like I'm still unable to use the google coral in an LXC. I have done the steps as above without any issues.

lsusb gives the expected output from within the LXC:
Bus 002 Device 002: ID 18d1:9302 Google Inc.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub


LXC config is unchanged:
lxc.mount.entry: /dev/serial/by-id dev/serial/by-id none bind,optional,create=dir
lxc.mount.entry: /dev/ttyUSB0 dev/ttyUSB0 none bind,optional,create=file
lxc.mount.entry: /dev/ttyUSB1 dev/ttyUSB1 none bind,optional,create=file
lxc.mount.entry: /dev/ttyACM0 dev/ttyACM0 none bind,optional,create=file
lxc.mount.entry: /dev/ttyACM1 dev/ttyACM1 none bind,optional,create=file
lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.cgroup2.devices.allow: c 29:0 rwm
lxc.mount.entry: /dev/fb0 dev/fb0 none bind,optional,create=file
lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file


But I still get the following error in frigate:

2023-12-09 17:47:22.078721637 [2023-12-09 17:47:22] frigate.detectors.plugins.edgetpu_tfl ERROR : No EdgeTPU was detected. If you do not have a Coral device yet, you must configure CPU detectors.

any way to check what could still be wrong?

Looking back, I'm using a USB TPU, most likely these steps are only for PCI TPU is that correct?
Click to expand...
Those steps are for an Intel integrated GPU, not the PCIe Coral. I've heard bad things about the USB Coral, so if Frigate supports GPU accelerated detection and you have an Intel CPU/integrated graphics you might want to give that a try w/the steps above.
 
I cant get any of these steps to work, be it removing gasket-dkms, installing dkms, or anything else. I always get the same errors:

Code: 
Error! One or more modules failed to install during autoinstall.
Refer to previous errors for more information.
dkms: autoinstall for kernel: 6.5.11-7-pve failed!
run-parts: /etc/kernel/postinst.d/dkms exited with return code 11
Failed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/proxmox-kernel-6.5.11-7-pve-signed.postinst line 20.
dpkg: error processing package proxmox-kernel-6.5.11-7-pve-signed (--configure):
 installed proxmox-kernel-6.5.11-7-pve-signed package post-installation script subprocess returned error exit status 2
dpkg: dependency problems prevent configuration of proxmox-kernel-6.5:
 proxmox-kernel-6.5 depends on proxmox-kernel-6.5.11-7-pve-signed | proxmox-kernel-6.5.11-7-pve; however:
  Package proxmox-kernel-6.5.11-7-pve-signed is not configured yet.
  Package proxmox-kernel-6.5.11-7-pve is not installed.
  Package proxmox-kernel-6.5.11-7-pve-signed which provides proxmox-kernel-6.5.11-7-pve is not configured yet.

dpkg: error processing package proxmox-kernel-6.5 (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of proxmox-default-kernel:
 proxmox-default-kernel depends on proxmox-kernel-6.5; however:
  Package proxmox-kernel-6.5 is not configured yet.

dpkg: error processing package proxmox-default-kernel (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of proxmox-ve:
 proxmox-ve depends on proxmox-default-kernel; however:
  Package proxmox-default-kernel is not configured yet.

dpkg: error processing package proxmox-ve (--configure):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 proxmox-kernel-6.5.11-7-pve-signed
 proxmox-kernel-6.5
 proxmox-default-kernel
 proxmox-ve
 
I managed to solve the above error, but now I can't see the PCI Coral TPU from the Frigate LXC. Can anyone point me to the proper setup?

I've had them working in the past, but after the update to the new kernel I'm rebuilding the LXC for Frigate and can't remember the exact steps to get the TPUs to show up.

My old Frigate LXC is working fine. The new one throws this error in frigate:

Code: 
2023-12-11 14:57:05.888824880  [2023-12-11 14:57:05] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as pci:1
2023-12-11 14:57:05.888826383  [2023-12-11 14:57:05] frigate.detectors.plugins.edgetpu_tfl ERROR   : No EdgeTPU was detected. If you do not have a Coral device yet, you must configure CPU detectors.
2023-12-11 14:57:05.888827475  [2023-12-11 14:57:05] frigate.detectors.plugins.edgetpu_tfl ERROR   : No EdgeTPU was detected. If you do not have a Coral device yet, you must configure CPU detectors.
2023-12-11 14:57:05.889261497  Process detector:coralpci2:
2023-12-11 14:57:05.889262609  Process detector:coralpci1:
2023-12-11 14:57:05.889418524  Traceback (most recent call last):
2023-12-11 14:57:05.889419566    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 160, in load_delegate
2023-12-11 14:57:05.889420117      delegate = Delegate(library, options)
2023-12-11 14:57:05.889420698    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 119, in __init__
2023-12-11 14:57:05.889421179      raise ValueError(capture.message)
2023-12-11 14:57:05.889421540  ValueError
2023-12-11 14:57:05.889421910
2023-12-11 14:57:05.889422371  During handling of the above exception, another exception occurred:
2023-12-11 14:57:05.889422732
2023-12-11 14:57:05.889815325  Traceback (most recent call last):
2023-12-11 14:57:05.889816437    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2023-12-11 14:57:05.889816918      self.run()
2023-12-11 14:57:05.889817449    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2023-12-11 14:57:05.889817900      self._target(*self._args, **self._kwargs)
2023-12-11 14:57:05.889818361    File "/opt/frigate/frigate/object_detection.py", line 98, in run_detector
2023-12-11 14:57:05.889818872      object_detector = LocalObjectDetector(detector_config=detector_config)
2023-12-11 14:57:05.889819323    File "/opt/frigate/frigate/object_detection.py", line 52, in __init__
2023-12-11 14:57:05.889819754      self.detect_api = create_detector(detector_config)
2023-12-11 14:57:05.889820244    File "/opt/frigate/frigate/detectors/__init__.py", line 24, in create_detector
2023-12-11 14:57:05.889829452      return api(detector_config)
2023-12-11 14:57:05.889830013    File "/opt/frigate/frigate/detectors/plugins/edgetpu_tfl.py", line 37, in __init__
2023-12-11 14:57:05.889830584      edge_tpu_delegate = load_delegate("libedgetpu.so.1.0", device_config)
2023-12-11 14:57:05.889831105    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 162, in load_delegate
2023-12-11 14:57:05.889843959      raise ValueError('Failed to load delegate from {}\n{}'.format(
2023-12-11 14:57:05.889844510  ValueError: Failed to load delegate from libedgetpu.so.1.0
2023-12-11 14:57:05.889844941
2023-12-11 14:57:05.889845342  Traceback (most recent call last):
2023-12-11 14:57:05.889845843    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 160, in load_delegate
2023-12-11 14:57:05.889846294      delegate = Delegate(library, options)
2023-12-11 14:57:05.889846825    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 119, in __init__
2023-12-11 14:57:05.889847256      raise ValueError(capture.message)
2023-12-11 14:57:05.889858246  ValueError
2023-12-11 14:57:05.889858657
2023-12-11 14:57:05.889859118  During handling of the above exception, another exception occurred:
2023-12-11 14:57:05.889859469
2023-12-11 14:57:05.889859879  Traceback (most recent call last):
2023-12-11 14:57:05.889860420    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2023-12-11 14:57:05.889860801      self.run()
2023-12-11 14:57:05.889861312    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2023-12-11 14:57:05.889861753      self._target(*self._args, **self._kwargs)
2023-12-11 14:57:05.889876080    File "/opt/frigate/frigate/object_detection.py", line 98, in run_detector
2023-12-11 14:57:05.889876661      object_detector = LocalObjectDetector(detector_config=detector_config)
2023-12-11 14:57:05.889877132    File "/opt/frigate/frigate/object_detection.py", line 52, in __init__
2023-12-11 14:57:05.889877583      self.detect_api = create_detector(detector_config)
2023-12-11 14:57:05.889878124    File "/opt/frigate/frigate/detectors/__init__.py", line 24, in create_detector
2023-12-11 14:57:05.889878535      return api(detector_config)
2023-12-11 14:57:05.889879016    File "/opt/frigate/frigate/detectors/plugins/edgetpu_tfl.py", line 37, in __init__
2023-12-11 14:57:05.889879487      edge_tpu_delegate = load_delegate("libedgetpu.so.1.0", device_config)
2023-12-11 14:57:05.889880038    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 162, in load_delegate
2023-12-11 14:57:05.889880499      raise ValueError('Failed to load delegate from {}\n{}'.format(
2023-12-11 14:57:05.889880939  ValueError: Failed to load delegate from libedgetpu.so.1.0
2023-12-11 14:57:05.889881300
2023-12-11 14:57:05.892266806  [2023-12-11 14:57:05] frigate.app                    INFO    : Output process started: 1222

So the issue must be with the setup of the new LXC. I have all the cgroup2 passthroughs set in the LXC conf file. One works, One doesnt.

Edit: Ok, a little more progress. If I make the LXC privileged, it works. So now I just need to figure out how to make it work *unprivileged*.


Is anyone in this thread using a PCIe Coral TPU in an unprivileged LXC?
 
Last edited:
LordRatner said:
I managed to solve the above error, but now I can't see the PCI Coral TPU from the Frigate LXC. Can anyone point me to the proper setup?

I've had them working in the past, but after the update to the new kernel I'm rebuilding the LXC for Frigate and can't remember the exact steps to get the TPUs to show up.

My old Frigate LXC is working fine. The new one throws this error in frigate:

Code: 
2023-12-11 14:57:05.888824880  [2023-12-11 14:57:05] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as pci:1
2023-12-11 14:57:05.888826383  [2023-12-11 14:57:05] frigate.detectors.plugins.edgetpu_tfl ERROR   : No EdgeTPU was detected. If you do not have a Coral device yet, you must configure CPU detectors.
2023-12-11 14:57:05.888827475  [2023-12-11 14:57:05] frigate.detectors.plugins.edgetpu_tfl ERROR   : No EdgeTPU was detected. If you do not have a Coral device yet, you must configure CPU detectors.
2023-12-11 14:57:05.889261497  Process detector:coralpci2:
2023-12-11 14:57:05.889262609  Process detector:coralpci1:
2023-12-11 14:57:05.889418524  Traceback (most recent call last):
2023-12-11 14:57:05.889419566    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 160, in load_delegate
2023-12-11 14:57:05.889420117      delegate = Delegate(library, options)
2023-12-11 14:57:05.889420698    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 119, in __init__
2023-12-11 14:57:05.889421179      raise ValueError(capture.message)
2023-12-11 14:57:05.889421540  ValueError
2023-12-11 14:57:05.889421910
2023-12-11 14:57:05.889422371  During handling of the above exception, another exception occurred:
2023-12-11 14:57:05.889422732
2023-12-11 14:57:05.889815325  Traceback (most recent call last):
2023-12-11 14:57:05.889816437    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2023-12-11 14:57:05.889816918      self.run()
2023-12-11 14:57:05.889817449    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2023-12-11 14:57:05.889817900      self._target(*self._args, **self._kwargs)
2023-12-11 14:57:05.889818361    File "/opt/frigate/frigate/object_detection.py", line 98, in run_detector
2023-12-11 14:57:05.889818872      object_detector = LocalObjectDetector(detector_config=detector_config)
2023-12-11 14:57:05.889819323    File "/opt/frigate/frigate/object_detection.py", line 52, in __init__
2023-12-11 14:57:05.889819754      self.detect_api = create_detector(detector_config)
2023-12-11 14:57:05.889820244    File "/opt/frigate/frigate/detectors/__init__.py", line 24, in create_detector
2023-12-11 14:57:05.889829452      return api(detector_config)
2023-12-11 14:57:05.889830013    File "/opt/frigate/frigate/detectors/plugins/edgetpu_tfl.py", line 37, in __init__
2023-12-11 14:57:05.889830584      edge_tpu_delegate = load_delegate("libedgetpu.so.1.0", device_config)
2023-12-11 14:57:05.889831105    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 162, in load_delegate
2023-12-11 14:57:05.889843959      raise ValueError('Failed to load delegate from {}\n{}'.format(
2023-12-11 14:57:05.889844510  ValueError: Failed to load delegate from libedgetpu.so.1.0
2023-12-11 14:57:05.889844941
2023-12-11 14:57:05.889845342  Traceback (most recent call last):
2023-12-11 14:57:05.889845843    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 160, in load_delegate
2023-12-11 14:57:05.889846294      delegate = Delegate(library, options)
2023-12-11 14:57:05.889846825    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 119, in __init__
2023-12-11 14:57:05.889847256      raise ValueError(capture.message)
2023-12-11 14:57:05.889858246  ValueError
2023-12-11 14:57:05.889858657
2023-12-11 14:57:05.889859118  During handling of the above exception, another exception occurred:
2023-12-11 14:57:05.889859469
2023-12-11 14:57:05.889859879  Traceback (most recent call last):
2023-12-11 14:57:05.889860420    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2023-12-11 14:57:05.889860801      self.run()
2023-12-11 14:57:05.889861312    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2023-12-11 14:57:05.889861753      self._target(*self._args, **self._kwargs)
2023-12-11 14:57:05.889876080    File "/opt/frigate/frigate/object_detection.py", line 98, in run_detector
2023-12-11 14:57:05.889876661      object_detector = LocalObjectDetector(detector_config=detector_config)
2023-12-11 14:57:05.889877132    File "/opt/frigate/frigate/object_detection.py", line 52, in __init__
2023-12-11 14:57:05.889877583      self.detect_api = create_detector(detector_config)
2023-12-11 14:57:05.889878124    File "/opt/frigate/frigate/detectors/__init__.py", line 24, in create_detector
2023-12-11 14:57:05.889878535      return api(detector_config)
2023-12-11 14:57:05.889879016    File "/opt/frigate/frigate/detectors/plugins/edgetpu_tfl.py", line 37, in __init__
2023-12-11 14:57:05.889879487      edge_tpu_delegate = load_delegate("libedgetpu.so.1.0", device_config)
2023-12-11 14:57:05.889880038    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 162, in load_delegate
2023-12-11 14:57:05.889880499      raise ValueError('Failed to load delegate from {}\n{}'.format(
2023-12-11 14:57:05.889880939  ValueError: Failed to load delegate from libedgetpu.so.1.0
2023-12-11 14:57:05.889881300
2023-12-11 14:57:05.892266806  [2023-12-11 14:57:05] frigate.app                    INFO    : Output process started: 1222

So the issue must be with the setup of the new LXC. I have all the cgroup2 passthroughs set in the LXC conf file. One works, One doesnt.

Edit: Ok, a little more progress. If I make the LXC privileged, it works. So now I just need to figure out how to make it work *unprivileged*.


Is anyone in this thread using a PCIe Coral TPU in an unprivileged LXC?
Click to expand...
Having similar errors in a similar setup but in an privileged container. Do you mind sharing your proxmox container config (indeed, with the cgroup mappings and all)? Getting to work in an unprivilged mode would be my next step as well!
 
aabbccdd said:
Having similar errors in a similar setup but in an privileged container. Do you mind sharing your proxmox container config (indeed, with the cgroup mappings and all)? Getting to work in an unprivilged mode would be my next step as well!
Click to expand...
What is your exact setup? What GPU/TPU/APU hardware are you using, standalone Proxmox or cluster, and what storage config?
 
LordRatner said:
What is your exact setup? What GPU/TPU/APU hardware are you using, standalone Proxmox or cluster, and what storage config?
Click to expand...
Hi. I don't see how the GPU or the storage configuration could be relevant, but perhaps that is why I can't figure it out myself so thanks for your interest in troubleshooting

GPU is a RX480 with a Coral M.2 Accelerator with Dual Edge TPU (only one side is recognized apparently). Using Proxmox standalone (not in a cluster) using ZFS for root and all other storage applications.

Coral
07:00.0 System peripheral [0880]: Global Unichip Corp. Coral Edge TPU [1ac1:089a]

GPU
0c:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev c7) 0c:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1002:aaf0]

Error in frigate
2023-12-14 21:28:26.924383292 [INFO] Starting Frigate...2023-12-14 21:28:27.468146300 [2023-12-14 21:28:27] frigate.app INFO : Starting Frigate (0.12.1-367d724)2023-12-14 21:28:27.482742643 [2023-12-14 21:28:27] peewee_migrate INFO : Starting migrations2023-12-14 21:28:27.485622162 [2023-12-14 21:28:27] peewee_migrate INFO : There is nothing to migrate2023-12-14 21:28:27.498167211 [2023-12-14 21:28:27] frigate.app INFO : Output process started: 6812023-12-14 21:28:27.499631892 [2023-12-14 21:28:27] detector.coral1 INFO : Starting detection process: 6792023-12-14 21:28:27.499799935 [2023-12-14 21:28:27] frigate.detectors.plugins.edgetpu_tfl INFO : Attempting to load TPU as pci:02023-12-14 21:28:27.499950288 [2023-12-14 21:28:27] frigate.detectors.plugins.edgetpu_tfl ERROR : No EdgeTPU was detected. If you do not have a Coral device yet, you must configure CPU detectors.2023-12-14 21:28:27.500820996 Process detector:coral1:2023-12-14 21:28:27.500822286 Traceback (most recent call last):2023-12-14 21:28:27.500823056 File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 160, in load_delegate2023-12-14 21:28:27.500824726 delegate = Delegate(library, options)2023-12-14 21:28:27.500825606 File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 119, in __init__2023-12-14 21:28:27.500835466 raise ValueError(capture.message)2023-12-14 21:28:27.500836006 ValueError2023-12-14 21:28:27.500836386 2023-12-14 21:28:27.500837226 During handling of the above exception, another exception occurred:2023-12-14 21:28:27.500837636 2023-12-14 21:28:27.500838086 Traceback (most recent call last):2023-12-14 21:28:27.500838616 File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap2023-12-14 21:28:27.500850587 self.run()2023-12-14 21:28:27.500851167 File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run2023-12-14 21:28:27.500851677 self._target(*self._args, **self._kwargs)2023-12-14 21:28:27.500852207 File "/opt/frigate/frigate/object_detection.py", line 98, in run_detector2023-12-14 21:28:27.500861527 object_detector = LocalObjectDetector(detector_config=detector_config)2023-12-14 21:28:27.500862077 File "/opt/frigate/frigate/object_detection.py", line 52, in __init__2023-12-14 21:28:27.500862617 self.detect_api = create_detector(detector_config)2023-12-14 21:28:27.500863157 File "/opt/frigate/frigate/detectors/__init__.py", line 24, in create_detector2023-12-14 21:28:27.500863607 return api(detector_config)2023-12-14 21:28:27.500864107 File "/opt/frigate/frigate/detectors/plugins/edgetpu_tfl.py", line 37, in __init__2023-12-14 21:28:27.500864607 edge_tpu_delegate = load_delegate("libedgetpu.so.1.0", device_config)2023-12-14 21:28:27.500875547 File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 162, in load_delegate2023-12-14 21:28:27.500876107 raise ValueError('Failed to load delegate from {}\n{}'.format(2023-12-14 21:28:27.500876577 ValueError: Failed to load delegate from libedgetpu.so.1.02023-12-14 21:28:27.500876987 2023-12-14 21:28:27.512518018 [2023-12-14 21:28:27] frigate.app INFO : Camera processor started for cam2: 6832023-12-14 21:28:27.512519728 [2023-12-14 21:28:27] frigate.app INFO : Camera processor started for cam1: 6842023-12-14 21:28:27.512520488 [2023-12-14 21:28:27] frigate.app INFO : Capture process started for cam3: 6852023-12-14 21:28:27.513300674 [2023-12-14 21:28:27] frigate.app INFO : Capture process started for cam4: 6862023-12-14 21:28:32.717579600 [2023-12-14 21:28:32] frigate.record ERROR : Error occurred when attempting to maintain recording cache2023-12-14 21:28:32.717706723 [2023-12-14 21:28:32] frigate.record ERROR : list index out of range
 
aabbccdd said:
Hi. I don't see how the GPU or the storage configuration could be relevant, but perhaps that is why I can't figure it out myself so thanks for your interest in troubleshooting

GPU is a RX480 with a Coral M.2 Accelerator with Dual Edge TPU (only one side is recognized apparently). Using Proxmox standalone (not in a cluster) using ZFS for root and all other storage applications.

Coral
07:00.0 System peripheral [0880]: Global Unichip Corp. Coral Edge TPU [1ac1:089a]

GPU
0c:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev c7) 0c:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1002:aaf0]

Error in frigate
2023-12-14 21:28:26.924383292 [INFO] Starting Frigate...2023-12-14 21:28:27.468146300 [2023-12-14 21:28:27] frigate.app INFO : Starting Frigate (0.12.1-367d724)2023-12-14 21:28:27.482742643 [2023-12-14 21:28:27] peewee_migrate INFO : Starting migrations2023-12-14 21:28:27.485622162 [2023-12-14 21:28:27] peewee_migrate INFO : There is nothing to migrate2023-12-14 21:28:27.498167211 [2023-12-14 21:28:27] frigate.app INFO : Output process started: 6812023-12-14 21:28:27.499631892 [2023-12-14 21:28:27] detector.coral1 INFO : Starting detection process: 6792023-12-14 21:28:27.499799935 [2023-12-14 21:28:27] frigate.detectors.plugins.edgetpu_tfl INFO : Attempting to load TPU as pci:02023-12-14 21:28:27.499950288 [2023-12-14 21:28:27] frigate.detectors.plugins.edgetpu_tfl ERROR : No EdgeTPU was detected. If you do not have a Coral device yet, you must configure CPU detectors.2023-12-14 21:28:27.500820996 Process detector:coral1:2023-12-14 21:28:27.500822286 Traceback (most recent call last):2023-12-14 21:28:27.500823056 File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 160, in load_delegate2023-12-14 21:28:27.500824726 delegate = Delegate(library, options)2023-12-14 21:28:27.500825606 File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 119, in __init__2023-12-14 21:28:27.500835466 raise ValueError(capture.message)2023-12-14 21:28:27.500836006 ValueError2023-12-14 21:28:27.500836386 2023-12-14 21:28:27.500837226 During handling of the above exception, another exception occurred:2023-12-14 21:28:27.500837636 2023-12-14 21:28:27.500838086 Traceback (most recent call last):2023-12-14 21:28:27.500838616 File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap2023-12-14 21:28:27.500850587 self.run()2023-12-14 21:28:27.500851167 File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run2023-12-14 21:28:27.500851677 self._target(*self._args, **self._kwargs)2023-12-14 21:28:27.500852207 File "/opt/frigate/frigate/object_detection.py", line 98, in run_detector2023-12-14 21:28:27.500861527 object_detector = LocalObjectDetector(detector_config=detector_config)2023-12-14 21:28:27.500862077 File "/opt/frigate/frigate/object_detection.py", line 52, in __init__2023-12-14 21:28:27.500862617 self.detect_api = create_detector(detector_config)2023-12-14 21:28:27.500863157 File "/opt/frigate/frigate/detectors/__init__.py", line 24, in create_detector2023-12-14 21:28:27.500863607 return api(detector_config)2023-12-14 21:28:27.500864107 File "/opt/frigate/frigate/detectors/plugins/edgetpu_tfl.py", line 37, in __init__2023-12-14 21:28:27.500864607 edge_tpu_delegate = load_delegate("libedgetpu.so.1.0", device_config)2023-12-14 21:28:27.500875547 File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 162, in load_delegate2023-12-14 21:28:27.500876107 raise ValueError('Failed to load delegate from {}\n{}'.format(2023-12-14 21:28:27.500876577 ValueError: Failed to load delegate from libedgetpu.so.1.02023-12-14 21:28:27.500876987 2023-12-14 21:28:27.512518018 [2023-12-14 21:28:27] frigate.app INFO : Camera processor started for cam2: 6832023-12-14 21:28:27.512519728 [2023-12-14 21:28:27] frigate.app INFO : Camera processor started for cam1: 6842023-12-14 21:28:27.512520488 [2023-12-14 21:28:27] frigate.app INFO : Capture process started for cam3: 6852023-12-14 21:28:27.513300674 [2023-12-14 21:28:27] frigate.app INFO : Capture process started for cam4: 6862023-12-14 21:28:32.717579600 [2023-12-14 21:28:32] frigate.record ERROR : Error occurred when attempting to maintain recording cache2023-12-14 21:28:32.717706723 [2023-12-14 21:28:32] frigate.record ERROR : list index out of range
Click to expand...
Ok, first off, you need this to use both TPUs: https://www.makerfabs.com/dual-edge-tpu-adapter.html

Next, have you ever had the TPU working? If so, did you follow the steps LiloBZH put in this thread? This should be done on the Proxmox host, not in the LXC. PVE8.1 broke the old drivers.

Next, you need to give the LXC access to the TPU(s), which is done in the LXC conf file. ls -l /dev and look for apex_0 (and when you get the adapter for the second TPU, apex_1). To the left you'll see two numbers separated by a comma. Add this line to your .conf, but put the numbers you see, not the ones I used:

lxc.cgroup2.devices.allow: c 120:1 rwm

You also need to add this line:
lxc.mount.entry: /dev/apex_0 dev/apex_0 none bind,optional,create=file (with another line for apex_1 if you get the adapter)

You should already have similar lines in the .conf for your GPU. If not, you need to add these entry lines:

Code: 
lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
lxc.mount.entry: /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-modeset dev/nvidia-modeset none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,create=file
lxc.mount.entry: /dev/nvram dev/nvram none bind,optional,create=file

And you need to add the associated devices.allow lines like you did for the apex_0 device, this time for the devices that match the entry lines (dev/nvidia0, dev/nvidia-modset, etc). Pay close attention to the slashes. The first path has a leading slash, the second does not.

You also need the nvidia-container-toolkit installed in the LXC. You need the nvidia drivers installed in the host, and again in the LXC, but this time with the --no-kernel-module flag.

Additionally, if you are using an pre-8.1 LXC, your docker instance might be using the wrong storage overlay. The 6.5 kernel in PVE8.1 has zfs 2.2.2 which fixes the docker issue and allows for the use of overlay2 with a zfs pool. I just started from scratch with a new LXC since I wanted to update Frigate at the same time.

Let me know if you get stuck.
 
Weird, the Proxmox host does not show the /dev/apex* devices but the Frigate 'system' status overview displays the 'coral1' tpu (m.2 dual) with the logs stating errors.

Yes, it used to work.

1702675065351.png
1702675115746.png

Do you, @LordRatner have a reference for the impact of the docker issue of pre PVE8.1 containers please? Might also be related to a problem I'm having elsewhere Invisible ZFS dataset on zfs 2.2.2 kernel 6.5.11-7-pve.
 
Last edited:
aabbccdd said:
Weird, the Proxmox host does not show the /dev/apex* devices but the Frigate 'system' status overview displays the 'coral1' tpu (m.2 dual) with the logs stating errors.
Click to expand...
It'll put that no matter what. The 10 ms inference is a giveaway that it's not working. See "No EdgeTPU was detected" in the logs.

If you don't have the apex devices on the host, you can't make it work. Go through the steps from LiloBZH and make sure you get the TPUs on the host before worrying about the LXC. Since it was working before, you most likely just need those steps to install the functional drivers, and then update the .conf file for the new cgroup2 numbers.

aabbccdd said:
Do you, @LordRatner have a reference for the impact of the docker issue of pre PVE8.1 containers please? Might also be related to a problem I'm having elsewhere Invisible ZFS dataset on zfs 2.2.2 kernel 6.5.11-7-pve.
Click to expand...
You can google around for it, I don't have the threads handy. Basically it would create a ton of ghost data and cause the LXC to run slowly. But this only applies to running docker in an LXC on a zfs pool.

The nice part about Docker is you can easily spin things back up, so I would consider just starting new LXCs for docker to ensure the storage overlay issue is fixed. If you have data on the LXC you can't easily move, you maybe could uninstall docker, upgrade the LXC then reinstall docker and make sure the "overlay2" filesystem is being used.
 
Thank you for your reply. I'll troubleshoot and indeed get the /dev/apex_0 device back first, carefully re-reading your posts.

Is there a way to reboot without starting all VM's and containers, a maintenance mode if you will, to speed up testing btw?
 
Hello,
I am trying to follow the recommended steps but while trying to run "apt install dh-dkms" I am getting "E: Unable to locate package dh-dkms". Can someone help me here?
Thanks.
 
Hello all.
I am also asking for help, I have done all the steps and read several times and nothing. I do not get any errors but no apex drivers get install. running proxmox 8,1,3. kernel 6.5.11-7-pve.
Do not know what prints do i need to see were is the problem. So Help
Thank you
 
Same issue as jorgeg73 above on kernel 6.5.11-7-pve. I'm able to install gasket-dkms via the built .deb, and no errors from update and upgrade. However, no apex listed in /dev. I've tried to reinstall libedgetpu1-std after getting gasket installed, still no luck.

As of now, I think I've hit the limit of my ability to troubleshoot this.

More information which may be useful:

Markdown (GitHub flavored): 
root@pve:~# lsmod | grep apex
apex                   28672  0
gasket                126976  1 apex

Markdown (GitHub flavored): 
root@pve:~# dmesg |grep gasket
[   21.324419] gasket: module verification failed: signature and/or required key missing - tainting kernel
root@pve:~# dmesg |grep apex
[   21.359736] apex 0000:65:00.0: enabling device (0140 -> 0142)
[   26.492924] apex 0000:65:00.0: Apex performance not throttled due to temperature

Markdown (GitHub flavored): 
root@pve:~# modinfo gasket
filename:       /lib/modules/6.5.11-7-pve/updates/dkms/gasket.ko
author:         Rob Springer <rspringer@google.com>
license:        GPL v2
version:        1.1.4
description:    Google Gasket driver framework
import_ns:      DMA_BUF
srcversion:     2E9213B784051969509C03E
depends:       
retpoline:      Y
name:           gasket
vermagic:       6.5.11-7-pve SMP preempt mod_unload modversions
sig_id:         PKCS#7
signer:         DKMS module signing key
sig_key:        71:BA:3A:4A:0D:D2:C3:14:52:6C:59:49:15:CF:8D:D8:47:2E:04:EA
sig_hashalgo:   sha512
signature:      51:CD:9E:12:43:44:C6:BE:AE:BD:DF:26:0B:A4:F7:15:0A:2A:F1:3B:
                6B:57:8D:F5:A0:6D:DF:FC:CF:73:4E:BF:1C:42:E3:82:C6:A4:E9:58:
                97:09:A2:C4:A4:C3:B7:B3:06:2E:F9:86:4E:D1:B2:DF:29:8D:26:68:
                AD:E1:8A:74:36:D2:95:71:79:57:38:33:36:38:AC:49:8D:BF:19:80:
                32:6D:FA:08:BD:F1:8D:BB:11:1A:E1:A4:3A:C2:66:1B:CC:B9:A9:FD:
                61:45:B6:FB:F7:38:1D:D2:40:2C:9B:B2:96:DF:19:F0:10:89:3A:6B:
                A6:92:67:E5:2A:27:07:C4:CA:72:20:AF:EB:7C:48:6B:2C:B5:BB:DA:
                D0:ED:25:38:59:70:ED:31:C9:E1:26:00:AA:BF:8D:BB:B0:1B:42:87:
                BB:CC:93:C9:1D:A5:58:63:F4:5F:7A:83:EC:BC:73:B2:BB:65:84:4F:
                FB:3F:D0:61:88:4D:CD:EB:17:DA:C6:7C:71:21:CF:E4:1B:52:88:35:
                59:4F:D1:8E:00:21:DC:C4:3A:33:7C:8F:CE:2C:30:42:F6:B4:16:52:
                54:5B:27:B1:26:23:76:D7:D0:69:87:9A:58:0E:83:46:85:EF:DA:38:
                BF:35:93:A6:A1:85:8B:6F:92:76:94:13:F9:CA:51:1B
parm:           dma_bit_mask:int
root@pve:~# modinfo apex
filename:       /lib/modules/6.5.11-7-pve/updates/dkms/apex.ko
author:         John Joseph <jnjoseph@google.com>
license:        GPL v2
version:        1.2
description:    Google Apex driver
srcversion:     700E8BBBE9CC23C6EC17712
alias:          pci:v00001AC1d0000089Asv*sd*bc*sc*i*
depends:        gasket
retpoline:      Y
name:           apex
vermagic:       6.5.11-7-pve SMP preempt mod_unload modversions
sig_id:         PKCS#7
signer:         DKMS module signing key
sig_key:        71:BA:3A:4A:0D:D2:C3:14:52:6C:59:49:15:CF:8D:D8:47:2E:04:EA
sig_hashalgo:   sha512
signature:      5A:07:83:7F:2C:89:05:9D:3A:A9:88:C8:98:7A:A7:D6:D6:F5:08:45:
                1C:D4:46:A3:4F:64:D8:7F:02:8E:C9:21:33:27:CD:CE:C7:43:9E:B3:
                A9:44:1C:FB:E8:B0:C7:7C:BD:F1:CE:C1:3A:ED:05:31:6F:3A:88:41:
                2E:C8:3F:ED:06:92:20:08:B2:FF:73:12:FD:AC:08:65:68:72:72:99:
                E7:5A:9C:FB:7A:E3:82:DA:15:F0:D8:B7:72:91:12:50:6F:D4:EE:1E:
                15:55:71:A9:6D:56:AA:F7:C5:82:36:BA:70:4F:68:D5:45:AD:46:38:
                04:BD:F1:A4:52:E6:C2:5D:6E:D8:5D:43:95:4A:68:DD:AC:72:92:37:
                18:9D:D3:2A:0D:EA:E4:52:69:0B:D6:4A:9C:56:24:A7:BF:23:08:8F:
                BA:86:14:9B:78:31:E5:78:A3:AE:90:16:16:13:88:BC:EA:41:98:0E:
                4D:8C:7C:25:74:59:67:F7:8D:29:30:85:E7:AF:46:FA:A4:EB:72:32:
                D3:29:43:6B:B9:6E:16:58:E1:EE:E4:71:8B:0B:E9:0C:A9:F8:C2:7C:
                FF:AD:D4:F7:4C:1E:EA:CB:00:A4:93:DD:DC:44:7C:BE:FD:D4:B2:47:
                0E:35:2E:53:FF:DC:33:D2:41:32:89:AB:5E:43:03:16
parm:           allow_power_save:int
parm:           allow_sw_clock_gating:int
parm:           allow_hw_clock_gating:int
parm:           bypass_top_level:int
parm:           trip_point0_temp:int
parm:           trip_point1_temp:int
parm:           trip_point2_temp:int
parm:           hw_temp_warn1:int
parm:           hw_temp_warn2:int
parm:           hw_temp_warn1_en:bool
parm:           hw_temp_warn2_en:bool
parm:           temp_poll_interval:int
 
Last edited:
I also have this issue (/dev/apex_0 not appearing). This was working fine in 8.0, did a clean install of proxmox 8.1.3 and no dice now :(
 
For those of you still having issues, have a look at the logs when installing the built .deb. In my case the debian package wasn't installing because for whatever reason the kernel headers were missing.

To fix this:
Code: 
apt install pve-headers

Now everything is back to working as it was, thanks for the help guys :)
 
  • Like
Reactions: shbatm
Followed the steps above & pci coral shows up just fine in frigate. This doesn't survive a 2nd reboot however? Any suggestions?


Code: 
root@pve:~# modinfo gasket
filename:       /lib/modules/6.5.11-7-pve/updates/dkms/gasket.ko
author:         Rob Springer <rspringer@google.com>
license:        GPL v2
version:        1.1.4
description:    Google Gasket driver framework
import_ns:      DMA_BUF
srcversion:     2E9213B784051969509C03E
depends:       
retpoline:      Y
name:           gasket
vermagic:       6.5.11-7-pve SMP preempt mod_unload modversions
sig_id:         PKCS#7
signer:         DKMS module signing key
sig_key:        0D:52:BC:12:9F:BC:85:DA:7F:A8:C1:6A:0F:3D:BF:EE:50:D6:EB:75
sig_hashalgo:   sha512
signature:      39:32:68:AF:26:CA:D1:B5:25:97:B1:07:32:A6:FF:A3:5B:72:14:B6:
                AC:62:F9:DD:B2:28:55:14:0E:1B:22:C4:D0:6D:73:E7:B7:24:EB:3B:
                C7:00:23:A7:55:F9:53:9D:35:DD:E7:CA:86:C5:E6:EC:3E:F3:FA:9B:
                68:40:78:9F:FF:D7:35:7C:CA:E5:A7:85:A9:24:28:3F:93:76:E1:F8:
                D9:54:7F:FD:A4:B3:E6:58:FA:05:C7:EC:39:FD:C9:35:6E:D7:F4:E6:
                48:6F:E2:23:BD:79:50:E8:FB:C0:E7:F1:43:9A:99:70:D8:37:28:27:
                71:86:DE:56:35:41:DC:9E:78:9B:FA:60:E9:0F:D6:52:06:CB:BA:1C:
                4A:CC:EE:17:29:15:C3:19:4E:1A:26:72:98:2A:F1:4E:71:69:DE:0A:
                07:C9:15:78:09:FD:CC:6C:A2:4D:46:8B:76:66:64:A5:87:E6:D1:2B:
                79:60:2C:26:8B:22:9D:C6:77:48:8F:24:B4:87:F1:FC:0D:38:39:B3:
                04:27:1A:63:72:43:D4:23:A7:F8:54:51:85:08:35:A7:DB:2C:9D:78:
                72:2C:20:66:E4:ED:95:10:2A:CD:8F:53:B7:1B:FB:92:89:25:5E:DC:
                49:6F:88:F7:01:05:A5:B2:81:FB:B7:91:9D:1F:A8:47
parm:           dma_bit_mask:int
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back