Hi there @IAK, welcome on board!
Hmm, since you’ve seen this on both the Raspberry Pi and your laptop, it might be related to the runtime environment in Docker rather than the board firmware?
First quick thing to try: can you run this command inside your Docker container and share the output?
glxinfo | grep "OpenGL renderer"
This will help confirm whether OpenGL ES is available and functioning properly, which is required for inference display on the Pi5. If it’s working, missing or misconfigured, we can go from there 
Hi @Spanner
Thanks, glad to be here!
After running the glxinfo | grep "OpenGL renderer" command inside my docker I have the following output:
⬢ nDocker] ❯ glxinfo | grep "OpenGL renderer"
OpenGL renderer string: V3D 7.1
Guess it is configured fine, isn’t it?
Ah, that’s good news! Well, not good news that you’re having an issue, but it’s good that we’ve confirmed that OpenGL is working 
Thanks for the info.
Maybe we should check that the Metis card is being recognised correctly? If you could run the following command inside the container and share the output:
axdevice
That should give us some info about its status, firmware, etc.
Hello @IAK
May I ask which RPi HAT are you using?
I also recommend you to take a look at our troubleshooting guide in https://support.axelera.ai/hc/en-us/articles/26261515755538-PCIe-Troubleshooting-Guide
Regards,
Victor
Ah, that’s good news! Well, not good news that you’re having an issue, but it’s good that we’ve confirmed that OpenGL is working Thanks for the info.
Maybe we should check that the Metis card is being recognised correctly? If you could run the following command inside the container and share the output:
axdevice
That should five us some info about its status, firmware, etc.
Facing the same issue.
Device 0: metis-1:1:0 board_type=m2 fwver='v1.2.5' clock=800MHz(0-3:800MHz) mvm=0-3:100%
using Waveshare pcie to m.2 hat
https://thepihut.com/collections/waveshare/products/pcie-to-m-2-adapter-for-raspberry-pi-5
OUTPUT OF dmesg -T
Tue May 27 14:49:08 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:49:09 2025] axl 0001:01:00.0: DMA WR CH0 queue timeout (00000000f2589e77 status 0)
aTue May 27 14:49:09 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:49:10 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:49:11 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:49:12 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:49:13 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:49:42 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:49:43 2025] axl 0001:01:00.0: DMA WR CH0 queue timeout (00000000fdc5a1e2 status 0)
aTue May 27 14:49:43 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:49:44 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:49:45 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:49:46 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:49:47 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:55:52 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:55:53 2025] axl 0001:01:00.0: DMA WR CH0 queue timeout (00000000cf307059 status 0)
aTue May 27 14:55:53 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:55:54 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:55:55 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:55:56 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:55:57 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:56:19 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:56:20 2025] axl 0001:01:00.0: DMA WR CH0 queue timeout (00000000c592c395 status 0)
aTue May 27 14:56:20 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:56:21 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:56:22 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:56:23 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:56:24 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
rTue May 27 14:58:11 2025] axl 0001:01:00.0: Unregister directory 0001:01:00.0
0Tue May 27 14:58:11 2025] axl 0001:01:00.0: Unregistered triton-1:1:0 (0 0)
0Tue May 27 14:58:11 2025] axl 0001:01:00.0: Release dma mem triton-1:1:0
nTue May 27 14:58:11 2025] pci_bus 0001:01: busn_res: 1bus 01] is released
eTue May 27 14:58:12 2025] triton: debugfs root directory triton removed
eTue May 27 14:58:13 2025] pci 0001:00:00.0: 014e4:2712] type 01 class 0x060400 PCIe Root Port
oTue May 27 14:58:13 2025] pci 0001:00:00.0: PCI bridge to Pbus 01]
[Tue May 27 14:58:13 2025] pci 0001:00:00.0: bridge window mem 0x1b80000000-0x1b82ffffff]
8Tue May 27 14:58:13 2025] pci 0001:00:00.0: PME# supported from D0 D3hot
Tue May 27 14:58:13 2025] brcm-pcie 1000110000.pcie: clkreq-mode set to safe
tTue May 27 14:58:13 2025] brcm-pcie 1000110000.pcie: link up, 8.0 GT/s PCIe x1 (!SSC)
xTue May 27 14:58:13 2025] pci 0001:01:00.0: 1f9d:1100] type 00 class 0x120000 PCIe Endpoint
Tue May 27 14:58:13 2025] pci 0001:01:00.0: BAR 0 0mem 0x1b82010000-0x1b82010fff 64bit]
fTue May 27 14:58:13 2025] pci 0001:01:00.0: BAR 2 0mem 0x1b80000000-0x1b81ffffff]
8Tue May 27 14:58:13 2025] pci 0001:01:00.0: ROM 1mem 0x00000000-0x0000ffff pref]
fTue May 27 14:58:13 2025] pci 0001:01:00.0: supports D1
pTue May 27 14:58:13 2025] pci 0001:01:00.0: PME# supported from D0 D1 D3hot
Tue May 27 14:58:13 2025] pci 0001:01:00.0: 7.876 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x1 link at 0001:00:00.0 (capable of 31.504 Gb/s with 8.0 GT/s PCIe x4 link)
Tue May 27 14:58:13 2025] pci 0001:00:00.0: bridge window mem 0x1b80000000-0x1b82ffffff]: assigned
Tue May 27 14:58:13 2025] pci 0001:01:00.0: BAR 2 0mem 0x1b80000000-0x1b81ffffff]: assigned
Tue May 27 14:58:13 2025] pci 0001:01:00.0: ROM 1mem 0x1b82000000-0x1b8200ffff pref]: assigned
Tue May 27 14:58:13 2025] pci 0001:01:00.0: BAR 0 0mem 0x1b82010000-0x1b82010fff 64bit]: assigned
Tue May 27 14:58:13 2025] pci 0001:00:00.0: PCI bridge to bus 01]
Tue May 27 14:58:13 2025] pci 0001:00:00.0: bridge window mem 0x1b80000000-0x1b82ffffff]
bTue May 27 14:58:13 2025] pcieport 0001:00:00.0: PME: Signaling with IRQ 39
iTue May 27 14:58:13 2025] pcieport 0001:00:00.0: AER: enabled with IRQ 39
iTue May 27 14:58:13 2025] pci 0001:01:00.0: Found target device: TRITON_OMEGA_DEVICE_ID
_Tue May 27 14:58:13 2025] pci 0001:01:00.0: Found target device: 0001:01:00.0
0Tue May 27 14:58:13 2025] pcieport 0001:00:00.0: Found bridge device: 0001:00:00.0
0Tue May 27 14:58:13 2025] pcieport 0001:00:00.0: Bridge window: mem 0x1b80000000-0x1b82ffffff]
bTue May 27 14:58:13 2025] pcieport 0001:00:00.0: Decoded memory behind bridge: 1b80000000-1b82ffffff
1Tue May 27 14:58:13 2025] pcieport 0001:00:00.0: Memory behind bridge is sufficient. Skipping reset.
iTue May 27 14:58:13 2025] triton: root directory for triton
fTue May 27 14:58:13 2025] axl 0001:01:00.0: MSI registered 32 (32)
eTue May 27 14:58:13 2025] axl 0001:01:00.0: irq vec number 186
nTue May 27 14:58:13 2025] axl 0001:01:00.0: Register directory 0001:01:00.0
0Tue May 27 14:58:13 2025] Triton Linux Driver, version 0.07.16, init OK
6Tue May 27 15:02:24 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
Tue May 27 15:02:25 2025] axl 0001:01:00.0: DMA WR CH0 queue timeout (000000001768cf1f status 0)
Tue May 27 15:02:25 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
Tue May 27 15:02:26 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
Tue May 27 15:02:27 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
Tue May 27 15:02:28 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
Tue May 27 15:02:29 2025] axl 0001:01:00.0: DMA RD CH0 timeout (irq 4, ctrl 0x3)
Tue May 27 15:03:54 2025] axl 0001:01:00.0: Unregister directory 0001:01:00.0
0Tue May 27 15:03:54 2025] axl 0001:01:00.0: Unregistered triton-1:1:0 (0 0)
:Tue May 27 15:03:54 2025] axl 0001:01:00.0: Release dma mem triton-1:1:0
iTue May 27 15:03:54 2025] pci_bus 0001:01: busn_res: 1bus 01] is released
sTue May 27 15:03:55 2025] triton: debugfs root directory triton removed
nTue May 27 15:03:56 2025] pci 0001:00:00.0: c14e4:2712] type 01 class 0x060400 PCIe Root Port
eTue May 27 15:03:56 2025] pci 0001:00:00.0: PCI bridge to 0bus 01]
tTue May 27 15:03:56 2025] pci 0001:00:00.0: bridge window mem 0x1b80000000-0x1b82ffffff]
xTue May 27 15:03:56 2025] pci 0001:00:00.0: PME# supported from D0 D3hot
rTue May 27 15:03:56 2025] brcm-pcie 1000110000.pcie: clkreq-mode set to safe
Tue May 27 15:03:56 2025] brcm-pcie 1000110000.pcie: link up, 8.0 GT/s PCIe x1 (!SSC)
ITue May 27 15:03:56 2025] pci 0001:01:00.0: p1f9d:1100] type 00 class 0x120000 PCIe Endpoint
CTue May 27 15:03:56 2025] pci 0001:01:00.0: BAR 0 0mem 0x1b82010000-0x1b82010fff 64bit]
1Tue May 27 15:03:56 2025] pci 0001:01:00.0: BAR 2 0mem 0x1b80000000-0x1b81ffffff]
xTue May 27 15:03:56 2025] pci 0001:01:00.0: ROM 0mem 0x00000000-0x0000ffff pref]
0Tue May 27 15:03:56 2025] pci 0001:01:00.0: supports D1
Tue May 27 15:03:56 2025] pci 0001:01:00.0: PME# supported from D0 D1 D3hot
Tue May 27 15:03:56 2025] pci 0001:01:00.0: 7.876 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x1 link at 0001:00:00.0 (capable of 31.504 Gb/s with 8.0 GT/s PCIe x4 link)
CTue May 27 15:03:56 2025] pci 0001:00:00.0: bridge window .mem 0x1b80000000-0x1b82ffffff]: assigned
fTue May 27 15:03:56 2025] pci 0001:01:00.0: BAR 2 0mem 0x1b80000000-0x1b81ffffff]: assigned
fTue May 27 15:03:56 2025] pci 0001:01:00.0: ROM 0mem 0x1b82000000-0x1b8200ffff pref]: assigned
fTue May 27 15:03:56 2025] pci 0001:01:00.0: BAR 0 0mem 0x1b82010000-0x1b82010fff 64bit]: assigned
tTue May 27 15:03:56 2025] pci 0001:00:00.0: PCI bridge to .bus 01]
Tue May 27 15:03:56 2025] pci 0001:00:00.0: bridge window :mem 0x1b80000000-0x1b82ffffff]
0Tue May 27 15:03:56 2025] pcieport 0001:00:00.0: PME: Signaling with IRQ 39
gTue May 27 15:03:56 2025] pcieport 0001:00:00.0: AER: enabled with IRQ 39
dTue May 27 15:03:56 2025] pci 0001:01:00.0: Found target device: TRITON_OMEGA_DEVICE_ID
ETue May 27 15:03:56 2025] pci 0001:01:00.0: Found target device: 0001:01:00.0
Tue May 27 15:03:56 2025] pcieport 0001:00:00.0: Found bridge device: 0001:00:00.0
Tue May 27 15:03:56 2025] pcieport 0001:00:00.0: Bridge window: .mem 0x1b80000000-0x1b82ffffff]
0Tue May 27 15:03:56 2025] pcieport 0001:00:00.0: Decoded memory behind bridge: 1b80000000-1b82ffffff
0Tue May 27 15:03:56 2025] pcieport 0001:00:00.0: Memory behind bridge is sufficient. Skipping reset.
iTue May 27 15:03:56 2025] triton: root directory for triton
rTue May 27 15:03:56 2025] axl 0001:01:00.0: MSI registered 32 (32)
tTue May 27 15:03:56 2025] axl 0001:01:00.0: irq vec number 186
eTue May 27 15:03:56 2025] axl 0001:01:00.0: Register directory 0001:01:00.0
Tue May 27 15:03:56 2025] Triton Linux Driver, version 0.07.16, init OK
Hello @dev.manek ,
Did you follow the guide in https://www.waveshare.com/wiki/PCIe_TO_M.2_HAT+ ?
Specifically this is very important:
1: Enable PCIe Interface:
The PCIe interface is not enabled by default on PI5B. To enable it, add the following configuration in /boot/firmware/config.txt:dtparam=pciex1
2: PCIE is gen2 by default, if you need to enable PCIE gen3, add it in /boot/firmware/config.txt:
dtparam=pciex1_gen=3
Please enable PCIE gen3 as described in step 2.
At the moment, we have tested Raspberry Pi 5 official M.2 HAT and Seeed Studio Dual M.2 HAT, both working successfully. We know of more users succesfully testing other HATs.
Hello @dev.manek ,
Did you follow the guide in https://www.waveshare.com/wiki/PCIe_TO_M.2_HAT+ ?
Specifically this is very important:
1: Enable PCIe Interface:
The PCIe interface is not enabled by default on PI5B. To enable it, add the following configuration in /boot/firmware/config.txt:dtparam=pciex1
2: PCIE is gen2 by default, if you need to enable PCIE gen3, add it in /boot/firmware/config.txt:
dtparam=pciex1_gen=3
At the moment, we have tested Raspberry Pi 5 official M.2 HAT and Seeed Studio Dual M.2 HAT, both working successfully. We know of more users succesfully testing other HATs.
Thank you for the reply I will try it with the seed hat and update you
Is your RPi5 set to PCIe Gen3, @dev.manek?
dtparam=pciex1_gen=3
Is your RPi5 set to PCIe Gen3, @dev.manek?
dtparam=pciex1_gen=3
Yes I did. I can confirm that it works with seed dual m.2 hat. But still no stream.
WARNING : Failed to get OpenCL platforms : clGetPlatformIDs failed: PLATFORM_NOT_FOUND_KHR
EDIT:- I am dumb I had no display on. Everything works. Thank you so much guys.
Is your RPi5 set to PCIe Gen3, @dev.manek?
dtparam=pciex1_gen=3
Yes I did. I can confirm that it works with seed dual m.2 hat. But still no stream.
WARNING : Failed to get OpenCL platforms : clGetPlatformIDs failed: PLATFORM_NOT_FOUND_KHR
EDIT:- I am dumb I had no display on. Everything works. Thank you so much guys.
Ah nice work! It’s easy to overlook these things Just glad it’s up and running!
What’s the project you’re building with it @dev.manek?
Hello everyone!
I’m still having this issue. I’ve tried to solve it with this guide: https://support.axelera.ai/hc/en-us/articles/26362016484114-Bring-up-Voyager-SDK-in-Raspberry-Pi-5 and some research but the result is still unsatisfying, this is the output of the inference trials:
(venv) root@iak:/voyager-sdk# AXELERA_OPENGL_BACKEND=gles,3,1 ./inference.py yolov8s-coco-onnx ./media/traffic1_480p.mp4
INFO : Using device metis-1:1:0
WARNING : Failed to get OpenCL platforms : clGetPlatformIDs failed: PLATFORM_NOT_FOUND_KHR
WARNING : Please check the documentation for installation instructions
INFO : Default OpenGL backend gl,3,3 overridden, using gles,3,1
INFO : Network type: NetworkType.SINGLE_MODEL
INFO : Input
INFO : └─detections
Stream Paused: 57%|████████████████████████████████████████ | 4/7 00:00<00:00, 106.96/s]ERROR]axeWaitForCommandList]: Uio wait kernel failed with return code -1600.
ERROR][axeCommandQueueExecuteCommandListsAsync]: Waiting for command lists failed: 0x70010001.
terminate called after throwing an instance of 'std::runtime_error'
what(): axr_run_model failed with Error at zeCommandQueueExecuteCommandLists(cmdqueue, n_cmdlists, cmdlists, nullptr): cmdqueue_run_cmdlists: 309: Exit with error code: 0x70010001 : ZE_RESULT_ERROR_NOT_AVAILABLE
Aborted (core dumped)
So my guess is that I cannot access the OpenCL platform which is needed for inference.
However when I run the OpenCL demo with gears (the glxgears command from the guide), it runs just fine.
also when I run clinfo command, it shows that I have 0 platforms:
(venv) root@iak:/voyager-sdk# clinfo
Number of platforms 0
Any ideas on the solution directions from here?
Best wishes,
Ivan
We’ll check this with @Victor Labian, but I don’t think OpenCL is support on the Pi5, in the Voyager SDK context. I think there’s been some experimentation around enabling it, but it’s not straight forward (or officially supported) yet - I could be wrong though.
During some quick research, I did see that --pipe=torch-aipu has been used to skip OpenCL entirely - might be worth a shot in the meantime? I’ve not tried this (?!) so forgive me if it wastes any time (I don’t have a spare Pi5 to test), but I’m guessing that’d be…
./inference.py yolov8s-coco-onnx ./media/traffic1_480p.mp4 --pipe=torch-aipu
Presumably slower, but might work at least, in the meantime. Let me know how it goes, and we’ll get our Raspberry Ninja (that’s Victor 
) to check in too
Hello @Spanner @IAK ,
OpenCL is not officially supported on Raspberry Pi 5. We are exploring possibilities of how to enable OpenCL in RPi 5 and then be able to work on Voyager SDK with it on RPi5 but that is something that we cannot disclosure more about at the moment.
However, OpenCL is not the issue, it is only a warning, as inference should also be able to run without OpenCL.
As I mentioned before to @dev.manek we have tested Raspberry Pi 5 official M.2 HAT and Seeed Studio Dual M.2 HAT, both working successfully. For other HATs you might need to read the documentation for the HAT and/or contact the manufacturer. I suggest that you test any of the two HATs we have confirmed that work well.
@IAK :
- which HAT are you using?
- Can you run inside the (venv) the command
axdevice --refresh and share the output with us?
Can you also share the output of running the following:
uname -rcat /sys/class/metis/versionlsmod | grep metis
Best,
Victor
Hiya,
I’m on the same team as @IAK. We’re running it on the official RPI M.2 HAT+. The result from the commands is:
(venv) root@iak:/voyager-sdk# axdevice --refresh
0001:01:00.0 : Axelera AI Metis AIPU (rev 02)
Device 0: metis-1:1:0 board_type=m2 fwver='v1.2.5' clock=800MHz(0-3:800MHz) mvm=0-3:100%
(venv) root@iak:/voyager-sdk# uname -r
6.12.25+rpt-rpi-2712
(venv) root@iak:/voyager-sdk# cat /sys/class/metis/version
0.07.16
(venv) root@iak:/voyager-sdk# lsmod | grep metis
metis 65536 0
When I run it with the pipe=torch-aipu (with -v as well), I get the following results:
The first time I ran it, I got this. It seems to me it managed some detections but then got stuck somewhere? Or that it was querying from None. It did get further than without the torch-aipu, as a little window with the Axelera logo opened up which hadn’t happened before.
(venv) root@iak:/voyager-sdk# ./inference.py yolov8s-coco-onnx ./media/traffic1_480p.mp4 --pipe=torch-aipu -v
DEBUG :axelera.app.inf_tracers: HostTracer will write to /tmp/tmp36tq5ajq
INFO :axelera.app.device_manager: Using device metis-1:1:0
DEBUG :axelera.app.network: Create network from /voyager-sdk/ax_models/zoo/yolo/object_detection/yolov8s-coco-onnx.yaml
DEBUG :axelera.app.network: Register custom operator 'decodeyolo' with class DecodeYolo from /voyager-sdk/ax_models/decoders/yolo.py
DEBUG :axelera.app.device_manager: Reconfiguring devices with ddr_size=1.0GB, device_firmware=1, mvm_utilisation_core_0=100%, clock_profile_core_0=800MHz
DEBUG :axelera.app.inf_tracers: Host FPS multiplier: 1
WARNING :axelera.app.utils: Failed to get OpenCL platforms : clGetPlatformIDs failed: PLATFORM_NOT_FOUND_KHR
WARNING :axelera.app.utils: Please check the documentation for installation instructions
WARNING :axelera.app.utils: pyglet could not access the display, OpenGL is not available: Could not create GL context
INFO :axelera.app.pipe.manager: Network type: NetworkType.SINGLE_MODEL
INFO :axelera.app.pipe.manager: Input
INFO :axelera.app.pipe.manager: └─detections
DEBUG :yolo: Model Type: YoloFamily.YOLOv8 (YOLOv8 pattern:
DEBUG :yolo: - 6 output tensors (anchor-free)
DEBUG :yolo: - 3 regression branches (64 channels)
DEBUG :yolo: - 3 classification branches (80 channels)
DEBUG :yolo: - Channel pattern: e64, 64, 64, 80, 80, 80]
DEBUG :yolo: - Shapes: : 1, 80, 80, 64], 01, 40, 40, 64], 01, 20, 20, 64], 01, 80, 80, 80], 01, 40, 40, 80], 01, 20, 20, 80]])
DEBUG :axelera.app.pipe.io: FPS of /root/.cache/axelera/media/traffic1_480p.mp4: 60
INFO :axelera.app.torch_utils: Using CPU based torch
DEBUG :axelera.app.pipe.io: Create image generator from VideoCapture
DEBUG :axelera.app.operators.inference: Loaded model : /voyager-sdk/build/yolov8s-coco-onnx/yolov8s-coco-onnx/1/model.json
DEBUG :axelera.app.operators.inference: Available ONNX runtime providers: 'AzureExecutionProvider', 'CPUExecutionProvider']
WARNING :axelera.app.stream: Timeout for querying an inference
DEBUG :axelera.app.operators.inference: Expected core model output shape: tp1, 64, 80, 80], 41, 64, 40, 40], 41, 64, 20, 20], 41, 80, 80, 80], 01, 80, 40, 40], 01, 80, 20, 20]]
ERROR :axelera.app.display: Exception in inference thread: InferenceThread
ERROR :axelera.app.display: Traceback (most recent call last):
ERROR :axelera.app.display: File "/voyager-sdk/axelera/app/display.py", line 270, in _target
ERROR :axelera.app.display: target(*args, **kwargs)
ERROR :axelera.app.display: File "/voyager-sdk/./inference.py", line 48, in inference_loop
ERROR :axelera.app.display: for frame_result in tqdm(
ERROR :axelera.app.display: File "/root/.cache/axelera/venvs/fd8d4664/lib/python3.10/site-packages/tqdm/std.py", line 1181, in __iter__
ERROR :axelera.app.display: for obj in iterable:
ERROR :axelera.app.display: File "/voyager-sdk/axelera/app/stream.py", line 175, in __iter__
ERROR :axelera.app.display: raise RuntimeError('timeout for querying an inference') from None
ERROR :axelera.app.display: RuntimeError: timeout for querying an inference
DEBUG :axelera.app.meta.object_detection: Total number of detections: 6
The second time I ran it I got the following:
(venv) root@iak:/voyager-sdk# ./inference.py yolov8s-coco-onnx ./media/traffic1_480p.mp4 --pipe=torch-aipu -v
DEBUG :axelera.app.inf_tracers: HostTracer will write to /tmp/tmpp_o2jqie
INFO :axelera.app.device_manager: Using device metis-1:1:0
DEBUG :axelera.app.network: Create network from /voyager-sdk/ax_models/zoo/yolo/object_detection/yolov8s-coco-onnx.yaml
DEBUG :axelera.app.network: Register custom operator 'decodeyolo' with class DecodeYolo from /voyager-sdk/ax_models/decoders/yolo.py
DEBUG :axelera.app.device_manager: Reconfiguring devices with ddr_size=1.0GB, device_firmware=1, mvm_utilisation_core_0=100%, clock_profile_core_0=800MHz
DEBUG :axelera.app.inf_tracers: Host FPS multiplier: 1
WARNING :axelera.app.utils: Failed to get OpenCL platforms : clGetPlatformIDs failed: PLATFORM_NOT_FOUND_KHR
WARNING :axelera.app.utils: Please check the documentation for installation instructions
WARNING :axelera.app.utils: pyglet could not access the display, OpenGL is not available: Could not create GL context
INFO :axelera.app.pipe.manager: Network type: NetworkType.SINGLE_MODEL
INFO :axelera.app.pipe.manager: Input
INFO :axelera.app.pipe.manager: └─detections
DEBUG :yolo: Model Type: YoloFamily.YOLOv8 (YOLOv8 pattern:
DEBUG :yolo: - 6 output tensors (anchor-free)
DEBUG :yolo: - 3 regression branches (64 channels)
DEBUG :yolo: - 3 classification branches (80 channels)
DEBUG :yolo: - Channel pattern: h64, 64, 64, 80, 80, 80]
DEBUG :yolo: - Shapes: yo1, 80, 80, 64], 11, 40, 40, 64], 11, 20, 20, 64], 11, 80, 80, 80], 11, 40, 40, 80], 11, 20, 20, 80]])
DEBUG :axelera.app.pipe.io: FPS of /root/.cache/axelera/media/traffic1_480p.mp4: 60
INFO :axelera.app.torch_utils: Using CPU based torch
DEBUG :axelera.app.pipe.io: Create image generator from VideoCapture
DEBUG :axelera.app.operators.inference: Loaded model : /voyager-sdk/build/yolov8s-coco-onnx/yolov8s-coco-onnx/1/model.json
DEBUG :axelera.app.operators.inference: Available ONNX runtime providers: t'AzureExecutionProvider', 'CPUExecutionProvider']
Detecting... : 0%| | 0/13363 00:02<?, ?frames/s]lERROR]faxeLoadAndExecuteQueueElf]: Uio wait kernel failed with return code -1406.
oERROR]0axeRunQueueCSourceCommandLists]: Load and execute queue elf failed: 0x70010001.
xERROR]0axeCommandQueueExecuteCommandListsSync]: Run command lists failed: 0x70010001.
ERROR :axelera.runtime: Error at zeCommandQueueExecuteCommandLists(cmdqueue, n_cmdlists, cmdlists, nullptr): cmdqueue_run_cmdlists: 309: Exit with error code: 0x70010001 : ZE_RESULT_ERROR_NOT_AVAILABLE
ERROR :axelera.app.pipe.torch: TorchPipe terminated due to RuntimeError at /root/.cache/axelera/venvs/fd8d4664/lib/python3.10/site-packages/axelera/runtime/objects.py:74: AXR_ERROR_RUNTIME_ERROR: Error at zeCommandQueueExecuteCommandLists(cmdqueue, n_cmdlists, cmdlists, nullptr): cmdqueue_run_cmdlists: 309: Exit with error code: 0x70010001 : ZE_RESULT_ERROR_NOT_AVAILABLE
ERROR :axelera.app.pipe.torch: Full traceback:
ERROR :axelera.app.pipe.torch: File "/voyager-sdk/axelera/app/pipe/torch.py", line 55, in _loop
ERROR :axelera.app.pipe.torch: image, result, meta = model_pipe.inference.exec_torch(
ERROR :axelera.app.pipe.torch: File "/voyager-sdk/axelera/app/operators/inference.py", line 750, in exec_torch
ERROR :axelera.app.pipe.torch: self._axr_modeli.run(inputs, outputs)
ERROR :axelera.app.pipe.torch: File "/root/.cache/axelera/venvs/fd8d4664/lib/python3.10/site-packages/axelera/runtime/objects.py", line 510, in run
ERROR :axelera.app.pipe.torch: _raise_error(self.context, res.value)
ERROR :axelera.app.pipe.torch: File "/root/.cache/axelera/venvs/fd8d4664/lib/python3.10/site-packages/axelera/runtime/objects.py", line 74, in _raise_error
ERROR :axelera.app.pipe.torch: raise exc(f"{err}: {msg}")
INFO : Core Temp : 39.0°C
INFO : prep : 37.9us
INFO : Memcpy-in : 562.5us
INFO : Kernel : 34663.2us
INFO : Memcpy-out : 459.3us
INFO : Host : 28.0fps
INFO : CPU % : 15.5%
INFO : End-to-end : 0.0fps
When running it with --pipe=torch it does work, but seems to be running on CPU with ~1 FPS.
Thanks for running those check @GeertL! This is good stuff.
So it looks like all system-level and SDK checks pass. The board is visible, the firmware and driver are correctly installed, and the Pi environment looks to be configured appropriately. The inference error may then be related to runtime resource issues, subtle PCIe instability, or pipeline issues?
Let’s ask @Victor Labian if he has any thoughts on the most practical next step!
Hello @GeertL
Couple of points:
Also, if you do inference.py --help you can see what each command line options means. In this case:
--pipe {gst,torch,torch-aipu,quantized}
specify pipeline type:
- gst: C++ pipeline based on GStreamer (uses AIPU)
- torch-aipu: PyTorch pre/post-processing with model on AIPU
- torch: PyTorch pipeline in FP32 for accuracy baseline (uses ONNXRuntime for ONNX models)
The fact that you see low FPS is expected as:
- With torch-aipu you are running non-optimized PyTorch pre/post-processing, that is really demanding on the host, and the RPi5 doesn’t have a very powerful CPU.
- With torch you are running the whole pipeline in the CPU which is extremely slow.
You should use gst, which is the default and using optimised pre/post processing and Metis for inference.
Note that even Metis is very fast, the Raspberry Pi 5 will be the bottleneck as it does the pre/post processing.
FYI @Spanner
Best,
Victor
