Skip to main content
Question

Metis M.2 113m: AI-core trace (axtrace --atrace) produces empty files; per-layer/kernel profiling options?

  • Nov 19, 2026 4:50 AM UTC
  • 0 replies
  • 3 views

Hi, I just bought 113m and trying to profile to find the bottlenecks.

Following is my setting.

Are there ways to enable per layer level profiling?

 

Environment

  • Board: Axelera Embedded 113m (Metis M.2), PCIe 03:00.0
  • Host A: Radxa Dragon Q8B, Ubuntu 26.04, kernel 7.0.11-qcom (aarch64)
  • Host B: x86_64 Ubuntu 26.04, kernel 7.0.0-43-generic, Secure Boot enabled
  • Voyager SDK 1.9.0 (axelera-rt, axelera-devkit[all])
  • Firmware: flver=1.9.0, bcver=7.4 (host B); flver=1.8.0 flashed with runtime firmware v1.9.0 loaded at connect (host A)
  • Driver: metis-dkms 1.7.16 (Ubuntu 24 apt suite on Ubuntu 26.04)

What I’m trying to do
Get per-layer or per-kernel device timing for a compiled model (a 1×1 conv + ReLU, 64 channels, int8).

What I tried

  1. Pipeline OperatorProfiler / axzoo benchmark yolov8n-coco: works, but only per operator (AxRuntimeModel is one row, flagged DEVICE WAIT).
  2. axruntime instance with device_profiling=True and/or host_profiling=True: no device-side output found. Host trace (AXE_LZ_PERF_LOG_PATH, AXE_PROFILING_CONFIG=HOST_TEMPLATE) works and gives per-frame kernel time.
  3. axtrace --atrace 0 --atrace-target 0:datapath and 0:megakernel, running 200 frames: the output file stays at 0 bytes on both hosts, with and without the instance profiling flags. axtrace --debug shows only the device selection.
  4. axlogdevice --atrace 0 --atrace-target 0:inf: the device returns an error for CMD_TRACE_AICORE.
  5. axlogdevice --alog 0 --alog-level 0:inf:runtime: no output while frames run. axlogdevice --file fails with a bytes-like object is required, not 'str'.
  6. Level Zero tools API (zetGetMetricGroupProcAddrTable, zetGetTracerExpProcAddrTable, zetGetKernelProcAddrTable, zetGetCommandListProcAddrTable): calls return success, but every function pointer in those tables is NULL.

Questions

  • Is AI-core trace (axtrace --atrace) supported on Metis M.2 with firmware 1.9.0 and driver 1.7.16? If yes, what must be enabled for it to record (firmware build, property, or boot option)?
  • Does device_profiling=True have any effect on the device? If it’s supposed to produce per-kernel timings, where do they go?
  • Is there a supported way to get per-layer or per-kernel device timing on Metis M.2 with SDK 1.9? If not, is one planned?
  • Is the axlogdevice --file string/bytes error known?

I can share the probe programs and the logs if they help.