Sharing my first project on Axelera Metis: end-to-end monocular 3D object detection running at 24.5 FPS on a single AIPU core, with a CPU host and no GPU anywhere in the pipeline.
What it does Takes a single camera image and predicts 3D bounding boxes for every car, pedestrian and cyclist in the scene position, dimensions and orientation , in real time. Built on MonoCon (CVPR 2022) with a DLA-34 backbone, evaluated on KITTI driving sequences.
How it is split The model had to be divided at a hard compiler boundary. AttnBatchNorm2d contains a ReduceMean op that Voyager cannot quantize, so the pipeline is:
- AIPU: backbone + neck + fused HeadConv1 (64 to 576 channels, single output tensor)
- CPU host: AttnBatchNorm2d + ReLU + 1x1 convs, implemented in hand-written C++
Performance
| Stage | Time |
|---|---|
| Preprocess + Quantize | 1.4 ms |
| AIPU inference | 31 ms |
| Dequantize | 6.5 ms |
| Head (C++) | 22 ms |
| Decode + NMS | 1 ms |
| Sequential total | 62 ms / 16 FPS |
| Pipelined throughput | 41 ms / 24.5 FPS |
Pipelined throughput uses a two-thread producer-consumer design: the AIPU processes frame N+1 while the CPU decodes frame N concurrently.
Key findings during deployment A few things that are not in the documentation and took real time to figure out:
-
per_tensor_histogram (default) silently clips HeadConv1 activations — true range is roughly -1350 to +1200, the histogram scheme covers only -250 to +250. Zero detections until switching to per_tensor_min_max.
-
Multi-output graphs get their outputs sorted alphabetically by the compiler. Discovered by comparing per-tensor statistics against a PyTorch reference. Fixed by fusing all nine head convs into one 64 to 576 conv with a single output tensor.
-
axrArgument.fd must be -1 for host-pointer mode. Setting it to 0 silently produces zero output with AXR_SUCCESS returned.
-
The first 2-3 inference calls return stale output regardless of input. Warm-up with dummy frames is required before trusting results.
-
ONNXRuntime was taking 55ms on a 24,000-parameter subgraph , not because of arithmetic but because of per-node dispatch overhead across ~150 ops. Replaced with a preallocated C++ implementation: 55ms to 22ms.
Repo Everything is open source: PyTorch model, ONNX export scripts,
Voyager compile script, C++ inference library with 3D box and bird's-eye view visualization.
https://github.com/sanket-pixel/monocon-metis
Happy to answer questions on any part of the deployment.
Bonus discussion point :
Running the backbone+neck+HeadConv1 graph alone via axrunmodel hits 90 FPS on four cores and 40 FPS on a single core. The compiled model in this project uses aipu_cores_used=1, resources_used=0.25, straightforward but not necessarily optimal. There is likely headroom in the compiler configuration that this project has not fully explored: tiling depth, DFS search constraints, IMC double buffer pipelining, and grouping IFDW tasks are all knobs that affect how efficiently the compiler maps the graph onto the MVM array. If anyone has found configurations that push closer to the axrunmodel ceiling on a similarly sized graph, would be very interested to hear what worked. Happy to discuss.
