Project Idea
Lidar based perception is very common for autonomous driving applications. While it has become relatively easy to achieve high accuracy with large and complex CNN architectures, deploying on resource constrained edge AI devices presents a whole new realm of challenges.
Models have to be almost always converted to reduced precision FP16 or INT8 that often results in significant accuracy loss. Furthermore, several conversion tools frequently produce fragmented computation graphs with a significant portion of the model running on CPU, thereby increasing inference latency.
This project aims to experiment and illustrate the capabilities of the Axelera AI accelerator together with the Voyager Wingman AI pipeline generator to deploy a trained CNN for object detection using bird’s eye view (BEV) images of lidar point clouds.
The architecture is a custom CNN built specifically for fast inference using BEV inputs and has been validated and published in a peer-reviewed paper.
https://ieeexplore.ieee.org/abstract/document/9564490
https://arxiv.org/pdf/2104.10780
A simplified and updated version of this will be used.
The inference latency and accuracy will be benchmarked against FP32 model on other edge AI device such as Nvidia Orin and Nvidia Xavier AGX devices.
The small form factor of the Dell XE5 with a 16GB Axelera Edge 130p is perfect for installing in test vehicles for perception tasks.
Deliverables:
- A live demo (recorded and/or live) fully integrated with ROS to simulate real sensor data processing in autonomous vehicles
- wingman prompts
- github repo with complete source code and trained model
- benchmark data against other edge AI computers
- List of key observations and findings about the overall setup - accelerator, SDK, AI tools
Not covered:
Since this is a personal project, deployment in an actual test vehicle is not included. However, the ROS integrated pipeline makes it trivial to integrated in a real car.
About me
I have several years of experience in the automotive domain specializing in designing, training and deployment of edge AI models for perception on embedded targets.
Therefore I have worked with several tools and workflows before that follow the general pipeline of train - export - quantize - compile - deploy.
A sample of my work in this domain can be found at:
https://www.hackster.io/sm_1991/hardware-accelerated-real-time-perception-in-3d-harp-3d-fde0c0
In this project I deployed a trained model on an entry level FPGA and won the 2nd prize in the segment.
However, using an AI tool as a wingman to create the deployment pipeline is new to me and is therefore very exciting. It greatly simplifies the process of creating deployment pipeline which is often quite cumbersome, under-documented and difficult to get exactly correct.
Since I already have a custom made CNN that is validated on embedded devices, I do not expect any significant obstructions to the project completion.
Therefore, I believe, if selected and granted a development kit, I can produce a useful, honest set of artifacts demonstrating both the strengths and the friction points of the Axelera Metis accelerator and the Voyager Wingman assistant.
Looking forward to some exciting weeks of building with the Axelera Metis accelerator and the Wingman AI assistant!
