It’s nice to be back for another interesting Axelera competition!
My one-minute pitch:
Advanced driving safety should not be limited to expensive new cars. MetisCopilot 360° will use four USB3 cameras and a single Metis PCIe card to provide 7 practical L0–L2 driver-assistance features, including lane, collision, blind-spot, and tailgating warnings at a fraction of the cost of comparable solutions. This is a project I have wanted to build for some time, and I believe Metis and Voyager Wingman provide the right tools to make it happen. I will use my prior Axelera HW/SW experience supercharged with the capabilities of Voyager Wingman to help build, integrate, and optimize the required multi-model pipelines rapidly, turning this idea into a working in-vehicle demonstration within one month.
Business case:
There is a strong business-case for a capable driver-assistance copilot aimed at lower and mid-range cars. Everyone deserves access to better safety, including those who can't afford high-end vehicles equipped with expensive (5k$-10k) autopilot packages.
Axelera hardware is unique in that if you compare cost and performance, it gives an order of magnitude advantage compared to other AI hardware. The multiple cores enable running multiple models simultaneously which is exactly what you need for ADAS. This makes it ideal for building a much lower-cost ADAS copilot while maintaining high levels of performance needed for ADAS.
I plan to demonstrate this advantage in a working, in-vehicle system through my proposed project.
Main features:
I have selected a set of useful features which in my assessment are feasible to build within a month, largely using models from the zoo along with some custom logic and CV functions.
| Feature | Main input | Model to use | Remarks/notes |
| Lane departure warning | Front camera, simulated turn lever input | yolo26nseg-coco-onnx/ yolo11nseg-coco-onnx (instance seg) – see note | Use segmentation masks to detect lane lines & drivable area. Turn lever = logic filter. |
| Lane keep assist | Front camera, simulated turn lever input | Same as above |
|
| Forward collision warning | Front Camera | yolo11n-coco-onnx | Distance estimation via bbox height + calib. |
| Intelligent Cruise control | Front camera, simulated speed input | yolo11n-coco-onnx + custom distance head | Maintain >3s distance from front car |
| Emergency braking | Front camera, simulated speed input | Same detector (yolo11n-coco-onnx) |
|
| Tailgater warning | Rear camera, simulated speed input | yolo11n-coco-onnx (rear-camera) | Speed input will help differentiate actual tailgaters from slow moving traffic/jam situation. |
| Blind spot Monitor/warning | Left & Right side cameras, simulated turn lever input | yolo11n-coco-onnx (2 x side cameras) | Turn lever = directional alert logic. |
A few notes on models selection: For Lane detection, some dedicated models exist (Ultra-Fast, CLRNet, LaneAF) which I’ll investigate compiling/porting on Metis, otherwise I will use segmentation model as fallback. I do plan to try some monocular depth estimation models as well for distance estimation, otherwise bounding box sizes will be used for distance estimation/approach speed estimates.
I have also a few “Stretch Goals”, if time (and Metis core utilization) permits, in order of preference:
- Build and run a lightweight VSLAM algorithm live, based on “Superpoints” model I ported earlier. Idea is to build and store a map of the surroundings as I drive.
- Log license plates of tailgaters --> This will require LPRNet.
- Log data from system while driving, run local LLM at the end of the drive (when Metis cores are free) to derive interesting insights from the data about the drive. For example, license plate numbers in my country indicate registration year and state. The LLM can analyze and present things like distribution of ages of cars encountered on road, distribution of registration state, etc. Like “stats for nerds” in YouTube :)
Important note on project scope: For this project, I will focus on implementing the stated ADAS functionality, however, the system will operate in “open loop”. Meaning all the system outputs (warnings /commands) will be visual/audible driver advisories only and no commands will be sent directly to the car via OBD. Simply, it’s safer and avoids any regulatory/liability issues. Also, integrating with OBD is a time consuming task. Lastly, all camera data will be acquired during daytime because my cameras have limited performance in low light.
Demo plan and target metrics:
A dash-mounted screen will show the four live camera feeds with clear ADAS overlays, status indicators. Audible alerts via speaker. The display will also show real-time performance statistics such as FPS, latency, and HW utilization (axmonitor).
However, note that I will build the demo such that it can run completely offline using previously acquired datasets (that’s how I plan to develop&test anyway!)
Target metrics: >30 FPS per camera, <150 ms latency for every function.
High-level Implementation plan:
I have previously built for Axelera competition under 1 month build constraint. That, and based on my prior experience with Axelera hardware and wingman, my plan is as following:
- Where possible, I will use existing models from model zoo on priority to save time. I have identified the ones to begin with.
- If an existing model is unavailable/not suitable for a particular task, I will attempt to port a compatible model (I have some experience with this on Metis).
- I will acquire a dataset very early (week 1) with my own hardware and use it for development/testing. I already have all the hardware.
- I will add simulated data for inputs I would normally take from OBD-II. This will save time and let me focus on core functionality.
- I will build a general architecture first, then add each feature as a module
Hardware:
- 12V lighter port to 220V inverter
- Dell XE5 with Metis PCIE card (provided by Axelera)
- 4 x USB3 cameras (Front, rear, left, right) with appropriate C mount lenses for appropriate FOVs.
- 4 x suction cup adjustable mounts for mounting cameras inside car facing each direction.
- LCD screen for visual output and warning
- Audio speaker for audible warnings.
NOTE: I already possess all the hardware above, including good quality USB3 cameras with C mount lenses in required quantities.
Why I am the right person to build this:
- I have been working with Axelera hardware/SW for more than a year and have a fair understanding of things in the domain.
- I participated in Axelera’s “Pioneer 10” challenge and my project submission won 2nd place in the competition.
- I have been an early beta-tester of Axelera wingman. In the process, I managed to port two unique new models to Metis, different than the typical model categories in the zoo. (This experience will be very valuable during the competition.)
- One is “Superpoints”, a model that detects key points and creates math descriptions for images in one single step. These can be used by downstream algorithms (VSLAM, tracking etc). Github repo coming soon.
- Second is SiamFC, a model for visual ROI tracking without prior knowledge of object type etc (It can track anything visually distinct). I have previously posted about this. Also made available on github for everyone:
https://github.com/SaadTiwana/siamfc-metis
- I recently noticed that I currently have the highest total points on the Axelera Community leaderboard among non-Axelera employees. I like to think that counts for something, hopefully reflecting how actively I have engaged with the community, shared my work, and continued learning from others.
