Share your thoughts, suggest ideas, provide feedback, and help shape the future of Axelera
Sharing my first project on Axelera Metis: end-to-end monocular 3D object detection running at 24.5 FPS on a single AIPU core, with a CPU host and no GPU anywhere in the pipeline.What it does Takes a single camera image and predicts 3D bounding boxes for every car, pedestrian and cyclist in the scene position, dimensions and orientation , in real time. Built on MonoCon (CVPR 2022) with a DLA-34 backbone, evaluated on KITTI driving sequences.How it is split The model had to be divided at a hard compiler boundary. AttnBatchNorm2d contains a ReduceMean op that Voyager cannot quantize, so the pipeline is:AIPU: backbone + neck + fused HeadConv1 (64 to 576 channels, single output tensor) CPU host: AttnBatchNorm2d + ReLU + 1x1 convs, implemented in hand-written C++ PerformanceStage Time Preprocess + Quantize 1.4 ms AIPU inference 31 ms Dequantize 6.5 ms Head (C++) 22 ms Decode + NMS 1 ms Sequential total 62 ms / 16 FPS Pipelined throughput 41 ms / 24.5 FPS Pipelined throughput uses a two-thread producer-consumer design: the AIPU processes frame N+1 while the CPU decodes frame N concurrently.Key findings during deployment A few things that are not in the documentation and took real time to figure out: per_tensor_histogram (default) silently clips HeadConv1 activations — true range is roughly -1350 to +1200, the histogram scheme covers only -250 to +250. Zero detections until switching to per_tensor_min_max. Multi-output graphs get their outputs sorted alphabetically by the compiler. Discovered by comparing per-tensor statistics against a PyTorch reference. Fixed by fusing all nine head convs into one 64 to 576 conv with a single output tensor. axrArgument.fd must be -1 for host-pointer mode. Setting it to 0 silently produces zero output with AXR_SUCCESS returned. The first 2-3 inference calls return stale output regardless of input. Warm-up with dummy frames is required before trusting results. ONNXRuntime was taking 55ms on a 24,000-parameter subgraph , not because of arithmetic but because of per-node dispatch overhead across ~150 ops. Replaced with a preallocated C++ implementation: 55ms to 22ms. Repo Everything is open source: PyTorch model, ONNX export scripts,Voyager compile script, C++ inference library with 3D box and bird's-eye view visualization.https://github.com/sanket-pixel/monocon-metisHappy to answer questions on any part of the deployment.Bonus discussion point :Running the backbone+neck+HeadConv1 graph alone via axrunmodel hits 90 FPS on four cores and 40 FPS on a single core. The compiled model in this project uses aipu_cores_used=1, resources_used=0.25, straightforward but not necessarily optimal. There is likely headroom in the compiler configuration that this project has not fully explored: tiling depth, DFS search constraints, IMC double buffer pipelining, and grouping IFDW tasks are all knobs that affect how efficiently the compiler maps the graph onto the MVM array. If anyone has found configurations that push closer to the axrunmodel ceiling on a similarly sized graph, would be very interested to hear what worked. Happy to discuss.
Hello,As per original POST I’d like to request compatibility for IBM’s Granite modelsEmbedding and LLM ModelThank you,Peter
The examples that come with the SDK mostly use pipelines that do all the work of getting the data from the source (files or usb camera), sending for inference, and then displaying the inference results on the images. This is great for many use cases, I am sure, and enable a high degree of efficiency. However, for many users, the AI inference is part of a C/C++/python application where the images are obtained via OpenCV, processed (scale/crop/etc) in opencv based on application logic, and then a part of the image is sent for AI detection. Once the result is obtained, then the application can display the results over the opencv image using it’s own custom logic.So in summary, it would be great to have some examples that demonstrate this process. A simple example that shows how to use a detection model from the model zoo in C or Python along with opencv for obtaining image and displaying the results. Another good example would be running the model inference as a gstreamer pipeline. This would allow developers to feed the images into it, and get the results out using tools they are already familiar with (opencv, gstreamer, etc). p.s I am very new to Axlera/Metis so I apologize if I missed those examples :)
Hi all! I wanted to contribute a little bit about my experience bringing up the Metis M.2 in my LattePanda Sigma. Im an EE who is self taught, I have worked for Harley Davidson, Span.io, Enel X NA, and I’ve done several consulting projects on synthesizers and drum machines. My interest in AI is hobby and curiosity based, not professional. Software engineering is a new skill Im creating.I first tried to run Metis on the Windows 11 operating system that came with LattePanda Sigma, and following the instructions on the git was extremely confusing. I have ADD, and some other learning differences, but that wasn’t the reason installing was so hard on Windows. It’s because there isn’t a single page with coherent instructions telling the user how to bring up the hardware in a simple step by step format. Having the user click between git pages on firmware, installation of drivers, wsl, putting Windows into test mode, using multiple programming environments, etc is painful and makes installing the hardware on Windows a miserable, confusing, and difficult experience.I’ll probably make my own step by step guide for Windows at some point since the instructions on git are confusing.I then tried to do the install on an old version of Ubuntu (support 24.04 please!) and I had much less trouble. I HIGHLY RECOMMEND using Ubuntu 22.04 to run the hardware and software over Windows. The instructions to install the Voyager-SDK on Ubuntu 22.04 actually worked pretty well. I was surprised.I ran into an issue at one point after installing the SDK: I couldn’t detect the Metis M.2 card. I couldn’t really figure out how to install the driver from the instructions for Ubuntu installation, so I had to download the .deb driver and install it similar to the “instructions for installing Voyager-SDK using Docker” to get the driver installed. Once the driver is installed, and the clunky SDK is installed and running, the demos with YOLO worked great, with the Metis M.2 card barely breaking a sweat doing inferences.
I'm developing a new way of video with or without AI. I am marketing both small & large companies using their people as limited actors. I saw Titanium and i was very curious, can I merge dynamic ai actors into video with real actors? I am getting paperwork together to start, so I have time to learn. My background is in 2 & 3D animation & graphic design (degrees in both bs & bfa) What is the learning curve?
Looks like tags are a thing on posts and questions, and there’s a tag cloud in the sidebar. I can select existing tags but can’t create new ones. If community members could add their own tags it could be a good way to categorize content going forward.
Already have an account? Login
No account yet? Create an account
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.