Share your thoughts, suggest ideas, provide feedback, and help shape the future of Axelera
Hey Axelera team and community! 👋I saw the Prompt Challenge ad on my phone screen back on July 23, 2026, and it got me really excited enough to dig back into an old, paused concept I'd been sitting on BN-Twin and finally turn it into a working software demo.Now, I want to take this working engine and supercharge it on real Axelera hardware using Voyager Wingman!💡 What BN-Twin DoesBN-Twin is a single deterministic engine designed for two Industry 5.0 needs: Trustworthy Cobot Investment Decisions: Running transparent, non-black-box simulations comparing manual, automatic, and collaborative production lines. High-Precision Robotic Control: Controlling robotic arms with 50-digit precision kinematics while while damping motion instabilities at singularities instead of letting control corrections spike.Instead of just logging errors, it calculates the real-time joint-angle correction needed to close the loop, the actual control math, ready to be wired to real actuators once hardware integration begins. 🖼️ Current Working UI & Features (Screenshots attached) Robotic Arm Console: Live kinematics visualization, 50-digit precision (mpmath), singularity damping status (SAFE), and live joint correction signals ($\Delta\theta$). Production Line Dashboard: Multi-criteria decision engine comparing Manual vs. Full Auto vs. Collaborative 5.0 (analyzing throughput, OEE, cost per unit, and ergonomics). Cryptographic Proof: Every run generates a green TWIN SYNC HASH at the bottom to guarantee 100% reproducible results, paired with Fernet+PBKDF2 encryption for IP security. 🛠️ The Mission with Voyager Wingman & MetisSince the core software logic is already running, my goal during the build month is to port and accelerate it on Axelera silicon: Metis Pipeline Setup: I will use Voyager Wingman prompts to automatically generate, wire, and optimize the Metis execution pipeline on the Dell XE5 setup without writing complex YAML configurations manually. Upgrading to Live Vision: Right now, the arm module uses simulated virtual sensors. I plan to prompt Wingman to help integrate a real-time computer vision pipeline via the Voyager SDK, replacing virtual sensors with camera feedback to track actual physical end-effector positions. 🚀 Why This Proposal?Having a working software baseline means zero time wasted starting from scratch. I can jump straight into testing Voyager Wingman, generating pipelines, and benchmarking performance on real Axelera hardware from day one.Excited to see how fast Wingman can take this project to the next level. Let me know what you think! 🚀 below link github :https://github.com/BenNugraha/DT-BN_TWIN_Tech
𝕋ℍ𝔼 ℙ𝕀𝕋ℂℍ: Take a photo of a computer. Get back a fully custom Linux system — hardware identified, every device mapped, a complete validated build, and a desktop that already knows what it's running on the moment you log in. No board notes, no digging through datasheets, no hand-assembling BitBake layers. With img2linux, its easy as shoot n boot. How it works• See it: A photo runs through a component-detection pipeline that locates the SoC, RAM, connectors, M.2 slots, and add-on cards, then OCRs the actual part numbers and board names printed on the silkscreen. That's the real identification signal — visual shape alone isn't trustworthy for chips that look alike — so every guess gets confirmed against a hardware database and, once the board boots, against what the board itself reports.• Map it: Once the base board is known, img2linux pulls the real device tree source for it - the same file the kernel uses to understand its own buses, addresses, and pins - and folds every other piece of detected hardware into that map as an overlay. An Axelera Metis card, a camera, an NVMe drive: all of it becomes one structured picture of the whole machine, rendered as an interactive block diagram.• Cook it: With the hardware side settled, img2linux searches the Yocto/OpenEmbedded ecosystem — plus Axelera's own meta-axelera layer and BSP releases - for the board support package and machine definition that fits, folds in the kernel version and OS features requested, and assembles the full BitBake build.• Prove it: Before it spends an hour on a real build, img2linux parse-checks the recipe, runs static compatibility checks over the whole device map, and boot-tests it in a virtual machine. If anything fails, it redesigns the recipe and tries again - automatically - until the virtual boot passes clean.• Build it: Once it passes, img2linux kicks off the real build and hands over a finished image, ready to flash.• Show it: Every image it builds ships with one more thing: a desktop that renders the exact block diagram it built for that machine, with every device as a live tile - temperature, clock speed, memory, utilization - updating itself the instant something changes, no polling. Where the hardware allows it, settings can be edited right from the desktop, safely, because the same compatibility rules that built the recipe are the ones gating what's allowed to change. With img2linux, you photograph a computer, get a bespoke, working Linux distribution back, complete with a desktop that already understands its own hardware - needs zero technical explanation to land. Non-experts get it instantly. And underneath the "wow," every stage is real, deterministic engineering: known device trees, indexed BSPs, a database that won't let the model guess its way into an incompatible build, and a validation loop that only ships what's provably going to work.It's also a genuinely live, on-Metis demo in two places, not just a batch job: the photo-to-hardware-ID stage is a real vision pipeline running on the Metis card, and the desktop dashboard reads live telemetry: clocks, temperature, utilization - straight off the Metis card through its own device APIs while it's running. What we're not claiming:img2linux doesn't invent board support for hardware nobody's ever built a BSP for - bootloader and DDR bring-up for genuinely new silicon isn't something that can be responsibly automated from a datasheet. If a photographed board has no indexed BSP, img2linux says so plainly instead of guessing. It's a generator built on real, existing embedded-Linux infrastructure - not a silicon bring-up wizard.*img2linux will only use free and publicly available software. $THE PROMPT$Ok Wingman, ive got a *special* objective today and i need YOUR help. Lets build a system that takes a photo (or description) of a piece of hardware and produces a validated, ready-to-boot custom Linux image for it, and even includes a live desktop canvas that displays and lets you edit the hardware it's running on. Build it in these stages:1. Hardware ID (vision). Take a photo or video showing all components of a target system. Run a component-detection pipeline (a fine-tuned YOLO-family model) over the image to locate the SoC package, RAM, connectors, M.2 slots, PMICs, camera modules, and any add-ons. Run OCR over each detected region to read board names, revisions, and IC part numbers off the silkscreen and package markings - and treat this as the real identification signal, since visual appearance alone is unreliable for SoC packages that might look alike. Look up the OCR identifiers and compare against a hardware knowledge database, falling back to web/datasheet lookup for anything not already indexed. Surface the result as a candidate identification for me to confirm, and once the board is actually running, cross-check it against live probing (lspci, device tree, /proc/cpuinfo).2. Device tree resolution. Once the base board is identified, pull its authoritative device tree source (DTS) from its BSP rather than inferring one from photos or datasheets - the DTS is the real machine map: buses, peripheral base addresses, interrupts, clocks, pinmux. Parse it into a graph model of the base machine.3. Full-system composition. Add every other piece of hardware detected in the photo - including an Axelera Metis card, cameras, storage, or other accessories - into that graph as device tree overlays, so the graph represents the complete system as built, not just the base board.4. Block diagram. Render the completed machine graph as an interactive block diagram of the whole system: every device, its connections, and where it sits in the machine. This will serve as the absolute identity from which all the following steps build upon.5. BSP and recipe resolution. Search the OpenEmbedded Layer Index plus Axelera-specific sources (meta-axelera, Metis Compute Board BSP release notes, supported kernel versions, Voyager SDK runtime dependencies) for the BSP and MACHINE definition matching the identified hardware, preferring Yocto layers wherever one exists. Only generate a configuration for hardware with a known, indexed BSP layer — if nothing matches, tell me clearly rather than attempting to invent bootloader or DDR-init support from datasheets alone.6. Kernel and distro assembly. Using the user-specified kernel version and desired distro features, resolve the full set of recipes and companion layers needed, confirming every layer's compatible Yocto release agrees with the others as a deterministic database check, not a guess. Assemble the complete BitBake configuration — machine, distro, layers including meta-axelera, and the DISTRO_FEATURES/IMAGE_INSTALL lines - as a kas YAML.7. Virtualized validation loop. Before any real build, validate the whole recipe virtually: parse-check the generated config (bitbake -p / bitbake -n), run static checks over the device tree graph for bus conflicts, address overlaps, pinmux collisions, and power budget, and boot-test the result in QEMU where a machine model exists. If anything fails, revise the recipe and re-test until the virtual path passes clean. Tell me plainly that Metis-specific and other vendor peripherals still need hardware-in-the-loop confirmation once the board is in hand — don't claim the virtual pass covers that.8. Build. Once validated, run the real build and hand me the finished artifact — a bootable image or ISO for the target device.9. A living desktop. Every image you produce should include one more recipe: a desktop canvas that renders the block diagram from step 4, with each device as a live widget. Build this as a small event-driven system service that owns the machine graph and subscribes to the hardware's own event sources - udev/netlink hotplug events, sysfs attribute changes, D-Bus signals already broadcast by things like UPower and NetworkManager - instead of polling, with a lazy-refresh fallback only for the few attributes that don't emit change events. Widgets should be pure clients of that service's D-Bus API, with zero direct hardware access. Where a device supports it - including the Metis card, through its axdevice and tracer interfaces for clocks, temperature, memory, and MVM utilization - let the widget edit the setting directly, but gate every write behind the same compatibility checks used to build the recipe, so nothing lets me set a value the hardware doesn't actually support, and journal changes so the service can tell desired state from actual state after a reboot. Render it as one full-canvas application rather than separate floating widgets, so it works consistently across desktop environments and on the target board's own display output.Walk me through each stage as you build it, and ask me directly if anything about the target hardware or desired features is ambiguous before generating the recipe.
Sharing my first project on Axelera Metis: end-to-end monocular 3D object detection running at 24.5 FPS on a single AIPU core, with a CPU host and no GPU anywhere in the pipeline.What it does Takes a single camera image and predicts 3D bounding boxes for every car, pedestrian and cyclist in the scene position, dimensions and orientation , in real time. Built on MonoCon (CVPR 2022) with a DLA-34 backbone, evaluated on KITTI driving sequences.How it is split The model had to be divided at a hard compiler boundary. AttnBatchNorm2d contains a ReduceMean op that Voyager cannot quantize, so the pipeline is:AIPU: backbone + neck + fused HeadConv1 (64 to 576 channels, single output tensor) CPU host: AttnBatchNorm2d + ReLU + 1x1 convs, implemented in hand-written C++ PerformanceStage Time Preprocess + Quantize 1.4 ms AIPU inference 31 ms Dequantize 6.5 ms Head (C++) 22 ms Decode + NMS 1 ms Sequential total 62 ms / 16 FPS Pipelined throughput 41 ms / 24.5 FPS Pipelined throughput uses a two-thread producer-consumer design: the AIPU processes frame N+1 while the CPU decodes frame N concurrently.Key findings during deployment A few things that are not in the documentation and took real time to figure out: per_tensor_histogram (default) silently clips HeadConv1 activations — true range is roughly -1350 to +1200, the histogram scheme covers only -250 to +250. Zero detections until switching to per_tensor_min_max. Multi-output graphs get their outputs sorted alphabetically by the compiler. Discovered by comparing per-tensor statistics against a PyTorch reference. Fixed by fusing all nine head convs into one 64 to 576 conv with a single output tensor. axrArgument.fd must be -1 for host-pointer mode. Setting it to 0 silently produces zero output with AXR_SUCCESS returned. The first 2-3 inference calls return stale output regardless of input. Warm-up with dummy frames is required before trusting results. ONNXRuntime was taking 55ms on a 24,000-parameter subgraph , not because of arithmetic but because of per-node dispatch overhead across ~150 ops. Replaced with a preallocated C++ implementation: 55ms to 22ms. Repo Everything is open source: PyTorch model, ONNX export scripts,Voyager compile script, C++ inference library with 3D box and bird's-eye view visualization.https://github.com/sanket-pixel/monocon-metisHappy to answer questions on any part of the deployment.Bonus discussion point :Running the backbone+neck+HeadConv1 graph alone via axrunmodel hits 90 FPS on four cores and 40 FPS on a single core. The compiled model in this project uses aipu_cores_used=1, resources_used=0.25, straightforward but not necessarily optimal. There is likely headroom in the compiler configuration that this project has not fully explored: tiling depth, DFS search constraints, IMC double buffer pipelining, and grouping IFDW tasks are all knobs that affect how efficiently the compiler maps the graph onto the MVM array. If anyone has found configurations that push closer to the axrunmodel ceiling on a similarly sized graph, would be very interested to hear what worked. Happy to discuss.
Hello,As per original POST I’d like to request compatibility for IBM’s Granite modelsEmbedding and LLM ModelThank you,Peter
The examples that come with the SDK mostly use pipelines that do all the work of getting the data from the source (files or usb camera), sending for inference, and then displaying the inference results on the images. This is great for many use cases, I am sure, and enable a high degree of efficiency. However, for many users, the AI inference is part of a C/C++/python application where the images are obtained via OpenCV, processed (scale/crop/etc) in opencv based on application logic, and then a part of the image is sent for AI detection. Once the result is obtained, then the application can display the results over the opencv image using it’s own custom logic.So in summary, it would be great to have some examples that demonstrate this process. A simple example that shows how to use a detection model from the model zoo in C or Python along with opencv for obtaining image and displaying the results. Another good example would be running the model inference as a gstreamer pipeline. This would allow developers to feed the images into it, and get the results out using tools they are already familiar with (opencv, gstreamer, etc). p.s I am very new to Axlera/Metis so I apologize if I missed those examples :)
Hi all! I wanted to contribute a little bit about my experience bringing up the Metis M.2 in my LattePanda Sigma. Im an EE who is self taught, I have worked for Harley Davidson, Span.io, Enel X NA, and I’ve done several consulting projects on synthesizers and drum machines. My interest in AI is hobby and curiosity based, not professional. Software engineering is a new skill Im creating.I first tried to run Metis on the Windows 11 operating system that came with LattePanda Sigma, and following the instructions on the git was extremely confusing. I have ADD, and some other learning differences, but that wasn’t the reason installing was so hard on Windows. It’s because there isn’t a single page with coherent instructions telling the user how to bring up the hardware in a simple step by step format. Having the user click between git pages on firmware, installation of drivers, wsl, putting Windows into test mode, using multiple programming environments, etc is painful and makes installing the hardware on Windows a miserable, confusing, and difficult experience.I’ll probably make my own step by step guide for Windows at some point since the instructions on git are confusing.I then tried to do the install on an old version of Ubuntu (support 24.04 please!) and I had much less trouble. I HIGHLY RECOMMEND using Ubuntu 22.04 to run the hardware and software over Windows. The instructions to install the Voyager-SDK on Ubuntu 22.04 actually worked pretty well. I was surprised.I ran into an issue at one point after installing the SDK: I couldn’t detect the Metis M.2 card. I couldn’t really figure out how to install the driver from the instructions for Ubuntu installation, so I had to download the .deb driver and install it similar to the “instructions for installing Voyager-SDK using Docker” to get the driver installed. Once the driver is installed, and the clunky SDK is installed and running, the demos with YOLO worked great, with the Metis M.2 card barely breaking a sweat doing inferences.
I'm developing a new way of video with or without AI. I am marketing both small & large companies using their people as limited actors. I saw Titanium and i was very curious, can I merge dynamic ai actors into video with real actors? I am getting paperwork together to start, so I have time to learn. My background is in 2 & 3D animation & graphic design (degrees in both bs & bfa) What is the learning curve?
Already have an account? Login
No account yet? Create an account
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.