Developers and engineers looking for new options for their fleets of devices using edge AI are getting them.

BalenaOS has become popular precisely because it solves real deployment problems. The container-focused operating system enables over-the-air updates, remote management, and scalable fleets across embedded and edge devices.

This approach works perfectly for large-scale solutions in smart retail, healthcare, autonomous systems, robotics, and manufacturing. The challenge has been finding AI hardware that matches balenaOS's deployment philosophy.

Current solutions often create vendor lock-in and setup complexity that defeats the purpose of choosing balenaOS in the first place. New AI accelerators solve this problem by delivering right-sized performance that integrates naturally with container-native workflows. Businesses can optimize both system performance and power consumption without sacrificing the deployment flexibility that makes balenaOS valuable.

Axelera AI's Metis AIPU is purpose-built for computer vision and LLM inferencing at the edge with the best performance-to-cost ratio in the market. Metis' ease of use is the perfect match for balenaOS, so we set out to test whether we could bring this powerful combination to innovators at the edge.

What We Tested

Our validation was performed on an x86-based Lenovo P360 Ultra workstation, equipped with a Metis PCIe card. We used Voyager SDK version 1.3.3 and built a custom balenaOS image tailored for this setup. The goal was to enable full inference capabilities within a Docker container running on balenaOS, using the Metis accelerator.

Out-of-the-box balenaOS images do not include certain kernel features required by the Metis driver; specifically, support for system DMA-BUF heaps. Without these, the driver cannot access the necessary memory interfaces, preventing inference workloads from executing.

To address this, we built a custom balenaOS image from source, enabling the required kernel options. This involved identifying the correct device configuration for our x86 platform, modifying kernel build parameters, and compiling the OS using balena’s Yocto-based build system.

Driver and SDK Integration

Once the custom OS was in place, we compiled the Metis driver against the specific kernel version used in our balenaOS build. This step is critical, as balenaOS does not support DKMS, so drivers must be built manually for each kernel variant.

With the driver installed and loaded, we deployed the Voyager SDK inside a Docker container. Using a prebuilt image that includes all necessary runtime components, we configured the container to access the Metis device and other system resources required for inference.

Running Inference on balenaOS

After setting up the environment, we successfully ran inference workloads using the Voyager SDK inside the container. The system was able to process video input and deliver high-throughput inference performance, confirming that the Metis accelerator is fully functional under balenaOS.

To streamline future deployments, we also created a startup script that prepares the device environment on boot, loading the driver, setting permissions, and configuring device links so that inference can begin immediately after power-on.

Why This Matters

The combination of Metis processing power and balenaOS flexibility creates new possibilities for production-scale edge AI in practical deployment scenarios.

• Retail AI at the Edge Multi-camera video analytics becomes feasible at scale. Think queue tracking across multiple checkout lanes, shelf monitoring for inventory management, or customer behavior analysis across store sections. Metis handles the stream concurrency that these applications demand, while balenaOS manages secure fleet updates across hundreds of retail locations without requiring on-site technical staff.
• Intelligent Transport Monitoring Vehicle-mounted systems can now run sophisticated AI workloads locally. License plate recognition, passenger analytics, and incident detection operate on low-power hardware, like Metis’ 8-15 watts, with real-time processing capabilities. When new models or detection algorithms become available, balenaOS delivers over-the-air updates to entire transit fleets without service interruption.
• Factory Floor and Infrastructure Inspection High-resolution defect detection systems can deploy across manufacturing environments with enterprise-grade management. Metis hardware even supports LLM-based analysis for complex inspection scenarios. Container-based deployment provides the audit trails and version control that regulatory environments require, while Metis delivers the processing power needed for real-time quality control.

The low cost and ease of deployment mean development teams can test and iterate new systems and solutions with speed and flexibility.

Looking Ahead

This validation advances our core mission: ensuring that breakthrough AI capabilities are within reach for every innovator building edge applications. If you’re working on an edge AI project and see potential in this approach, feel free to ask any questions on implementation below, share ideas for adapting it to your use case, or just tell us about the project you're building so we can help you get there.