The Problem, Personally
As a drone builder and hardware practitioner, I have spent a lot of time developing and testing different aerial platforms, including FPV systems, while working with low-latency video transmission and real-world flight environments. In my practical experiments with airborne anomaly detection using classical computer vision (standard contour, edge, and motion detection), I ran straight into a frustrating limitation: false alarms caused by dynamic sky environments.
When a camera is pointed at the sky from a moving platform, classical edge detectors treat moving cloud boundaries, changing light textures, and sun flares as airborne objects. The system constantly generates false positives. For real-world drone operations, aerial inspection, or pilot awareness, simple pixel-contrast heuristics are simply not reliable enough.
To evaluate real physical objects against a dynamic sky background, simple pixel boundaries aren't enough — you need local Edge AI inference capable of processing deep learning models in real time without introducing stream delay.
The Solution & Architecture
EdgeVision Sentinel is a platform-independent Edge AI perception module designed to process live video streams for real-time aerial awareness with low-latency processing.
The FPV platform is used only as an affordable and flexible real-world testing environment. The developed perception module itself is independent of the flight platform; the **main focus is creating an efficient Edge AI perception layer optimized for Axelera Metis using Voyager Wingman**, which can be integrated into different aerial systems:
- Consumer drones, industrial inspection UAVs, agricultural drones, and future autonomous aerial platforms
- Fixed-wing aircraft and VTOL systems
- Stationary aerial monitoring cameras
The objective of this project is not autonomous flight control, but an AI-assisted perception layer. The initial implementation is designed as a human-in-the-loop assistance tool — providing additional perception information to improve operator awareness and reaction time while keeping final decisions strictly with the human operator. By separating perception from flight control, the module remains platform-independent and can support both human-operated and future autonomous aerial systems requiring environmental understanding and obstacle awareness.
Hardware & Edge AI Architecture
Video Feed: Low-latency HD stream from the camera source (OpenHD / USB / RTSP).
Local Acceleration (Axelera Metis): Deep learning inference runs locally on the Metis PCIe card inside the host ground station, eliminating the need for cloud-based inference and enabling operation in environments where cloud connectivity is unavailable or undesirable.
Perception Engine (Voyager Wingman Integration): A custom pipeline engineered with Voyager Wingman that detects and analyzes potential airborne objects, estimates confidence scores, and evaluates false positives against complex cloud backgrounds.
Host Telemetry Overlay: Real-time visual interface displaying target detections, confidence, overall FPS, and per-stage latency straight from the Metis card.
Why Voyager Wingman & Axelera Metis
I approach this project from a hardware integration and systems engineering perspective, combining practical aerial platform experience with AI-assisted software development. **Voyager Wingman will be used as an AI engineering assistant throughout the development cycle:** translating system requirements into Metis-compatible pipelines, generating the initial inference workflow, testing deployment on hardware, analyzing performance, and iterating toward an optimized solution.
The Metis accelerator provides exactly what edge aerial vision demands: high-throughput local processing, low latency, and independence from cloud-based inference.
1-Month MVP Scope, Development Journey & Reproducibility
To demonstrate how Voyager Wingman dramatically accelerates complex Edge AI engineering within a tight one-month timeline, the development journey and deliverables are structured as follows:
- Live Feed Ingestion & Wingman Setup: Prompt Wingman to generate an initial Metis-compatible pipeline ingesting a live video feed.
- Accelerated Perception Pipeline: Deploy and run accelerated airborne object perception fully on Metis silicon.
- Telemetry & Visual Feedback: Integrate real-time preview with bounding boxes, confidence scores, FPS, and Metis latency metrics.
- False-Alarm & Performance Evaluation: Test, document, and iterate the pipeline’s performance and false-positive behavior against dynamic sky backgrounds and clouds.
Measurable Success Criteria: Success will be evaluated by real-time inference performance, latency measurements, FPS stability, and a documented evaluation of false-positive behavior compared with classical computer vision approaches.
Reproducibility, Working Demo & Development Story: Final project deliverables will document the full AI-assisted development journey — including all Voyager Wingman prompts, YAML configurations, prompt iteration logs, source code repository, hardware Bill of Materials (BOM), and step-by-step setup instructions required to fully recreate the working demonstration. The final submission will include a recorded working demonstration running on Axelera Metis hardware, showing the complete pipeline from live video input to AI detection output.
THE PROMPT
Wingman, help me build an Edge AI aerial awareness pipeline running on the Axelera Metis PCIe accelerator. Act as my AI engineering partner throughout this build — translate my system requirements into Metis-compatible pipelines, generate the initial inference workflow, test deployment on hardware, analyze performance, and iterate toward an optimized solution.
Goal: Process a live video stream from a remote camera/drone feed to analyze and detect potential airborne objects in real time, while evaluating host overlay performance, latency, and background noise suppression.
Step 1 — Model Selection & Baseline Pipeline: Before selecting a model, analyze available Metis-supported models and recommend the best trade-off between accuracy, latency, and resource usage. Help me select and deploy the most suitable lightweight object detection model. Generate a Voyager pipeline (YAML/wiring) that ingests a live video feed (OpenHD/USB stream) and outputs bounding boxes with confidence scores.
Step 2 — Model Optimization & SDK Tuning: Guide me through quantization, compilation, and Metis SDK settings to minimize latency and maintain high, stable frame rates required for real-time aerial video processing.
Step 3 — False-Positive Analysis & Robustness: Help me configure host logic and background-aware filtering to evaluate and suppress false alarms caused by dynamic sky backgrounds, moving cloud edges, and lighting changes.
Step 4 — Telemetry, Visualization & Reproducibility: Create a live annotated output display showing detected objects, class labels, confidence scores, overall pipeline FPS, and per-stage inference latency reported directly from Metis hardware. Help me structure the repository, configuration, and prompt log for full project reproducibility.
