AeroFlood

Last winter, roads near my home disappeared under water during exceptional flooding. This summer, forests in the same area have been hit by major wildfires.

In both situations, the same problem appears: the disaster evolves faster than the operational map. A road can be usable one hour and underwater the next; a neighbourhood can become isolated. A drone can show responders what is happening almost immediately, but its video still has to be interpreted and turned into geographic information before anyone can act on it.

What if every frame captured by a drone could immediately update the map used to respond to the disaster?

Flooding near my home this winter, the kind of rapidly changing situation AeroFlood is designed for.

What we're building

AeroFlood turns a live drone flight into a continuously updated map of flooded areas, affected infrastructure and usable roads. A DJI Spark surveys the area while a portable ground station built around the Dell XE5 and its 16GB Axelera Metis PCIe card processes the video locally.

Flooded roads, flooded buildings and water surfaces are segmented, projected into geographic coordinates from drone telemetry and accumulated as the drone moves. Intersecting this hazard layer with the road network lets us invalidate flooded road edges and recompute an alternative route.

The final demo should tell the whole story in one sequence:

drone observes flood  =>  Metis segments it  =>  map updates  =>  road becomes unavailable  =>  route changes

For field testing, we can safely fly over ponds and lakes to exercise live acquisition, segmentation and georeferencing; recorded flood imagery and FloodNet will validate flooded-road scenarios.

What we already have

This project does not start from zero. In a previous drone hackathon, some of us built an experimental backend that projects DJI imagery and detections into geographic coordinates, plus a React and Mapbox command centre to display them. The public mapping backend is here: https://github.com/etyl/drone-mapping
We will reuse that geospatial core where it is sound. What is new here is the live Metis inference, FloodNet segmentation, live DJI bridge, persistent flood fusion, road-state analysis and routing.

Why Metis fits this project

AeroFlood is exactly the kind of system where edge inference matters. Metis gives us high-throughput vision inference with a very small power footprint, rather than relying on a power-hungry GPU or sending every drone frame to the cloud. Axelera positions Metis at up to 15 TOPS/W and specifically targets low-power edge deployments. In a disaster area, processing the video directly at the ground station also means low latency, no dependency on a reliable internet connection, and no need to continuously upload sensitive aerial footage. The raw video can stay local while only the resulting flood map and alerts need to be shared. For AeroFlood, edge AI is not just a deployment constraint; it is part of what makes the system useful in the field.

How it works

Flood perception

FloodNet provides 2,343 high-resolution UAV images collected after Hurricane Harvey with ten semantic classes: Background, Building Flooded, Building Non-Flooded, Road Flooded, Road Non-Flooded, Water, Tree, Vehicle, Pool and Grass. Its flooded/non-flooded road and building classes matter because we need to distinguish normal water from water covering infrastructure.

Our primary architecture is the U-Net FCN 512x512 already supported by the Voyager Model Zoo, retrained on FloodNet. Axelera reports 34 FPS for the Cityscapes reference version with a 0.01 mIoU loss after quantization; we will use the same deployment path but measure accuracy and throughput on our own weights.

Voyager's custom-weight segmentation path through SegDataAdapter gives us a conservative hardware-compatible baseline. If Road Flooded quality becomes the bottleneck, a second architecture is an upside rather than a dependency.

Dataset: https://github.com/BinaLab/FloodNet-Supervised_v1.0

Model Zoo: https://docs.axelera.ai/sdk/reference/models/model-zoo/

Custom weights: https://docs.axelera.ai/sdk/tutorials/custom-weights/

Drone acquisition

Our DJI Spark is supported by DJI Mobile SDK v4. The target acquisition chain is:

DJI Spark  =>  Android bridge  =>  H.264 video + synchronized telemetry  =>  Dell XE5

The phone exposes video, GPS, altitude, aircraft attitude and gimbal state; the AI pipeline remains on the ground station.

DJI Mobile SDK v4: https://developer.dji.com/mobile-sdk-v4

Georeferencing and map fusion

For each frame we combine camera intrinsics, GPS, altitude, aircraft attitude and gimbal orientation to cast image rays onto the ground. Segmentation masks are warped with OpenCV, converted to geographic coordinates and fused over time into a persistent hazard layer.

Road analysis and routing

OpenStreetMap is loaded as a graph with OSMnx and NetworkX. Road edges intersecting confident Road Flooded observations are removed or penalized, and routes are recomputed with weighted shortest paths.

The division of work is intentional:

Metis: semantic segmentation

CPU: telemetry synchronization, georeferencing, temporal fusion, road intersection and routing

Frontend: React + Mapbox GL JS

Building it with Wingman

We split the work into three engineering targets rather than one giant prompt.

1. Perception — deploy the FloodNet U-Net on Metis, expose masks to Python, validate against the original model and profile until the pipeline is genuinely real-time.

2. DJI bridge — start from recorded H.264 plus telemetry, then swap in the live Spark while keeping the same downstream interface.

3. Closed loop — reuse the existing geospatial code and connect segmentation to map fusion, road-state analysis and routing.

Roadmap: training and validation  =>  Metis deployment  =>  recorded-video pipeline  =>  live DJI  =>  map fusion  =>  routing  =>  field testing.

What success looks like

We will report mIoU and per-class IoU before and after Metis quantization, Road Flooded precision/recall, Metis FPS, end-to-end frame-to-map latency, projection error on known landmarks and route-recomputation success.

AeroFlood will not be a certified emergency-navigation product after one month. The Spark is not RTK, so we target an operational hazard layer rather than survey-grade mapping. The drone remains manually operated, and a route marked safe only means it avoids hazards currently observed by the system; it does not replace emergency-service instructions.

Flooding is the first use case because we can test the full system safely, but the pipeline after segmentation is largely hazard-agnostic. Swap in a fire or smoke model and the same acquisition, georeferencing, fusion and road analysis can progressively update a wildfire perimeter.

A drone already has a great view of a disaster. AeroFlood's job is to make sure the map does too.

Starting prompts

Prompt 1: Flood perception on Metis

Wingman, we are building AeroFlood, a real-time UAV flood mapping system on a Dell XE5 with a 16GB Metis PCIe card.

Start by inspecting the Voyager U-Net FCN 512 semantic-segmentation pipeline. Our target model keeps this architecture but uses custom weights trained on the 10-class FloodNet dataset: Background, Building Flooded, Building Non-Flooded, Road Flooded, Road Non-Flooded, Water, Tree, Vehicle, Pool and Grass.

I will provide the model weights, the FloodNet dataset and our project repository.

Adapt the existing Voyager segmentation pipeline for these custom weights and classes. Use the SDK semantic-segmentation data adapter and representative FloodNet images for quantization. Compile and run the model on the Metis AIPU. The segmentation mask must remain programmatically accessible to a host-side Python application, not only rendered as an annotated video.

Build a validation procedure that compares the original model and Metis output using overall mIoU, per-class IoU and specifically precision/recall for Road Flooded. Measure actual inference FPS and latency.

Target at least 10 FPS at 512x512. Your task is not complete when the model merely compiles. If the target is not reached, profile the bottleneck and iterate on the pipeline until no reasonable pipeline-side optimization remains. Preserve all configuration, code and commands required to reproduce the result.

Prompt 2: Connect the DJI Spark

Wingman, the AeroFlood Metis segmentation pipeline now works from recorded video.

I will add the DJI Mobile SDK v4 documentation and sample code to the project knowledge base, together with our existing drone-mapping repository.

Build the smallest reliable bridge between a DJI Spark and the XE5. We need synchronized video frames plus GPS, altitude, aircraft attitude and gimbal orientation.

First implement the interface using recorded H.264 video and recorded DJI telemetry. Keep this source independent from the downstream segmentation pipeline.

Once the recorded path is stable, replace the source with the live DJI Mobile SDK v4 feed and telemetry from the Spark. Keep the Metis pipeline unchanged if possible.

The target is reached only when the same downstream Python application can switch between a recorded flight and the live Spark without changing its input contract. Document the synchronization strategy, timestamps, buffering and any latency introduced by the bridge.

Prompt 3: Close the loop from flood to route

Wingman, we now have synchronized drone imagery, telemetry and a 10-class flood segmentation mask.

Inspect our previous drone-mapping and command-centre code before modifying it. Reuse the existing image-to-ground projection and Mapbox infrastructure where it is sound, but do not assume the old hackathon code is production-ready.

Extend the application so that successive segmentation masks are georeferenced and fused into a persistent GeoJSON hazard layer. Load the surrounding road network from OpenStreetMap with OSMnx. Detect road edges intersecting confident Road Flooded observations, mark those edges unavailable and recompute a route that avoids them.

Keep perception on Metis and geometry, map fusion and routing on the host CPU unless profiling shows a clear reason to change that split.

The target is one closed-loop demo:

drone observes flood  →  Metis segments it  →  map updates  →  road becomes unavailable  →  route changes

Keep iterating until that complete sequence can be demonstrated reproducibly from an input video and telemetry sequence. Then connect the same flow to the live DJI source from Prompt 2. Preserve the repository structure, configuration and instructions needed to reproduce the demo.