Hi Axelera team!
The problem.
White canes and guide dogs are brilliant at what they do: ground-level obstacles, a metre or two ahead. What they can't do is answer the questions a blind pedestrian actually asks: where is the door? the elevator button? the platform? the exit? and they can't warn about what's coming: a bike from the left, a branch at head height, a descending kerb.
Most assistive apps respond by narrating everything ("bin ahead, person ahead, wall ahead"), which is exhausting. Continuous audio guidance has the same flaw: it trades one cognitive load for another.
The idea.
Walkr treats the camera as a visual sensor you can talk to and its default state is silence.
Two interaction modes, nothing else:
- Ask-and-locate. The user asks: "Where is the door?" Walkr finds it in the scene and answers with a short spatialised audio ping coming from the door's direction, plus one concise phrase: "Door, three metres, slightly right." A punctual 3D ping is more precise than words alone for direction, and far less tiring than any continuous stream. The user's head orientation acts like a gaze: ask again while turning, and Walkr re-anchors the answer.
- Guardian. Fully automatic, hazard-only: "bike, left", "branch, head height", "kerb down, one metre", "obstacle ahead". Powered by an internal free-corridor risk engine (detection + segmentation + depth over a zone grid, scored for time-to-contact), but the user only ever hears the conclusion — never the scene.
Silence means: all clear.
That's the design principle, validated by the flaw we see in every narrating app.
How it works (the pipeline Wingman builds).
A chest- or head-mounted camera feeds a cascaded Metis pipeline: INT8 YOLO detection (doors, stairs, elevators, signs, people, bikes, vehicles, head-height obstacles) + semantic segmentation of ground classes (sidewalk, road, crosswalk, stairs) + monocular depth.
On the host: a zone-grid risk engine for Guardian mode, a lightweight voice-command layer (fixed query vocabulary) for ask-and-locate, and spatialised (HRTF) audio output. Perception-heavy, latency-critical, low-power: squarely Metis's sweet spot. Wingman generates and iterates the vision pipeline so we spend the month on what's genuinely hard: query-to-object grounding and the alert policy (what deserves to interrupt silence, and when).
The one-month MVP live on Metis.
Not "it would work if". A live demo where a user (blindfolded tester for the demo; real blind users are the honest next step) walks an indoor/outdoor course and:
- asks "where is the door / the stairs / the exit?" → gets a directional ping + short answer, and reaches it;
- gets interrupted only by real hazards: an approaching bike, a head-height obstacle, a kerb.
Events the pipeline emits: query_answered (class + bearing + distance), hazard_bike|vehicle (side + time-to-contact), obstacle_head_height, kerb_step (up/down), leaving_sidewalk, crossing_ahead — with on-screen latency/FPS from the Metis card.
Week-by-week.
- Week 1 — Wingman generates the base pipeline (INT8 YOLO + segmentation) on the Dell XE5 / Metis; live camera in, detections + ground classes out in real time.
- Week 2 — Monocular depth + zone-grid risk engine (time-to-contact, head-height layer); Guardian alerts working end-to-end.
- Week 3 — Voice-command layer (fixed vocabulary), query-to-object grounding, spatialised ping + concise TTS answers.
- Week 4 — Full course demo (XE5 in a backpack, camera worn), latency/FPS + power figures overlay, video, repo cleanup, full Wingman prompt journey write-up.
Deliverables.
Open-source GitHub repo (code + configs), the full Wingman prompt and every iteration, photos/video of the live run on Metis, and a write-up of what Wingman generated vs what we hand-built.
What we're not claiming.
Walkr doesn't replace the cane or the guide dog. It covers the layer they can't: head-height obstacles, anticipation at 3-10 m, and answering where questions. No open-ended scene description / VLM in this build. The query vocabulary is a fixed set of ~15 urban and indoor classes, stated plainly. And the interaction design (what earns an interruption, how a ping should sound) only proves itself with real blind users; the challenge MVP is the instrument to run that validation right after.
From desktop to wearable.
Let's be clear: the Dell XE5 is our development and proof bench, not the product. A blind pedestrian won't carry a desktop. The whole point of proving the pipeline on Metis is that the same silicon family scales down: the target is a compact, wearable, battery-powered device (Metis M.2 class, e.g. the new Axelera AI Mini PC form factor or an embedded design) running the exact same Voyager pipeline. For the challenge demo, the XE5 rides in a small backpack with the camera chest or head-mounted. Honest about what it is, while showing the path to the real thing. Power and thermal figures measured during the build will feed directly into sizing that wearable version.
Why it matters.
The “WHO” counts 43 million blind people and 295 million with moderate-to-severe vision impairment worldwide. A fully-local, low-power guidance device on European edge silicon; no cloud, no privacy trade-off; is genuinely useful, not a demo toy. This challenge is the fastest way to prove the perception-and-answer core runs in real time on Metis, and to measure what the wearable version needs.
Who we are and why this challenge, honestly.
Alyce is a French company specialised in mobility-data collection and valorisation, already working with Axelera / Kontron on edge vision. Here's the honest part: we bought a Metis PCIe card a while ago… and our roadmap has never let us properly get our hands on it. It's been sitting there, quietly judging us. This challenge is the perfect forcing function: a real project, a hard deadline, and a Wingman to get us from zero hands-on experience to a working pipeline in a month.
If Wingman is as fast as you say it is, a team starting from scratch is exactly the test it deserves. Build team: a hands-on group of four (product, computer vision, edge integration).
Excited to build this with a Wingman on board.
Good luck everyone! 🚀
