Skip to main content

BuildWingman — AI Visual Memory for Every Build, Repair and Modification

  • Aug 9, 2026 2:34 PM UTC
  • 1 reply
  • 49 views

 

Can a workshop remember what happened during a build—without forcing the builder to stop, take photos, write notes, or record and search through hours of video later?

Anyone who builds, repairs or modifies physical things knows the moment: a device is half disassembled, screws are sorted into questionable piles, connectors are unplugged, cables are rerouted—and the question appears:

“How was this connected before I took it apart?”

This happens with electronics, vehicles, PCs, laptops, 3D printers, robots, sim rigs, power tools, engine parts and almost every repair project. The usual solution is manual photographs and notes, often taken too late, inconsistently, or not at all. Video is not the answer either: nobody wants to scrub through six hours of footage to find the moment a connector was removed.

I want to build BuildWingman: an AI visual-memory system for garages, workshops and makers. It observes a work areas through fixed cameras, identifies meaningful build events, and turns them into a searchable timeline of visual evidence.

Instead of becoming another generic workshop camera, BuildWingman creates a reversible visual memory:

It remembers what was there, what changed, and how it went back together.

A user should be able to ask:

  • “Show me how this controller was wired.”

  • “When did I remove this connector?”

  • “Which screws came from this cover?”

  • “Show the step before the motor was removed.”

  • “What changed between yesterday’s and today’s work session?”

The system returns the relevant image sequence, timestamp, detected components and a concise event summary—not a long recording to search manually.

The idea

BuildWingman combines local, real-time edge vision with optional richer processing when it is useful.

Two or three cameras cover a workbench from complementary angles:

  • an overhead view for components, tools, cables and work zones;

  • a side view for hands, depth, insertion/removal actions and occlusion handling;

  • optionally, a close view of the active assembly area.

A compact computer with an Axelera Metis M.2/NVMe device runs the continuous perception pipeline. It watches for meaningful changes in the scene rather than saving all footage.

When an event happens—such as a component being removed, a cable being disconnected, a tool being used in the active area, or an assembly being installed—the system stores only the useful evidence:

  • a short event clip;

  • before-and-after keyframes;

  • detected objects and work zone;

  • event type and confidence;

  • a timestamp;

  • optional user confirmation or note.

The result is a project history that becomes more useful over time.

Why Metis matters

A workshop visual-memory system needs continuous observation, but continuous cloud video upload is expensive, slow and unnecessary for the first stage of perception.

Metis enables local processing of multiple camera streams so the system can decide which moments are worth keeping. That means BuildWingman can:

  • process live workbench video with low latency;

  • reduce storage by retaining meaningful events instead of continuous recordings;

  • keep normal workshop footage on the local network;

  • continue capturing the core visual timeline even if an internet connection is unavailable;

  • run in an affordable compact workstation, garage or maker-space setup rather than requiring a GPU gaming PC.

This is not an “offline only” product. Optional cloud processing can add real value after local event capture:

  • encrypted project backup;

  • remote project access;

  • sharing a repair timeline with a friend, collaborator or customer;

  • richer visual-language summaries;

  • community repair guides;

  • improving general models from explicitly shared and anonymised data.

But the key edge-AI role remains local: watch continuously, understand enough to identify useful moments, and capture visual evidence at the moment it matters.

How BuildWingman works

Overhead camera + side camera + optional close camera
| v [1] WORKBENCH UNDERSTANDING Detect work zones, hands, tools, components and the active assembly area.
| v [2] CHANGE DETECTION Compare the stable workbench state over time: what entered, left, moved or changed?
| v [3] ACTION / EVENT DETECTION Identify meaningful events such as component removal, installation, cable connection, tool use or assembly change.
| v [4] EVENT CAPTURE Save pre-event and post-event keyframes, a short clip, timestamp, camera references and confidence values.
| v [5] VISUAL PROJECT TIMELINE Create an indexed sequence of steps for each build session.
| v [6] SEARCH, REVIEW AND CORRECTION Search by object, event, date or natural-language question. The user can confirm, rename or correct important events.
| v [7] FOCUSED MODEL IMPROVEMENT User-confirmed tools, components and actions become training examples for increasingly accurate workshop-specific models.

The first challenge demonstration

The one-month challenge version will focus on one real project and one defined workbench setup rather than pretending to understand every tool and every repair type.

My demonstration project will document a real electronics/mechanical build or repair involving identifiable components, connectors, wiring and fasteners—such hardware modification, laptop repair, or another workshop assembly.

The demo will show the full loop:

  1. A user begins with an assembled object or wiring layout.

  2. BuildWingman observes the workbench through two cameras.

  3. The user removes a cover, disconnects cables, removes components and performs a modification or repair.

  4. The system creates a visual event timeline instead of continuously storing the full session.

  5. Later, the user searches the timeline:

    • “Show before this connector was removed.”

    • “How was the controller wired?”

    • “Show the screws from the cover.”

  6. BuildWingman returns the relevant event, including before/after imagery and the moment it occurred.

  7. The user corrects or confirms an uncertain event, adding a useful labelled example for future recognition.

The final demonstration should make the value immediately obvious: disassemble something complex, then use BuildWingman to find the evidence needed to reassemble it correctly or record how it was built.

MVP scope

The purpose of the MVP is not to solve general robotics or understand every workshop task. It is to prove that local AI can turn physical work into searchable, useful memory.

Priority Deliverable
Core Two-camera live workbench pipeline on Metis
Core Work-zone, hand and broad object/component detection
Core Detecting meaningful state changes in a defined active work area
Core Event capture: before/after frames, short clip, timestamp and metadata
Core Local visual timeline for one documented build or repair
Core Search by event type, component label and time
Core User review and correction interface
Strong extension Tool-use detection and connector/cable insertion or removal events
Strong extension Optional cloud backup, sharing or richer event summaries
Strong extension Project-aware search: “show the step before this was removed”
Stretch Custom model trained on the workspace’s own tools and components
Stretch Automatic procedural build summary generated from confirmed events

 

Voyager Wingman starting prompts

I will use Voyager Wingman to create and iteratively improve each stage of the Metis perception pipeline. These are the initial staged prompts; the real prompt history, revisions, pipeline configurations, measurements and results will be published during the build.

Stage 1 — Camera and workbench understanding
“Build a real-time Metis pipeline for two fixed cameras covering a workshop workbench: one overhead camera and one side camera. Detect people, hands, workbench zones, broad tool classes and components in the active work area. Return timestamped structured detections with object class, confidence, bounding box and camera ID. Prioritise stable local inference and low false positives in an indoor workshop environment.”

Stage 2 — Change and event detection
“Extend the workbench pipeline to maintain a stable scene state and detect meaningful changes in configured work zones. Identify when an object or component enters, leaves, is moved, is removed from an assembly or is installed into an assembly. Ignore ordinary hand movement where no meaningful scene change remains after the hand leaves. Return event type, timestamp, object candidates, confidence and involved camera views.”

Stage 3 — Evidence capture
“For every high-confidence workbench event, save a configurable set of evidence: keyframes before and after the event, a short clip around the event, camera ID, work-zone ID, object detections and event confidence. Do not continuously archive raw video. Expose event records as structured JSON for a local timeline application.”

Stage 4 — Multi-camera event fusion
“Fuse detections from overhead and side workbench cameras into one event record. Use agreement between views to increase confidence and use the second view to reduce false events caused by hand occlusion. Preserve camera-specific evidence so the user can inspect both views later.”

Stage 5 — Searchable build timeline
“Create a lightweight local application that consumes BuildWingman event records and displays a chronological project timeline. Support filtering and search by date, camera, work zone, detected component, tool and event type. Selecting an event must show the before-and-after images, saved clip and detected metadata.”

Stage 6 — User correction and focused learning data
“Add a review interface for low-confidence or important events. Let the user correct the event type, component label, tool label and description. Export confirmed examples with their event frames and metadata into a training dataset for compact custom models specialised for that individual workshop and project type.”

Stage 7 — Natural-language project retrieval
“Add a retrieval layer over confirmed BuildWingman events so a user can search using questions such as ‘How was this controller wired?’, ‘Show the connector removal before the motor was removed’, or ‘Which screws came from the cover?’. Return the most relevant event evidence and do not invent details that are not visible in the recorded project history.”

Build plan

Week 1 — Make the workbench observable

  • Set up the cameras, lighting and defined work zones.

  • Capture example build and repair sessions.

  • Use Wingman to bring up live multi-camera inference on Metis.

  • Establish reliable detection of hands, the active work area and a constrained set of components/tools.

Week 2 — Make the system recognise change

  • Add stable-scene comparison and event detection.

  • Capture before/after keyframes and short evidence clips.

  • Test practical events: components entering/leaving zones, connectors being removed, parts being installed and covers being removed.

Week 3 — Make it useful after the build

  • Build the local visual project timeline.

  • Add filtering, component/event labels and review tools.

  • Run a full real build/repair session and use the timeline during reassembly.

Week 4 — Make it demonstrable and repeatable

  • Add multi-camera confidence fusion and user corrections.

  • Refine the event-selection logic to avoid saving unhelpful footage.

  • Add natural-language retrieval or structured visual search if the core loop is stable.

  • Record the final demonstration and publish the complete Wingman journey, code, pipeline configuration, dataset preparation and limitations.

Limits of the first version

The challenge version will be intentionally constrained:

  • one physical workbench layout;

  • two fixed camera angles;

  • a limited group of tools, components and event types;

  • one or two real build/repair projects;

  • user confirmation for uncertain events.

It will not claim to understand every object, every wiring system or every workshop activity without training. The objective is to prove a reliable and valuable workflow:

Build something. Take it apart. Change it. Later, find exactly how it was before—using visual evidence captured automatically while you worked.

Why I want to build it

I work with computers and computer repairs, and I like challenges and I am currently in the process of building a workshop where this kind of system could be helpful, in the club we have a lot of elderly people where memory might be a bit where it used to be. BuildWingman is designed to solve: wiring and component decisions are easy to make while standing at the workbench, but difficult to reconstruct days or weeks later, so digital memories on how this were are valuable.

I also work with local hardware, automation and agentic systems. BuildWingman is a practical extension of that approach: AI should not merely generate text about a project after the fact. It should observe the real build process, capture useful evidence automatically, and give the builder a durable memory of their own work or on our case transfer generational knowledge if the goal is to make a fishing lure in the workshop.

Final deliverables

The final project submission will include:

  • a working live BuildWingman demo running on Axelera hardware;

  • the original full Voyager Wingman prompt and all meaningful prompt iterations;

  • pipeline configuration and application code;

  • photos and video of the physical camera/workbench setup;

  • a documented real build or computer repair session;

  • a demonstration of event capture, timeline search and reassembly lookup;

  • measured event-detection behaviour, storage reduction and known limitations;

  • model/data preparation notes and user-correction workflow.

BuildWingman is not a camera system that records a workshop. It is a workshop that remembers how things were built.

1 reply

  • Nov 20, 2026 4:20 AM UTC

i want to creat an app can anyone help me