Super Phone Jail is a program that protects you from the social media addiction, specifically from picking up your phone while you work on the computer.
We all have the problem of social media adiction and it can make us feel really bad when we are on the computer and we randomly pick up the phone and waste half an hour without noticing.
Think about it, the corporations that run social media have unlimited money and the best engineers to hack your brain, and so they do… Until now, because Super Phone Jail keeps you away from your phone by recording your working session with a cheap camera and detecting if you grab your phone using AI on the edge. The algorithm comprises 3 ML models. The first one detects your skeleton using a YOLO so that we can know if you are on camera and if you are grabbing something. The second one detects the phone. The third model detects the gaze (where you are looking). By smartly ensembling the results of the three models we obtain a very clear result of if you are distracted or not by your phone. If the user is distracted by the phone, an LED flashes
Challenges: while there is support in the Axelera SDK for standard object detection as well as pose estimation, there isn’t to my knowledge support for gaze estimation. That is fine, because I believe that with the help of Axeleras agent and one month of time I could potentially add support for this new task! The current plan is to use MobileGaze which uses operations fully supported by Axelera’s chip.
I'll build the whole pipeline with Voyager Wingman: it generates and wires the three models, helps me compile MobileGaze onto the Metis card, and lets me iterate live on the hardware until it works. Here is the prompt:
$ THE PROMPT $
Alright Wingman, special objective. I'm building a warden that catches me picking up my phone while I work, and it all has to run on the Metis card, with nothing leaving the box. Let's build it as a three-model cascade. I want to walk through each stage live on the hardware.
Stage 1: The skeleton
Run a YOLO pose model on the full webcam frame. I need two things from it:
- The person's wrist keypoints, with confidence scores, so I can tell whether a hand is reaching for something.
- The head keypoints (eyes, nose, and ears), with confidence scores, which I'll use to crop the face for Stage 3 so I don't need a separate face detector.
Stage 2: The phone
Run a COCO object detector on the same frame, filtered to only the cell phone class. Return the phone's bounding box and confidence score.
Then determine whether the phone is actually in a hand by checking the proximity of the wrist keypoints to the phone bounding box. A phone lying on the desk should not trigger a positive result.
Stage 3: The gaze
This model is not in the model zoo. I'm providing MobileGaze, a lightweight MobileNet-backbone gaze estimator, as an ONNX file using opset 17.
Compile it for the Metis datapath and report clearly:
- Which layers, if any, fall back to the host.
- Any unsupported operations or other issues that could prevent clean execution.
Before making any assumptions, ask me for:
- The MobileGaze input shape.
- The gaze-bin geometry, if the model outputs angle bins.
Crop the face using the Stage 1 head keypoints, then feed the crop into MobileGaze. Return gaze pitch and yaw.
If the model outputs angle bins instead of direct angles, perform the softmax-expectation decoding on the host rather than the AIPU.
The compiler strips preprocessing, so explicitly define the face-crop resize and normalization steps as pipeline operators.
Verdict
Combine the three signals into a single boolean verdict.
Mark me as distracted when:
- The phone is in my hand; or
- My gaze is angled downward and away from the screen, toward the phone's location.
Ideally, both signals should agree.
Add a temporal threshold or cooldown so that a brief two-second glance does not trigger a conviction. The distracted state should require sustained evidence.
Consequence
When the verdict changes to distracted, flash an LED. Implement this as a simple GPIO or serial trigger from the final event so it can later be replaced with another output device.
Walk me through each stage as we build it on the device. Also estimate the frame rate I can expect while running all three models together.
ILLUSTRATION:
This is the hardware sequel to Phone Jail (https://github.com/ehxuban11/vercel-hackathon), which won 2nd place at the Vercel London hackathon.
