SideBySide

Hi, I'm Kilian. I work in AI research, mostly on computer vision, at a European startup. A little while ago, I had to have a conversation with someone who spoke a language I don't understand. We ended up relying on a phone translator, and it worked, but the interaction felt strangely broken: you look down at a screen, wait, read or listen, then look back up. You lose a lot of the natural contact that makes a conversation feel human.

That gave me the idea for a translator that almost disappears from the interaction: the translation would be played continuously through bone-conduction headphones, so you can still hear the person in front of you and your surroundings normally, without looking at a screen or feeling isolated by earbuds. I want to build a fully offline speech-to-speech translator with low latency.

The idea is simple: one person speaks, the system translates continuously, and the other person hears the result through open-ear headphones. No phone to look at, no internet, and ideally no need to wait until the end of a sentence before hearing the translation.

The interesting part for me is the TTS. I went through the Voyager documentation and also found a February 2026 community post asking whether STT and TTS models are supported on Metis. The answer from Axelera was clear: “Metis is mainly designed for vision workloads today, and audio models are not a natively supported use case.”

Still, looking at the supported ONNX operators, I think it is worth testing properly.

Voyager already supports many of the operations used by modern speech models: MatMul, Gemm, Softmax, LayerNorm, Conv, ConvTranspose, GELU, Sigmoid, reshaping and slicing operations. The main question is therefore not simply "does Metis support TTS?", but rather how much of a real TTS graph can actually be mapped to the AIPU, and what blocks the rest?

My first candidate would be Pocket TTS from Kyutai. It is small, streaming, supports French and English, and can condition on a short speaker reference. Its architecture is also interesting for Metis because the main generation path is Transformer-based and the audio codec contains a lot of convolutional computation.

I would not try to compile the whole model at once. I would split it into smaller inference components, starting with the most expensive generation block, export fixed-shape ONNX graphs, and use Wingman to inspect the graph against Voyager's supported operations.

The goal would be to answer a few concrete questions:

• which parts run directly on Metis, which operations fall back to the CPU
• whether some unsupported operations can be rewritten with supported equivalents
• whether INT8 or mixed-precision quantization keeps the audio quality acceptable
• Most importantly, whether moving part of the model to Metis actually improves latency or CPU usage.

If the full TTS pipeline cannot run on Metis, I would still consider partial acceleration useful. The generation model could run on Metis while the audio decoder stays on the CPU.

The complete translator would look like this: 

microphone -> streaming ASR -> translation -> streaming TTS -> open-ear headphones

For the first version, I would keep Zipformer ASR on the host CPU. Zipformer uses some less standard components such as BiasNorm and Swoosh activations, so I think it is a less obvious first target for the Metis compiler.

Translation is the safer part. Voyager already provides Llama 3.2 1B for Metis, so I would keep it loaded on the AIPU and use it as a constrained French-English translator.

The pipeline should run continuously rather than sequentially: 

listen N+1 | translate N | speak N-1

The main latency problem is deciding when an ASR fragment is stable enough to send to translation. Waiting for full sentences would make the system feel slow even if each model is fast. I would therefore measure ASR commit delay, translation time-to-first-token, TTS time-to-first-audio and total speech-to-speech lag separately.

A realistic first target is sub-second perceived delay once the pipeline is warm. I would keep 300 ms as a stretch target for the compute path after an ASR fragment has been committed, not as a promise for the full microphone-to-ear latency.

For Wingman, the first task would be very focused: take Pocket TTS, inspect the exported ONNX graph, compare every operator with Voyager's current support, compile the largest useful subgraph, then benchmark it against the CPU version. If it works, move to the next TTS component. If it does not, I want a clear operator-level explanation of why.

And if TTS turns out to be a bad fit for Metis today, the project still works:

Zipformer [CPU] -> Llama 3.2 1B [METIS] -> Pocket TTS [CPU]

So there are really two outputs I care about: a working offline speech translator, and a proper answer to a more experimental question:

How far can Voyager push an audio workload onto hardware that was originally designed for vision