Low Utilization on Portenta

Question

Hi, I have a custom U-Net model with MobileNetV2 encoder running on the Axelera portenta with voyager v1.4.0. I’ve based my code for inference on the axruntime python example in the Voyager github repository, with the model being compiled by the Voyager compiler. When running inference the maximum utilization of the Metis, as shown by axmonitor, only reaches 6% on all four cores.

I have tried running it with all pre-processing and post-processing commented out and a dummy input consisting of all zeroes, but utilization also did not surpass 6%.

I’m wondering whether this utilization limit might be caused by the way the model is compiled, perhaps due to needing multiple iterations in the LowerTIR stage, and whether there might be more optimal compiler configs to achieve a higher utilization.

I have included some logging / debug files the CLI compiler generated for an older version of my model. For my current model I’m using the API compiler but it does not generate these files, the only differences between these model versions is that the model is a binary classifier now instead of 2 classes and the quantization process is more accurate. The 6% utilization is for both the 2 class and binary class model.

Spanner · Answer

Hi ​@nick!Hmm, with utilisation at ~6% even with pre/post-processing removed and a dummy zero input, that looks like it rules out the host pipeline and points at the compile config. Good work there.By default Voyager compiles models for single-core execution if I recall, so only one core does work while axmonitor reports per-core usage across all four (which is consistent with what you're seeing).How about recompiling for multi-core (Batch-1), so the runtime instantiates a copy per core and dispatches frames across all four?compilation_config:  aipu_cores_used: 1  resources_used: 0.25   # 1.0 / 4 cores(The same settings can be passed via CompilerConfig with the Python API.)If you specifically want lowest single-stream latency on one heavy model, try Batch-4 instead (aipu_cores_used: 4, resources_used: 1.0).Let me know if this moved the needle at all!

Sign up

Log in, or create an Axelera AI account

Login to the community

Log in, or create an Axelera AI account