Custom model deployement C++ Windows

Hi there ​@gma, welcome on board!

Well, I think your install looks fine, so that’s some good news out of the way 😄. The reason you can't find any .lib files is that the native Windows SDK currently supports inference only, through the AxRuntime API, and the examples for that path are Python based. Full SDK development, model deployment and the GStreamer plugins (like AxInferenceNet) are Linux only right now. There's a feature comparison table in the Windows installation guide with a bit more info on this, which is pretty useful here, I think.

One quick test to confirm the runtime and card are working on your system. From your activated Windows venv:

axdownloadmodel resnet50-imagenet-onnx
axrunmodel C:\Axelera\voyager-sdk\build\resnet50-imagenet-onnx\resnet50-imagenet-onnx\1\model.json

(Adjust the path to wherever you cloned the SDK, of course.) That runs a prebuilt model through the runtime and prints performance stats, which should prove the Windows stack end to end.

Let me know if/how that works out, and we can go from there 👍

Hi, 
Thank you for your answer. I will try to use python and the AxRuntime API. 
When following the Windows installation guide I get the following error:

PS C:\Axelera\voyager-sdk> .\venv-win\Scripts\Activate.ps1
(venv-win) PS C:\Axelera\voyager-sdk> pip install --extra-index-url https://software.axelera.ai/artifactory/api/pypi/axelera-pypi/simple axelera-runtime==1.6.1 axelera-llm==1.6.1 axelera-types==1.6.1
Looking in indexes: https://pypi.org/simple, https://software.axelera.ai/artifactory/api/pypi/axelera-pypi/simple
ERROR: Could not find a version that satisfies the requirement axelera-runtime==1.6.1 (from versions: 1.6.0)

I am only able to install version 1.6.0 (but windows packages are 1.6.1). 
With version 1.6.0 the Axelera card is well detected: 

(venv-win) PS C:\Axelera\voyager-sdk> axdevice
Device 0: metis-0:0:0 1GiB m2 flver=1.3.2 bcver=1.4 clock=800MHz(0-3:800MHz) mvm=0-3:100%

But running a model does not work: 
 

(venv-win) PS C:\Axelera\voyager-sdk> axrunmodel C:\Axelera\voyager-sdk\build\resnet50-imagenet-onnx\resnet50-imagenet-onnx\1\model.json
C:\Axelera\voyager-sdk\build\resnet50-imagenet-onnx\resnet50-imagenet-onnx\1\model.json,[ERROR][waitForQueueBinaryCompletion]: Wait kernel failed with return code 1.
[libaxldev.c:140] Failed to wait for MSI 1: No error
[ERROR][waitForQueueBinaryCompletion]: Wait kernel failed with return code -1.
FAIL: exception: access violation reading 0x0000004E6B060000

Thank you.

Hi ​@gma ! Hmm, weird about v1.6.1, but if it’s working well in v1.6 let’s look there first. At least we know the card’s fine!

Looks like the model loaded and started, then fell over at the point where the runtime waits for the per-core completion interrupt to come back, which is what the Failed to wait for MSI and the access violation point toward.

One quick thing to narrow it down is to maybe try the same model pinned to a single core:

axrunmodel --aipu-cores=1 C:\Axelera\voyager-sdk\build\resnet50-imagenet-onnx\resnet50-imagenet-onnx\1\model.json

If it runs clean on one core and only fails when more are involved, that could suggest it's down to how the interrupts are being delivered on this host rather than anything wrong with your model or card. Let me know how that goes and we'll take it from there 👍

Hi ​@Spanner r,

I tested with:

axrunmodel --aipu-cores=1 C:\Axelera\voyager-sdk\build\resnet50-imagenet-onnx\resnet50-imagenet-onnx\1\model.json

and the model runs successfully : 

(venv-win) PS C:\Axelera\voyager-sdk> axrunmodel --aipu-cores=1 C:\Axelera\voyager-sdk\build\resnet50-imagenet-onnx\resnet50-imagenet-onnx\1\model.json
C:\Axelera\voyager-sdk\build\resnet50-imagenet-onnx\resnet50-imagenet-onnx\1\model.json,dev:412.7 host:382.4 system:375.5 latency:14.5ms frames:3759 temp:39C

This seems consistent with what you mentioned regarding interrupt delivery.

Are there any additional diagnostics I can run to verify whether this is a Windows driver/MSI issue? Also, is multi-core execution currently expected to work on Windows with runtime 1.6.0, or could this be related to the fact that only axelera-runtime 1.6.0 is available while the Windows packages appear to be 1.6.1?

Thank you.