Unanswered questions

Hello,I am trying to compile an IBM Granite embedding model for the Metis AIPU.Current test model:```textibm-granite/granite-embedding-311m-multilingual-r2https://huggingface.co/ibm-granite/granite-embedding-311m-multilingual-r2```My final target model is:```textibm-granite/granite-switch-4.1-3b-previewhttps://huggingface.co/ibm-granite/granite-switch-4.1-3b-preview```Target hardware and software:```textMetis SBC16 GB board RAM4 GB AIPUPython 3.12Voyager SDK 1.6.0```I tried to compile the embedding model with `axcompile`, but it fails during quantization.Command:```bashaxcompile \  --input granite_embedding_metis_work/granite_embedding_311m_static.onnx \  --config granite_embedding_metis_work/granite_embedding_metis_4gb.json \  --output granite_embedding_metis_work/axcompile_out \  --overwrite \  --dataset-len 2 \  --log-level DEBUG \  --quantize-only```The ONNX export succeeds.Current ONNX details:```textONNX opset: 17Input: inputs_embeds [1, 128, 768]Output: embeddings [1, 768]```The

Hi,We are using the Axelera Metis PCIe AI accelerator card with a ZCU102 evaluation board as the host system. We are facing a critical compatibility issue when running the Voyager SDK on the ZCU102.---Setup Details:- Host Board: AMD Xilinx ZCU102 (Zynq UltraScale+ MPSoC)- Host CPU: Quad-core ARM Cortex-A53 (ARMv8.0-A)- OS: Custom Yocto Linux (kernel 6.12.10-xilinx)- Axelera Card: Metis PCIe AI Accelerator- Voyager SDK Version: Latest (installed via install.sh --no-driver --media)- metis Driver Version: 1.4.16 (loads successfully)---Issue:When running inference.py or importing axelera.app, the process crashes immediately with:  Illegal instruction (core dumped)  SIGILL (signal 4)---Root Cause Identified:After detailed debugging, we found that the Voyager SDK Python packages (scipy, torch, cv2, grpc, onnx etc.) are compiled with LSE (Large System Extensions) atomic instructions such as ldadd, stadd, swpa, cas.The ZCU102 Cortex-A53 is ARMv8.0-A which does NOT support LSE instructions. LSE

Hello Axelera support team,I am running the Axelera Metis M.2 accelerator on a Radxa Rock 5B (RK3588 SoC) with Armbian and kernel 6.18.10-current-rockchip64. After extensive debugging over several days, I have identified a compatibility issue between the metis driver and the ARM SMMU-v3 on this platform. I would appreciate your guidance on resolving this.HARDWARE & SOFTWAREBoard: Radxa Rock 5B (RK3588, 8GB RAM, PCIE3.0-4X) OS: Armbian, kernel 6.18.10-current-rockchip64 (mainline) Accelerator: Axelera Metis M.2 Driver version: metis 1.4.16 SDK/Runtime version: voyager-sdk / runtime-1.6.0-1 PSU: 65WWHAT WORKSThe Metis is detected correctly via PCIe: lspci shows "Axelera AI Metis AIPU (rev 02)" at 0000:01:00.0 After patching the Device Tree to expand the PCIe memory window (the default 14MB window was too small; expanded the 64-bit prefetchable range at 0x900000000 to 128MB), the driver loads successfully: /dev/metis0 -> metis-0:1:0 is created "Axelera AIPU PCIe Driver, version 1.

Dear Axelera Support,I am currently following the Voyager SDK quick start guide (YOLO inference demo) on an RK3588-based Compute Board (received out-of-the-box), using Voyager SDK version 1.5.3. However, I am unable to successfully run inference due to persistent device communication issues with the Metis AIPU.--------------------------------------------------System Overview- Platform: RK3588 (Compute Board, out-of-the-box setup)- Voyager SDK: 1.5.3 (and also tested with 1.4.0)- Connection: PCIe (Metis AIPU detected as 01:00.0 [1f9d:1100])--------------------------------------------------Initial IssueRunning the quick start inference:./inference.py yolov5s-v7-coco usb:/dev/video1Result:ERROR: No devices foundAt that moment:- lspci returned no devices, indicating a PCIe enumeration issue.--------------------------------------------------After Cold BootAfter performing a full power cycle:- lspci correctly shows:  01:00.0 Processing accelerators: Axelera AI Metis AIPU (rev 02) [1f9d:1100]

Hello,There seems to be two different published benchmarks for ResNet50.The first is in the Voyager SDK’s documentation : 1756 FPS (M.2) / 1946 FPS (PCIe) :- https://github.com/axelera-ai-hub/voyager-sdk/blob/latest/docs/reference/model_zoo.mdThe second is on the Metis Marketing page : Up to 3200 frames per second (ResNet-50)- https://axelera.ai/ai-accelerators/aipu/metis I was able to achieve 2050 FPS, which is higher than the modelzoo documentation, but MUCH lower than the marketing claim:- https://mariobergeron.com/posts/edge-ai-power-p04-axelera-metis/ Where does the 3200 FPS benchmark claim come from ?   How was this benchmark achieved on Metis ?  Which hardware platform ?Which brings me to the next marketing claim : Industry-Leading FPS / Watt ratios on ResNet50v1 (graph showing 614 FPS/W- https://axelera.ai/ai-accelerators/aipu/metis I was only able to achieve 270 FPS/W :- https://mariobergeron.com/posts/edge-ai-power-p04-axelera-metis/ What does the 614 FPS/W benchmark come fro

My colleagues don't remember the Ubuntu password and I have to use the miniPC. I have tried some passwords they have given me, but no luck.

Hi,How can I implement the following preprocessing to run the end-to-end C/C++ pipeline?I've already implemented the build_gst function of the custom decoder with C++.def override_preprocess(self, img: PIL.Image.Image | np.ndarray) -> torch.Tensor:#trasformazione immagineimg = np.array(img)if img.ndim == 2:img = cv2.cvtColor(img, cv2.COLOR_GRAY2RGB)else:img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) img = np.float32(img/255.)#Stima rumoresigma_est = estimate_sigma(img, channel_axis=-1, average_sigmas=True)sigma_est = np.sqrt(sigma_est)sigma_est = min(sigma_est, 0.08) img = cv2.resize(img, (640,480), interpolation=cv2.INTER_LINEAR)img_tensor = torch.from_numpy(np.ascontiguousarray(img)).permute(2, 0, 1).float()noise_tensor = torch.full_like(img_tensor[:1, :, :], fill_value=sigma_est, dtype=torch.float32)final_tensor = torch.cat([img_tensor, noise_tensor], dim=0).to(torch.float32)return final_tensor

I honestly proud for the AI community in that they will be good functions and I creatively appreciate for all AI humans to create some earning benefits thanks for a lot of 💝

I am currently working with metis M2 card. Trying to run yolov8 model using inference.py file from the sdk. The throughput is very low. I set the AIPU core to 4 and verfied the same in the compile_config.json file. But there is another argument called multicore_mode("multicore_mode": "batch"). what exactly does the multi core mode do? and is there way to set it to other option than ‘batch’?

Hi, I was trying to get the voyager sdk v1.6.0 running on the arduino portenta board but I cannot get it to work. The board came with v1.4.0 preinstalled, which does work. I followed the guide for creating the docker container and used the install script with the no-driver flag to install v1.6.0, but every time I run axdevice I get the folowing error:axdevice[libaxldev_linux.c:1030] AXL_IOCTL_GET_RESOURCE_INFO failed: Inappropriate ioctl for device[AxeleraDevice.cpp:75] Device not found: metis-0:1:0WARNING: Failed to open device 0(metis-0:1:0)[libaxldev_linux.c:1030] AXL_IOCTL_GET_RESOURCE_INFO failed: Inappropriate ioctl for device[AxeleraDevice.cpp:75] Device not found: metis-0:1:0ERROR: Failed to open device metis-0:1:0ERROR: AXR_ERROR_CONNECTION_ERROR: Failed to open device metis-0:1:0I thought it might be due to the kernel driver version outside the container but the latest version I could find for the portenta is v1.4.0. I also tried using the v1.6.0 venv in the v1.4.0 container

I'm trying to deploy yolov8m-worldv2 on the Metis M.2, but it fails due to an error.Does Metis support yolov8m-worldv2?Is it because it's a VLM model that Metis can't deploy it?I would like to use a zero-shot model.Thank you for your support.

​ I have a 4 AIPU Axelera card, and it is plugged into the Dell PowerEdge R760 server.Currently the card fails to enumerate all the Metis devices, and I see PCIE AER errors in the logs relatd to the Metis devices.Below is the the output of the axdevice command. Attached problem report as well.Have you seen this issue before ? Command line logs:(axelera-env) user@r760-sales-demo:~$ sudo lspci -d 1f9d: -vt-[0000:0c]---01.0-[0d-12]----00.0--+-02.0-[11]----00.0 Axelera AI Metis AIPU (rev 02) \-03.0-[12]----00.0 Axelera AI Metis AIPU (rev 02)(axelera-env) user@r760-sales-demo:~$ cat /proc/cmdlineBOOT_IMAGE=/boot/vmlinuz-6.8.0-111-generic root=UUID=5c7f0039-5e33-4e4b-85e3-9d0215b77b59 ro iommu=pt loglevel=8(axelera-env) user@r760-sales-demo:~$ axdeviceWARNING: 4PCI device count mismatch: lspci=2, triton=0ERROR: min() iterable argument is empty(axelera-env) user@r760-sales-demo:~$ axdevice -vINFO: Found AIPU driver: metis 208896 0WARNING: 4

Hi,I’m interested to know if there is any way to measure AIPU power usage on Metis-equipped devices. The idea would be to measure yolo inference power usage on AIPU. 

Hello Axelera team and community,We are running a Metis PCIe card (16GB, rev 02) in a Proxmox VE 9.1.9 cluster with VFIO PCIe passthrough into a KVM guest. We understand this is listed as Beta in SDK 1.6, and we want to share detailed diagnostics to help improve it.Environment:Host: Proxmox VE 9.1.9, kernel 7.0.0-3-pve (Linux 6.14), Intel VT-d enabled (intel_iommu=on iommu=pt) Metis firmware: flver=1.6.0, bcver=7.4 DKMS driver: metis-dkms=1.4.16 SDK: axelera-rt==1.6.0, axelera-runtime==1.6.0 Guest: Ubuntu 24.04.3, kernel 6.8.0-111-generic IOMMU group 22: only the Metis card (isolated, ideal for passthrough)What works (bare metal on Proxmox host):axelera.runtime Python API works perfectly on the host:ResNet18: 1.29ms per inference RetinaFace: 5.32ms per inference, face detected with confidence=1.000 load_model_instance completes instantlyWhat fails (inside KVM guest via VFIO):conn.load_model_instance(model, num_sub_devices=1, aipu_cores=4) fails:  [libaxldev_linux.c:675] DMA_GET_XFER_SY

I would like to use the Metis M.2 in an embedded system running Open Embedded with a 4.19 kernel. (It doesn’t have the “modern” /dev/dma_heap/ interface, it uses /dev/ion). I have the driver installed in the host, and when I use the SDK in a Ubuntu 22.04 docker container I get this error:[libaxldev_linux.c:394] No dmabuf driver found[ERROR][axeShareMemAllocMap]: Fail to alloc dmabuf[ERROR][axeMemAllocShared]: Share memory allocate mao failed: 0x70010001.terminate called after throwing an instance of 'std::runtime_error'  what():  axr_load_model_instance failed : Failed to create executor for model: kernel_function_batchedAborted Can the driver/library be compiled to use ION instead of dmabuf?

I created a custom model based on yolo26n, 'yolo26n-aya'.The performance results for source code (bitmap (256x256) images and mp4 video) are shown below.(a) images    (venv) ubuntu@antelao-3588:~/voyager-sdk$ AXELERA_USE_CL_DOUBLE_BUFFER=0 ./inference.py yolo26n-aya ./data/aya100/images --show-stats --no-display    ========================================================================    Element                                         Time(??s)   Effective FPS    ========================================================================    axinplace-addstreamid0                               141         7,076.4    inference-task0:libtransform_resize_cl_0             371         2,692.7    inference-task0:libtransform_padding_0               844         1,184.0    inference-task0:inference                          2,961           337.7    inference-task0:Inference latency                208,874             n/a    inference-task0:libdecode_yolov8_0                   653         1,531.2  

Title says it all -- any timeline for a “wrynose” release for the M.2 card so it can be included in a wrynose Yocto Project build?

I'm trying to do a pci passthrough from proxmox to my VM.I keep getting the error when starting my VM. error writing '1' to '/sys/bus/pci/devices/0000:02:00.0/reset': Inappropriate ioctl for devicefailed to reset PCI device '0000:02:00.0', but trying to continue as not all devices need a resetkvm: ../hw/pci/pci.c:1815: pci_irq_handler: Assertion `0 <= irq_num && irq_num < PCI_NUM_PINS' failed.TASK ERROR: start failed: QEMU exited with code 1This is my machine details agent: 1bios: ovmfboot: order=scsi0;ide2;net0cores: 4cpu: x86-64-v2-AESefidisk0: local-lvm:vm-103-disk-0,efitype=4m,ms-cert=2023,pre-enrolled-keys=1,size=4Mhostpci0: 0000:02:00.0,pcie=1,rombar=1ide2: none,media=cdrommachine: q35memory: 4096meta: creation-qemu=10.1.2,ctime=1776892906name: axeleranet0: virtio=BC:24:11:A4:8C:9B,bridge=vmbr0,firewall=1numa: 0ostype: l26scsi0: local-lvm:vm-103-disk-1,iothread=1,size=32Gscsihw: virtio-scsi-singlesmbios1: uuid=49dcbf39-39e5-4a39-a8ad-cc02f5c886bfsockets: 1vmgenid: 9

SummaryVoyager SDK 1.6.0 quantizer rejects ONNX models that have a fan-out fork close tothe input (e.g., a tensor consumed by multiple parallel paths early in the graph)with the error:axelera.compiler.exceptions.QuantizationError: Input model cannot be quantized because the core graph has several inputs that are not currently supported by Quantizer.