Skip to main content
Question

System Freezes while running inference on Metis M.2

  • April 4, 2026
  • 2 replies
  • 72 views
R
Cadet

I am using Radxa Rock 5B (RK3588, 8GB RAM) board along with Metis M.2 accelerator. I have successfully done the initial troubleshooting for the RK3588 during installation (memory expansion in dts, etc.) and installed the driver. However, the Metis board is not working as expected. 

The board is detected during lspci scan:

reck@reck:~$ lspci
0000:00:00.0 PCI bridge: Rockchip Electronics Co., Ltd RK3588 (rev 01)
0000:01:00.0 Processing accelerators: Axelera AI Metis AIPU (rev 02)
0002:20:00.0 PCI bridge: Rockchip Electronics Co., Ltd RK3588 (rev 01)
0002:21:00.0 Network controller: Realtek Semiconductor Co., Ltd. RTL8852BE PCIe 802.11ax Wireless Network Controller
0004:40:00.0 PCI bridge: Rockchip Electronics Co., Ltd RK3588 (rev 01)
0004:41:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
reck@reck:~$ lspci -s 0000:01:00.0 -vv 
0000:01:00.0 Processing accelerators: Axelera AI Metis AIPU (rev 02)
	Subsystem: Axelera AI Metis AIPU (rev 02)
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 211
	Region 0: Memory at f5010000 (64-bit, non-prefetchable) [size=4K]
	Region 2: Memory at f6000000 (32-bit, non-prefetchable) [size=32M]
	Expansion ROM at f5000000 [virtual] [disabled] [size=64K]
	Capabilities: <access denied>
	Kernel driver in use: axl
	Kernel modules: metis


However, running axdevice or example inference causes a completely unrecoverable system freeze. Running triton_multi_ctx --fwver command results in bus error.

(venv) reck@reck:~/project/axl/voyager-sdk$ triton_multi_ctx --fwver
Bus error

 

(venv) reck@reck:~/project/axl/voyager-sdk$ triton_dma
Found 1 devices
Device : metis-0:1:0
Bus error


How should I fix this? Please help me run this. 
 

    2 replies

    Spanner
    Axelera Team
    Forum|alt.badge.img+3
    • SpannerAxelera Team
    • Axelera Team
    • April 7, 2026

    Hi ​@reck ! Nice work on the DTS memory expansion - that looks solid!

    The bus error reminds me of things we’ve seen on other RK3588 boards. I think the OS and kernel version are pretty critical here, so if you could share those, we can dig deeper:

    1. Your OS and kernel version — uname -a and lsb_release -a
    2. Which Voyager SDK version you're running
    3. The output of sudo dmesg | grep -iE "axl|pci|metis"

    👍

    R
    Cadet
    • reck
    • Author
    • Cadet
    • April 7, 2026

    Thanks ​@Spanner for your reply! 
    Here are the requested details: 
    Kernel Version

    reck@reck:~/project/axl/voyager-sdk$ uname -a
    Linux reck 5.10.0-1012-rockchip #12-Ubuntu SMP Wed Aug 14 22:22:22 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

    OS Version: 

    reck@reck:~/project/axl/voyager-sdk$ lsb_release -a
    No LSB modules are available.
    Distributor ID:	Ubuntu
    Description:	Ubuntu 22.04.5 LTS
    Release:	22.04
    Codename:	jammy

    I am using the Josua Riek’s Rock5B Ubuntu 22.04 image. I am using the latest version of the Voyager SDK (v1.5.3). 
     

    reck@reck:~/project/axl/voyager-sdk$ sudo dmesg | grep -iE "axl|pci|metis"
    [sudo] password for reck: 
    [   11.590650] PCI/MSI: /interrupt-controller@fe600000/msi-controller@fe640000 domain created
    [   11.590675] PCI/MSI: /interrupt-controller@fe600000/msi-controller@fe660000 domain created
    [   12.621542] reg-fixed-voltage vcc3v3-pcie2x1l0: Looking up vin-supply from device tree
    [   12.621547] vcc3v3_pcie2x1l0: supplied by vcc5v0_sys
    [   12.677674] vcc3v3_pcie2x1l0: 3300 mV, enabled
    [   12.677750] reg-fixed-voltage vcc3v3-pcie2x1l0: vcc3v3_pcie2x1l0 supplying 3300000uV
    [   12.677805] vcc3v3_pcie30: 3300 mV, disabled
    [   12.677854] reg-fixed-voltage vcc3v3-pcie30: Looking up vin-supply from device tree
    [   12.677859] vcc3v3_pcie30: supplied by vcc5v0_sys
    [   12.677890] reg-fixed-voltage vcc3v3-pcie30: vcc3v3_pcie30 supplying 3300000uV
    [   13.163307] PCI: CLS 0 bytes, default 64
    [   13.573573] rk-pcie fe150000.pcie: invalid prsnt-gpios property in node
    [   13.573584] rk-pcie fe150000.pcie: Looking up vpcie3v3-supply from device tree
    [   13.573717] rk-pcie fe170000.pcie: invalid prsnt-gpios property in node
    [   13.573728] rk-pcie fe170000.pcie: Looking up vpcie3v3-supply from device tree
    [   13.579553] rk-pcie fe170000.pcie: IRQ msi not found
    [   13.579575] rk-pcie fe170000.pcie: use outband MSI support
    [   13.579582] rk-pcie fe170000.pcie: Missing *config* reg space
    [   13.579608] rk-pcie fe170000.pcie: host bridge /pcie@fe170000 ranges:
    [   13.579634] rk-pcie fe170000.pcie:      err 0x00f2000000..0x00f20fffff -> 0x00f2000000
    [   13.579651] rk-pcie fe170000.pcie:       IO 0x00f2100000..0x00f21fffff -> 0x00f2100000
    [   13.579671] rk-pcie fe170000.pcie:      MEM 0x00f2200000..0x00f2ffffff -> 0x00f2200000
    [   13.579683] rk-pcie fe170000.pcie:      MEM 0x0980000000..0x09bfffffff -> 0x0980000000
    [   13.579721] rk-pcie fe170000.pcie: Missing *config* reg space
    [   13.579763] rk-pcie fe170000.pcie: invalid resource
    [   13.590376] rk-pcie fe150000.pcie: IRQ msi not found
    [   13.590404] rk-pcie fe150000.pcie: use outband MSI support
    [   13.590411] rk-pcie fe150000.pcie: Missing *config* reg space
    [   13.590437] rk-pcie fe150000.pcie: host bridge /pcie@fe150000 ranges:
    [   13.590461] rk-pcie fe150000.pcie:      err 0x00f0000000..0x00f00fffff -> 0x00f0000000
    [   13.590479] rk-pcie fe150000.pcie:       IO 0x00f0100000..0x00f01fffff -> 0x00f0100000
    [   13.590498] rk-pcie fe150000.pcie:      MEM 0x00f5000000..0x00f7ffffff -> 0x00f5000000
    [   13.590514] rk-pcie fe150000.pcie:      MEM 0x0900000000..0x093fffffff -> 0x0900000000
    [   13.590544] rk-pcie fe150000.pcie: Missing *config* reg space
    [   13.590586] rk-pcie fe150000.pcie: invalid resource
    [   13.796791] rk-pcie fe150000.pcie: PCIe Linking... LTSSM is 0x0
    [   13.822316] rk-pcie fe150000.pcie: PCIe Linking... LTSSM is 0x1
    [   13.842329] rk-pcie fe170000.pcie: PCIe Link up, LTSSM is 0x30011
    [   13.842535] rk-pcie fe170000.pcie: PCI host bridge to bus 0002:20
    [   13.842549] pci_bus 0002:20: root bus resource [bus 20-2f]
    [   13.842559] pci_bus 0002:20: root bus resource [??? 0xf2000000-0xf20fffff flags 0x0]
    [   13.842571] pci_bus 0002:20: root bus resource [io  0x0000-0xfffff] (bus address [0xf2100000-0xf21fffff])
    [   13.842580] pci_bus 0002:20: root bus resource [mem 0xf2200000-0xf2ffffff]
    [   13.842589] pci_bus 0002:20: root bus resource [mem 0x980000000-0x9bfffffff pref]
    [   13.842635] pci 0002:20:00.0: [1d87:3588] type 01 class 0x060400
    [   13.842666] pci 0002:20:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
    [   13.842747] pci 0002:20:00.0: supports D1 D2
    [   13.842756] pci 0002:20:00.0: PME# supported from D0 D1 D3hot
    [   13.848996] rk-pcie fe150000.pcie: PCIe Linking... LTSSM is 0x1
    [   13.856993] pci 0002:20:00.0: Primary bus is hard wired to 0
    [   13.857005] pci 0002:20:00.0: bridge configuration invalid ([bus 01-ff]), reconfiguring
    [   13.857273] pci 0002:21:00.0: [10ec:b852] type 00 class 0x028000
    [   13.857356] pci 0002:21:00.0: reg 0x10: initial BAR value 0x00000000 invalid
    [   13.857365] pci 0002:21:00.0: reg 0x10: [io  size 0x0100]
    [   13.857464] pci 0002:21:00.0: reg 0x18: [mem 0x00000000-0x000fffff 64bit]
    [   13.857961] pci 0002:21:00.0: PME# supported from D0 D3hot D3cold
    [   13.875658] rk-pcie fe150000.pcie: PCIe Linking... LTSSM is 0x0
    [   13.879550] pci_bus 0002:21: busn_res: [bus 21-2f] end is updated to 21
    [   13.879591] pci 0002:20:00.0: BAR 8: assigned [mem 0xf2200000-0xf22fffff]
    [   13.879604] pci 0002:20:00.0: BAR 6: assigned [mem 0xf2300000-0xf230ffff pref]
    [   13.879615] pci 0002:20:00.0: BAR 7: assigned [io  0x1000-0x1fff]
    [   13.879631] pci 0002:21:00.0: BAR 2: assigned [mem 0xf2200000-0xf22fffff 64bit]
    [   13.879688] pci 0002:21:00.0: BAR 0: assigned [io  0x1000-0x10ff]
    [   13.879711] pci 0002:20:00.0: PCI bridge to [bus 21]
    [   13.879722] pci 0002:20:00.0:   bridge window [io  0x1000-0x1fff]
    [   13.879731] pci 0002:20:00.0:   bridge window [mem 0xf2200000-0xf22fffff]
    [   13.883357] pcieport 0002:20:00.0: PME: Signaling with IRQ 149
    [   13.902321] rk-pcie fe150000.pcie: PCIe Linking... LTSSM is 0x0
    [   13.910402] ehci-pci: EHCI PCI platform driver
    [   13.928987] rk-pcie fe150000.pcie: PCIe Linking... LTSSM is 0x0
    [   13.955652] rk-pcie fe150000.pcie: PCIe Linking... LTSSM is 0x0
    [   13.982322] rk-pcie fe150000.pcie: PCIe Linking... LTSSM is 0x0
    [   14.008985] rk-pcie fe150000.pcie: PCIe Linking... LTSSM is 0x0
    [   14.035655] rk-pcie fe150000.pcie: PCIe Linking... LTSSM is 0x0
    [   14.341296] vcc3v3_pcie2x1l2: 3300 mV, enabled
    [   14.341537] reg-fixed-voltage vcc3v3-pcie2x1l2: Looking up vin-supply from device tree
    [   14.341567] vcc3v3_pcie2x1l2: supplied by vcc_3v3_s3
    [   14.341919] reg-fixed-voltage vcc3v3-pcie2x1l2: vcc3v3_pcie2x1l2 supplying 3300000uV
    [   14.356218] rk-pcie fe190000.pcie: invalid prsnt-gpios property in node
    [   14.356260] rk-pcie fe190000.pcie: Looking up vpcie3v3-supply from device tree
    [   14.357994] rk-pcie fe190000.pcie: IRQ msi not found
    [   14.358093] rk-pcie fe190000.pcie: use outband MSI support
    [   14.358099] rk-pcie fe190000.pcie: Missing *config* reg space
    [   14.358194] rk-pcie fe190000.pcie: host bridge /pcie@fe190000 ranges:
    [   14.358222] rk-pcie fe190000.pcie:      err 0x00f4000000..0x00f40fffff -> 0x00f4000000
    [   14.358239] rk-pcie fe190000.pcie:       IO 0x00f4100000..0x00f41fffff -> 0x00f4100000
    [   14.358262] rk-pcie fe190000.pcie:      MEM 0x00f4200000..0x00f4ffffff -> 0x00f4200000
    [   14.358274] rk-pcie fe190000.pcie:      MEM 0x0a00000000..0x0a3fffffff -> 0x0a00000000
    [   14.358321] rk-pcie fe190000.pcie: Missing *config* reg space
    [   14.358435] rk-pcie fe190000.pcie: invalid resource
    [   14.619028] rk-pcie fe190000.pcie: PCIe Link up, LTSSM is 0x130011
    [   14.619266] rk-pcie fe190000.pcie: PCI host bridge to bus 0004:40
    [   14.619279] pci_bus 0004:40: root bus resource [bus 40-4f]
    [   14.619286] pci_bus 0004:40: root bus resource [??? 0xf4000000-0xf40fffff flags 0x0]
    [   14.619298] pci_bus 0004:40: root bus resource [io  0x200000-0x2fffff] (bus address [0xf4100000-0xf41fffff])
    [   14.619303] pci_bus 0004:40: root bus resource [mem 0xf4200000-0xf4ffffff]
    [   14.619309] pci_bus 0004:40: root bus resource [mem 0xa00000000-0xa3fffffff pref]
    [   14.619354] pci 0004:40:00.0: [1d87:3588] type 01 class 0x060400
    [   14.619378] pci 0004:40:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
    [   14.619450] pci 0004:40:00.0: supports D1 D2
    [   14.619455] pci 0004:40:00.0: PME# supported from D0 D1 D3hot
    [   14.627483] pci 0004:40:00.0: Primary bus is hard wired to 0
    [   14.627491] pci 0004:40:00.0: bridge configuration invalid ([bus 01-ff]), reconfiguring
    [   14.627703] pci 0004:41:00.0: [10ec:8125] type 00 class 0x020000
    [   14.627754] pci 0004:41:00.0: reg 0x10: [io  0x0000-0x00ff]
    [   14.627812] pci 0004:41:00.0: reg 0x18: [mem 0x00000000-0x0000ffff 64bit]
    [   14.627851] pci 0004:41:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit]
    [   14.628181] pci 0004:41:00.0: supports D1 D2
    [   14.628185] pci 0004:41:00.0: PME# supported from D0 D1 D2 D3hot D3cold
    [   14.643417] pci_bus 0004:41: busn_res: [bus 41-4f] end is updated to 41
    [   14.643443] pci 0004:40:00.0: BAR 8: assigned [mem 0xf4200000-0xf42fffff]
    [   14.643450] pci 0004:40:00.0: BAR 6: assigned [mem 0xf4300000-0xf430ffff pref]
    [   14.643455] pci 0004:40:00.0: BAR 7: assigned [io  0x200000-0x200fff]
    [   14.643464] pci 0004:41:00.0: BAR 2: assigned [mem 0xf4200000-0xf420ffff 64bit]
    [   14.643499] pci 0004:41:00.0: BAR 4: assigned [mem 0xf4210000-0xf4213fff 64bit]
    [   14.643531] pci 0004:41:00.0: BAR 0: assigned [io  0x200000-0x2000ff]
    [   14.643544] pci 0004:40:00.0: PCI bridge to [bus 41]
    [   14.643549] pci 0004:40:00.0:   bridge window [io  0x200000-0x200fff]
    [   14.643555] pci 0004:40:00.0:   bridge window [mem 0xf4200000-0xf42fffff]
    [   14.645828] pcieport 0004:40:00.0: PME: Signaling with IRQ 167
    [   14.718994] rk-pcie fe150000.pcie: PCIe Link up, LTSSM is 0x230011
    [   14.719173] rk-pcie fe150000.pcie: PCI host bridge to bus 0000:00
    [   14.719183] pci_bus 0000:00: root bus resource [bus 00-0f]
    [   14.719189] pci_bus 0000:00: root bus resource [??? 0xf0000000-0xf00fffff flags 0x0]
    [   14.719195] pci_bus 0000:00: root bus resource [io  0x100000-0x1fffff] (bus address [0xf0100000-0xf01fffff])
    [   14.719200] pci_bus 0000:00: root bus resource [mem 0xf5000000-0xf7ffffff]
    [   14.719206] pci_bus 0000:00: root bus resource [mem 0x900000000-0x93fffffff pref]
    [   14.719239] pci 0000:00:00.0: [1d87:3588] type 01 class 0x060400
    [   14.719257] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
    [   14.719312] pci 0000:00:00.0: supports D1 D2
    [   14.719316] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
    [   14.727894] pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
    [   14.727978] pci 0000:01:00.0: [1f9d:1100] type 00 class 0x120000
    [   14.728032] pci 0000:01:00.0: reg 0x10: [mem 0x04380000-0x04380fff 64bit]
    [   14.728051] pci 0000:01:00.0: reg 0x18: [mem 0x08000000-0x09ffffff]
    [   14.728115] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
    [   14.728319] pci 0000:01:00.0: supports D1
    [   14.728324] pci 0000:01:00.0: PME# supported from D0 D1 D3hot
    [   14.743797] pci 0000:00:00.0: BAR 8: assigned [mem 0xf5000000-0xf7ffffff]
    [   14.743809] pci 0000:00:00.0: BAR 6: no space for [mem size 0x00010000 pref]
    [   14.743815] pci 0000:00:00.0: BAR 6: failed to assign [mem size 0x00010000 pref]
    [   14.743824] pci 0000:01:00.0: BAR 2: assigned [mem 0xf6000000-0xf7ffffff]
    [   14.743837] pci 0000:01:00.0: BAR 6: assigned [mem 0xf5000000-0xf500ffff pref]
    [   14.743844] pci 0000:01:00.0: BAR 0: assigned [mem 0xf5010000-0xf5010fff 64bit]
    [   14.743870] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
    [   14.743876] pci 0000:00:00.0:   bridge window [mem 0xf5000000-0xf7ffffff]
    [   14.746150] pcieport 0000:00:00.0: PME: Signaling with IRQ 209
    [   19.765169] metis: loading out-of-tree module taints kernel.
    [   19.767028] pci 0000:01:00.0: Found target device: AXLAIPU_OMEGA_DEVICE_ID
    [   19.767034] pci 0000:01:00.0: Found target device: 0000:01:00.0
    [   19.767039] pcieport 0000:00:00.0: Found bridge device: 0000:00:00.0
    [   19.767048] pcieport 0000:00:00.0: Bridge window: [mem 0xf5000000-0xf7ffffff]
    [   19.767053] pcieport 0000:00:00.0: Decoded memory behind bridge: f5000000-f7ffffff
    [   19.767057] pcieport 0000:00:00.0: Memory behind bridge is sufficient. Skipping reset.
    [   19.767103] axlaipu: root directory for axlaipu
    [   19.767252] axl 0000:01:00.0: enabling device (0000 -> 0002)
    [   19.767310] axl 0000:01:00.0: All AER errors masked
    [   19.767316] axl 0000:01:00.0: Memory windows prefetch 0x0000000000000000 0x0000000000000000
    [   19.767320] axl 0000:01:00.0: Memory windows no-prefetch 0x00000000f5010000 0x0000000002ff0000
    [   19.770910] axl 0000:01:00.0: Invalid hdrv magic 0
    [   19.770918] axl 0000:01:00.0: Fail to get hdrv area
    [   19.771107] axl 0000:01:00.0: Data Link Layer Link Active Reporting capability
    [   19.776928] axl 0000:01:00.0: MSI registered 32 (32)
    [   19.776939] axl 0000:01:00.0: irq vec number 210
    [   19.777745] axl 0000:01:00.0: Register directory metis-0000:01:00.0
    [   19.877470]  pci_device_probe+0xbc/0x12c
    [   19.877489]  __pci_register_driver+0x6c/0x7c

    Notice the bus overlap: 

    pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])

    which might be the possible reason for the bus error.
     

    Code of ConductAccessibility statement

    Address

    HTC5, High Tech Campus
    5656 AE Eindhoven
    The Netherlands
    Email: info@axelera.ai

    Menu

    AI Accelerators

    AI Software

    Metis Evaluation Systems

    Partners

    Product Inquiry

    Company

    Our Story

    Our Team

    Careers

    News

    Contact

    Follow Us

    LinkedInLinkedIn

    XX

    YouTubeYouTube

    Sign Up for Our Newsletter

    EU flag Axelera AI is co-funded by the European Union’s Horizon research and innovation programme.

    Privacy Policy  |  Cookie Policy  |  Terms of Sale  |  Vulnerability Disclosure Policy  |  End-User License Agreement  |  Copyright 2025 © All rights Reserved.