Considering the M.2 module – what can it realistically handle?

From my limited benchmarking with the M.2, I’ve found it to be a quite capable little beast! On my Core i5 I could run yolov8l-coco-onnx on 4 parallel RTSP streams of 30 FPS each (at 1080p) and get a smooth output without any dropped frames. So at least 120FPS for that model?

Here is the system I used:

$ lscpu
Architecture:             x86_64
  CPU op-mode(s):         32-bit, 64-bit
  Address sizes:          46 bits physical, 48 bits virtual
  Byte Order:             Little Endian
CPU(s):                   20
  On-line CPU(s) list:    0-19
Vendor ID:                GenuineIntel
  Model name:             13th Gen Intel(R) Core(TM) i5-13600
    CPU family:           6
    Model:                191
    Thread(s) per core:   2
    Core(s) per socket:   14
    Socket(s):            1
    Stepping:             2
    CPU max MHz:          5000.0000
    CPU min MHz:          800.0000

$ lspci | grep Metis
01:00.0 Processing accelerators: Axelera AI Metis AIPU (rev 02)
$ axdevice
Device 0: metis-0:1:0 board_type=pcie fwver='v1.2.5' clock=800MHz(0-3:800MHz) mvm=0-3:100%

I’m still kinda new to benchmarking devices, but in my understanding that looks solid!

What I would like to see is a comparison between the Metis M.2 module and Hailo M.2 module (and maybe the Coral TPU for completeness). I placed an order for two of these and it should be coming in a few days, but instead of me doing the legwork, is there something already out there?

  Ah, this might have some of what you’re looking for, ​@quant.geek 👍