Building a two-stage pipeline with a compiled detector

Hi shabaz,

How did you create your yaml file?
We already have an example where we cascade yolo+tracker into a second model, please search for the file yolov5m-tracker-resnet50.yaml.
Do you have a yaml file that only runs your custom model? If yes you can just replace the resnet model in the file above by your custom model.

Cheers,

Sascha

Hi,

I noticed that your pipeline is similar to these examples we have in the SDK:
-- https://github.com/axelera-ai-hub/voyager-sdk/blob/release/v1.4/ax_models/reference/cascade/with_tracker/face-recognition-with-vote.yaml
-- https://github.com/axelera-ai-hub/voyager-sdk/blob/release/v1.4/ax_models/reference/cascade/with_tracker/ssd-mobilenetv1-resnet50.yaml
-- https://github.com/axelera-ai-hub/voyager-sdk/blob/release/v1.4/ax_models/reference/cascade/with_tracker/yolov5m-tracker-resnet50.yaml

Please use this template as a base or add your models incrementally and then modify it to fit your specific needs.

Thanks!

Hi Sascha,

Thanks for the response!

I don’t have a yaml file for just the second model, but that second model (YOLOv8n) basically just is trained to detect only one item, license plates, ideally from 640x640 crop, doesn’t do anything else. It will look very close to the first model, i.e. yolov5m-v7-coco-tracker I expected.

The 2-stage yaml file was a best guess, because I couldn’t see any example with a user compiled directory, so it may be a path issue, but I cannot tell, because the deploy.py just silently exits. The yolov5m-tracker-resnet50.yaml file looks way more complex than what I need I think; I don’t have some of the files referred in there, such as .so files, so I cannot see how to translate from that file to what I need.

Is there a known working yaml file that uses the yolov5m-v7-coco-tracker with a second stage being object detection similar to what I’ve got, using a compiled directory (containing the model.json, pool_l2_const.bin, etc)? Does my yaml file guesstimate look too far out of whack, or could it be tweaked to work?

Many thanks for all your help,

Shabaz.

Hi shabaz,

yes your yaml file is very far from valid syntax. In the file yolov5m-tracker-resnet50.yaml, you can just remove the lines inside of the tracker task that use the .so files, as they are not important. Then you need to replace resnet by your new yolo model when you take yolov5m-tracker-resnet50.yaml as a basis for your application. But it will be easy as you already have a good example of how to configure the yolo model, and there is no way around configuring your yolo model in the pipeline and models sections of the yaml file. Probably this file will be helpful to you: ax_models/tutorials/yolo/yolov8n-license-plate.yaml, as it is a yolov8n

Hi Sascha and Habib,

Thanks for the help, I looked at the examples and face-recognition-with-vote.yaml looked understandable, so I’ve gone through and tried to replicate. 

I have attached the file zipped below (vehicles-then-plates5.zip). I have these questions:

1. I don't have scale: False, nor linear-scaling, is this needed? The yolov5m-v7-coco-tracker.yaml does not have it, so I left out those lines too.
2. imreader_backend: OpenCV - I have deleted this line, since the source will be USB webcam /dev/video20 (usb:20). Do I need to add anything here? The yolov5m-v7-coco-tracker.yaml had nothing, so I guess I can leave this out too?
3. For the second stage, according to yaml-operators.md there is no NMS postprocess listed, so I have now exported ONNX with NMS, so I have deleted the postprocess - that’s correct I think?
4. For the weight_path and weight_url I’ve placed the full path and filename. Is that OK or does it need to be a relative path or anything?

Does the yaml file look usable? For the next step, can I move to deploy.py, or is there any other validation just for sanity testing it?

EDIT: I have tried to deploy, but it fails. This is what I did:

My yaml file (renamed to vehicles-then-plates.yaml) is in /axelera/myapp, because that is where my Python app will reside too.

The lp_yolov8n.onnx file is at \axelera  (I have placed a copy on OneDrive, it is downloadable as lp_yolov8n.onnx_with_nms but I renamed it to lp_yolov8n.onnx when I placed it on the Aetina board \axelera folder). (There is an lp_yolov8n.onnx without nms on OneDrive too, but I didn’t use that).

Then:

aetina@aetina:/axelera/voyager-sdk$ source venv/bin/activate
(venv) aetina@aetina:/axelera/voyager-sdk$ echo $PYTHONPATH
/axelera/voyager-sdk:/opt/axelera/runtime-1.4.0-1/tvm/tvm-src:/axelera/voyager-sdk:
(venv) aetina@aetina:/axelera/voyager-sdk$ ./deploy.py -v --loglevel=DEBUG /axelera/myapp/vehicles-then-plates.yaml
usage: deploy.py [-h] [--help-network] [--build-root PATH] [--data-root PATH] [--model MODEL | --models-only | --pipeline-only]
                 [--mode {QUANTIZE,QUANTIZE_DEBUG,QUANTCOMPILE,PREQUANTIZED}] [--metis {auto,none,pcie,m2}]
                 [--pipe {gst,torch,torch-aipu}] [--export] [--loglevel LEVEL] [--logfile PATH] [--logtimestamp] [--brief-logging]
                 [-q | -v] [--enable-vaapi | --disable-vaapi | --auto-vaapi] [--enable-opencl | --disable-opencl | --auto-opencl]
                 [--enable-opengl | --disable-opengl | --auto-opengl] [--aipu-cores {0,1,2,3,4}] [--num-cal-images NUM_CAL_IMAGES]
                 [--cal-seed CAL_SEED] [--default-representative-images | --no-default-representative-images]
                 [--dump-core-model | --no-dump-core-model]
                 network
deploy.py: error: "Invalid network '/axelera/myapp/vehicles-then-plates.yaml', no close match found. Please `make help` to see all available models."
(venv) aetina@aetina:/axelera/voyager-sdk$ 

Many thanks,

Shabaz.

Hi ​@shabaz,

If the second model in your pipeline follows yolo architecture then there is no need to modify the onnx, the compiler will take care of extracting the post-proc graph and if high-level yaml is setup correctly, it should ideally use the correct decoder.

Can you confirm that you can compile the second model with a single pipeline-based high-level yaml? (that would be a first step towards making sure that your model is functional and then would integrate nicely in the full-pipeline)

Thanks!

---

Can you please try this pipeline? (I could compile this high-level yaml with your model)

axelera-model-format: 1.0.0

name: vehicle-plates-reference-design

description: vehicle then plates

pipeline:
  - detections:
      model_name: yolov5m
      input:
          type: image
      preprocess:
        - letterbox:
            width: 640
            height: 640
        - torch-totensor:
      inference:
        handle_all: False
      postprocess:
        - decodeyolo:
            box_format: xywh
            normalized_coord: False
            label_filter: ['car', 'truck', 'motorcycle', 'bus', 'person', 'bicycle', 'cat', 'dog']
            max_nms_boxes: 5000
            conf_threshold: 0.3
            nms_iou_threshold: 0.5
            nms_class_agnostic: False
            nms_top_k: 100
            use_multi_label: False
      render:
          show_annotations: False
          show_labels: False

  - tracking:
      model_name: oc_sort
      input:
        source: full
        color_format: RGB
      cv_process:
        - tracker:
            algorithm: oc-sort
            bbox_task_name: detections
            label_filter:
              - car
            history_length: 30
            algo_params:
              max_age: 50
              min_hits: 1
              iou_threshold: 0.3
              max_id: 0

  - plate-detections:
      model_name: lp_yolov8n
      input:
          type: image
          source: roi
          where: tracking
          which: AREA # AREA, SCORE, CENTER
          top_k: 1
      preprocess:
          - resize:
              width: 640
              height: 640
          - torch-totensor:
      inference:
        handle_all: False
      postprocess:
        - decodeyolo:
            box_format: xywh
            normalized_coord: False
            label_filter: ['car', 'truck']
            max_nms_boxes: 5000
            conf_threshold: 0.3
            nms_iou_threshold: 0.5
            nms_class_agnostic: False
            nms_top_k: 100
            use_multi_label: False

operators:
  decodeyolo:
    class: DecodeYolo
    class_path: $AXELERA_FRAMEWORK/ax_models/decoders/yolo.py

models:
  yolov5m:
    class: AxONNXModel
    class_path: $AXELERA_FRAMEWORK/ax_models/base_onnx.py
    weight_path: weights/yolov5m-v7.onnx
    weight_url: https://media.axelera.ai/artifacts/weights/yolo/op17/yolov5m-v7.onnx
    weight_md5: aad63caa41a389d409717a0bbaa17311
    task_category: ObjectDetection
    input_tensor_layout: NCHW
    input_tensor_shape: [1, 3, 640, 640]
    input_color_format: RGB
    num_classes: 80
    dataset: CocoDataset-COCO2017
    extra_kwargs:
      aipu_cores: 2
      max_compiler_cores: 4
      YOLO:
        anchors: # or, specified explicitly
        - [10,13, 16,30, 33,23]
        - [30,61, 62,45, 59,119]
        - [116,90, 156,198, 373,326]
        strides: [8, 16, 32]  # specify if you are using a special version  
  oc_sort:
    model_type: CLASSICAL_CV
    task_category: ObjectTracking
  lp_yolov8n:
    class: AxONNXModel
    class_path: $AXELERA_FRAMEWORK/ax_models/base_onnx.py
    weight_path: $AXELERA_FRAMEWORK/customers/lpr_community/lp_yolov8n.onnx
    task_category: ObjectDetection
    input_tensor_layout: NCHW
    input_tensor_shape: [1, 3, 640, 640]
    input_color_format: RGB
    num_classes: 1
    dataset: CocoDataset-COCO2017
    extra_kwargs:
      aipu_cores: 2
      max_compiler_cores: 4

datasets:
  CocoDataset-COCO2017:
    class: ObjDataAdapter
    class_path: $AXELERA_FRAMEWORK/ax_datasets/objdataadapter.py
    data_dir_name: coco
    labels_path: $AXELERA_FRAMEWORK/ax_datasets/labels/coco.names
    label_type: COCO2017

Hi Habib,

Thank you for trying this!

I tried your yaml, however I still get the deploy error, I suspect it may be to do with how I’m executing the deploy command.

This is what I did.

First, I tried compiling the model, to check that was possible:

(venv) aetina@aetina:/axelera$ ls -altr
total 35988
drwx------  2 aetina aetina    16384 Jul  8 16:39 lost+found
-rw-r--r--  1 aetina aetina        0 Jul  8 16:44 hello_from_ssd.txt
drwxrwxr-x  4 aetina aetina     4096 Sep  3 14:10 temp_folder
drwxr-xr-x  3 root   root       4096 Sep  3 18:20 ..
drwxr-xr-x  6 aetina aetina     4096 Sep  4 16:39 aetina_cache_axelera
drwxrwxr-x  4 aetina aetina     4096 Sep  6 01:34 compiled
-rwxr-xr-x  1 aetina aetina 12275248 Sep  8 20:12 lp_yolov8n.onnx_with_nms
-rwxr-xr-x  1 aetina aetina 12251040 Sep  8 20:15 lp_yolov8n.onnx_without_nms
drwxr-xr-x  9 aetina aetina     4096 Sep  8 20:16 .
-rwxr-xr-x  1 aetina aetina 12275248 Sep  8 20:16 lp_yolov8n.onnx
drwxrwxr-x 21 aetina aetina     4096 Sep  8 23:43 voyager-sdk
drwx--x--- 12 root   root       4096 Sep  9 17:09 docker
drwxrwxr-x  4 aetina aetina     4096 Sep 10 02:44 myapp
(venv) aetina@aetina:/axelera$ compile -i /axelera/lp_yolov8n.onnx -o my_lp_model_compiled_with_nms
/home/aetina/.cache/axelera/venvs/0ce26488/lib/python3.10/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 
  warn(f"Failed to load image Python extension: {e}")
02:46:43 [INFO] Dump used CLI arguments to: /mnt/m2ssd/my_lp_model_compiled_with_nms/cli_args.json
02:46:43 [INFO] Dump used compiler configuration to: /mnt/m2ssd/my_lp_model_compiled_with_nms/conf.json
02:46:44 [INFO] Input model has static input shape(s): ((1, 3, 640, 640),). Use it for quantization.
02:46:44 [INFO] Data layout of the input model: NCHW
02:46:44 [INFO] Using dataset of size 100 for calibration.
02:46:44 [INFO] In case of compilation failures, turn on 'save_error_artifact' and share the archive with Axelera AI.
02:46:44 [INFO] Quantizing '' using QToolsV2.
02:46:44 [INFO] ONNX model validation can be turned off by setting 'validate_operators' to 'False'.
02:47:00 [INFO] Checking ONNX model compatibility with the constraints of opset 12.
Calibrating... ✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨ | 100% |  1.04s/it | 100it |
02:48:52 [INFO] Exporting '' using GraphExporterV2.
/home/aetina/.cache/axelera/venvs/0ce26488/lib/python3.10/site-packages/torch/nn/functional.py:3734: UserWarning: nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.
  warnings.warn("nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.")
/home/aetina/.cache/axelera/venvs/0ce26488/lib/python3.10/site-packages/torch/jit/_trace.py:976: TracerWarning: Encountering a list at the output of the tracer might cause the trace to be incorrect, this is only valid if the container structure does not change based on the module's inputs. Consider using a constant container instead (e.g. for `list`, use a `tuple` instead. for `dict`, use a `NamedTuple` instead). If you absolutely need this and know the side effects, pass strict=False to trace() to allow this behavior.
  module._c._create_method_from_trace(
02:49:14 [INFO] Quantization finished.
02:49:14 [INFO] Quantization took: 150.7 seconds.
02:49:14 [INFO] Export quantized model manifest to JSON file: /mnt/m2ssd/my_lp_model_compiled_with_nms/quantized_model_manifest.json
02:49:15 [INFO] Lower input model to target device...
02:49:15 [INFO] In case of compilation failures, turn on 'save_error_artifact' and share the archive with Axelera AI.
02:49:15 [INFO] Lowering '' to target 'device' in 'multiprocess' mode for 1 AIPU core(s) using 100.0% of available AIPU resources.
02:49:24 [INFO] Running LowerFrontend...
02:50:18 [INFO] Running FrontendToMidend...
02:50:23 [INFO] Running LowerMidend...
02:50:34 [INFO] Running MidendToTIR...
02:52:17 [INFO] Running LowerTIR...
02:55:41 [WARNING] LowerTIR failed to fit buffers into memory after iteration 0/4.
  Pool usage: {L1: alloc:4,300,288B avail:4,194,304B over:105,984B util:102.53%, L2: alloc:25,300,480B avail:32,309,248B over:0B util:78.31%, DDR: alloc:2,759,808B avail:1,040,187,392B over:0B util:0.27%}
  Overflowing buffer IDs: {99, 100, 101, 1486, 1405, 1407}
02:59:04 [INFO] LowerTIR succeeded to fit buffers into memory after iteration 1/4.
  Pool usage: {L1: alloc:4,169,216B avail:4,194,304B over:0B util:99.40%, L2: alloc:25,300,480B avail:32,309,248B over:0B util:78.31%, DDR: alloc:2,759,808B avail:1,040,187,392B over:0B util:0.27%}
  Overflowing buffer IDs: set()
02:59:05 [INFO] Running TirToAtex...
02:59:12 [INFO] Running LowerATEX...
02:59:22 [INFO] Running AtexToArtifact...
02:59:22 [INFO] Allocate L1 memory pool pool_l1
02:59:22 [INFO] Allocating pool_l1 at 0x018006200 (size: 4169216 bytes, alignment: 64 bytes)
02:59:27 [INFO] Lowering finished!
02:59:27 [INFO] Compilation took: 611.96 seconds.
02:59:27 [INFO] Passes report was generated and saved to: /mnt/m2ssd/my_lp_model_compiled_with_nms/compiled_model/pass_benchmark_report.json
02:59:27 [INFO] Lowering finished. Export model manifest to JSON file: /mnt/m2ssd/my_lp_model_compiled_with_nms/compiled_model_manifest.json
02:59:27 [INFO] Total time: 763.42 seconds.
02:59:27 [INFO] Done.
(venv) aetina@aetina:/axelera$ 
(venv) aetina@aetina:/axelera$ ls -altr
total 35996
drwx------  2 aetina aetina    16384 Jul  8 16:39 lost+found
-rw-r--r--  1 aetina aetina        0 Jul  8 16:44 hello_from_ssd.txt
drwxrwxr-x  4 aetina aetina     4096 Sep  3 14:10 temp_folder
drwxr-xr-x  3 root   root       4096 Sep  3 18:20 ..
drwxr-xr-x  6 aetina aetina     4096 Sep  4 16:39 aetina_cache_axelera
drwxrwxr-x  4 aetina aetina     4096 Sep  6 01:34 compiled
-rwxr-xr-x  1 aetina aetina 12275248 Sep  8 20:12 lp_yolov8n.onnx_with_nms
-rwxr-xr-x  1 aetina aetina 12251040 Sep  8 20:15 lp_yolov8n.onnx_without_nms
-rwxr-xr-x  1 aetina aetina 12275248 Sep  8 20:16 lp_yolov8n.onnx
drwxrwxr-x 21 aetina aetina     4096 Sep  8 23:43 voyager-sdk
drwx--x--- 12 root   root       4096 Sep  9 17:09 docker
drwxrwxr-x  4 aetina aetina     4096 Sep 10 02:44 myapp
drwxrwxr-x  2 aetina aetina     4096 Sep 10 02:46 .pkl_memoize_py3
drwxr-xr-x 11 aetina aetina     4096 Sep 10 02:46 .
drwxrwxr-x  5 aetina aetina     4096 Sep 10 02:59 my_lp_model_compiled_with_nms
(venv) aetina@aetina:/axelera$ cd my_lp_model_compiled_with_nms/
(venv) aetina@aetina:/axelera/my_lp_model_compiled_with_nms$ ls -altr
total 44
-rw-rw-r--  1 aetina aetina  469 Sep 10 02:46 cli_args.json
drwxr-xr-x 11 aetina aetina 4096 Sep 10 02:46 ..
-rw-rw-r--  1 aetina aetina 3215 Sep 10 02:46 conf.json
drwxrwxr-x  2 aetina aetina 4096 Sep 10 02:46 input_model
-rw-rw-r--  1 aetina aetina 1119 Sep 10 02:49 quantized_model_manifest.json
drwxrwxr-x  2 aetina aetina 4096 Sep 10 02:49 quantized_model
-rw-rw-r--  1 aetina aetina 2536 Sep 10 02:59 compiled_model_manifest.json
drwxrwxr-x  3 aetina aetina 4096 Sep 10 02:59 compiled_model
-rw-rw-r--  1 aetina aetina 3196 Sep 10 02:59 compilation_log.txt
drwxrwxr-x  5 aetina aetina 4096 Sep 10 02:59 .
-rw-rw-r--  1 aetina aetina  121 Sep 10 02:59 compilation_report.json
(venv) aetina@aetina:/axelera/my_lp_model_compiled_with_nms$ cat compilation_report.json 
{
  "status": "SUCCEEDED",
  "target": "",
  "error_msg": "",
  "quantization_time_s": 150,
  "compilation_time_s": 611
}(venv) aetina@aetina:/axelera/my_lp_model_compiled_with_nms$ 

That succeeded, so I moved on to trying to deploy. 

Your yaml fine contains 

weight_path: $AXELERA_FRAMEWORK/customers/lpr_community/lp_yolov8n.onnx

so I created the lpr_community folder and copied the onnx file into there, so I could use the yaml file identically.

However, when I run it:

(venv) aetina@aetina:/axelera/voyager-sdk$ echo $AXELERA_FRAMEWORK
/axelera/voyager-sdk
(venv) aetina@aetina:/axelera/voyager-sdk$ ls -l customers/lpr_community/
total 11988
-rwxr-xr-x 1 aetina aetina 12275248 Sep 10 03:06 lp_yolov8n.onnx
(venv) aetina@aetina:/axelera/voyager-sdk$ 
(venv) aetina@aetina:/axelera/voyager-sdk$ ./deploy.py -v --loglevel=DEBUG --mode QUANTIZE_DEBUG /axelera/myapp/vehicles-then-plates.yaml
usage: deploy.py [-h] [--help-network] [--build-root PATH] [--data-root PATH] [--model MODEL | --models-only | --pipeline-only]
                 [--mode {QUANTIZE,QUANTIZE_DEBUG,QUANTCOMPILE,PREQUANTIZED}] [--metis {auto,none,pcie,m2}]
                 [--pipe {gst,torch,torch-aipu}] [--export] [--loglevel LEVEL] [--logfile PATH] [--logtimestamp] [--brief-logging]
                 [-q | -v] [--enable-vaapi | --disable-vaapi | --auto-vaapi] [--enable-opencl | --disable-opencl | --auto-opencl]
                 [--enable-opengl | --disable-opengl | --auto-opengl] [--aipu-cores {0,1,2,3,4}] [--num-cal-images NUM_CAL_IMAGES]
                 [--cal-seed CAL_SEED] [--default-representative-images | --no-default-representative-images]
                 [--dump-core-model | --no-dump-core-model]
                 network
deploy.py: error: "Invalid network '/axelera/myapp/vehicles-then-plates.yaml', no close match found. Please `make help` to see all available models."
(venv) aetina@aetina:/axelera/voyager-sdk$

I’m guessing I’m running deploy.py incorrectly?

Many thanks,

Shabaz.

I’m admittedly no expert on this, so hopefully I’m not leading us all into a dead end 😆

But could it be just that deploy.py won’t scan the /axelera/myapp/ folder? Is it expecting custom YAMLs to go under customers/ inside the Voyager SDK tree? And once in there, it can find them by name (vehicles-then-plates) instead of the full path?

(I admit, I’m making this up a little as I go 🤣 So this might be nonsense/unneeded)

​@shabaz 
Indeed! as the deploy.py says, we should do make help (to see if the model is recognized by SDK’s make system):

compile -i is not necessary, since deploy.py will compile your model from onnx

once, you have placed the yaml file in ${AXELERA_FRAMEWORK}/customers/lpr_community/ path, see:

> cd ${AXELERA_FRAMEWORK}/customers/lpr_community

> ls -lha
total 12M
drwxrwxr-x 2 ubuntu ubuntu 4.0K Sep  9 10:05 .
drwxrwxr-x 6 ubuntu ubuntu 4.0K Sep  9 10:21 ..
-rw-rw-r-- 1 ubuntu ubuntu  12M Sep  9 10:04 lp_yolov8n.onnx
-rw-rw-r-- 1 ubuntu ubuntu 3.5K Sep  9 12:57 lpr-high-level.yaml
-rw-rw-r-- 1 ubuntu ubuntu 1.4K Sep  9 10:04 vehicles-then-plates5.zip

then we can verify with make as follows:

> cd ${AXELERA_FRAMEWORK}

> make help | grep -e vehicle-plates-reference-design
|   vehicle-plates-reference-design  vehicle then plates                                                               |

and run deploy.py as follows:

> cd ${AXELERA_FRAMEWORK}

> ./deploy.py vehicle-plates-reference-design

the above should compile your model and you should be able to verify this by going to the build dir:

> cd ${AXELERA_FRAMEWORK}

> ls -lha build/vehicle-plates-reference-design/
total 32K
drwxrwxr-x 6 ubuntu ubuntu 4.0K Sep  9 12:50 .
drwxrwxr-x 5 ubuntu ubuntu 4.0K Sep  9 11:21 ..
drwxrwxr-x 2 ubuntu ubuntu 4.0K Sep  9 11:26 logs
drwxrwxr-x 5 ubuntu ubuntu 4.0K Sep  9 12:46 lp_yolov8n
drwxrwxr-x 2 ubuntu ubuntu 4.0K Sep  9 11:53 oc_sort
-rw-rw-r-- 1 ubuntu ubuntu 1.4K Sep  9 12:52 oc_sort.axnet
drwxrwxr-x 5 ubuntu ubuntu 4.0K Sep 10 10:22 yolov5m
-rw-rw-r-- 1 ubuntu ubuntu 1.1K Sep  9 12:52 yolov5m.axnet

after that we can try with inference.py as follows (or with your app):

> cd ${AXELERA_FRAMEWORK}

> ./inference.py vehicle-plates-reference-design media/output.mp4 \
                                --frames=3000 --disable-vaapi

Thanks!

Hi Habib and Spanner,

Thank you for the help! You were both completely right, the yaml file was not at the correct location. 

For some reason it didn’t mentally click last night that both the yaml and onnx file were to be placed there, and I’ve just tried now, and it deploys! : )  Log below.

The next step, will be to make mods to the code and run it, which I will try tonight.

I had another question; as well as the 2-stage, I’d like to do a small amount of processing on the Rockchip CPU (i.e. within the Python code for now), and then (if possible) run another 1-stage, based on the processed output. If that is possible, how is that architected, would it be another create_inference_stream, this time passing the processed output, and perhaps another thread running the second inference loop?

My current code (for the earlier 1-stage, I need to update it for the 2-stage) is closely based on a tutorial example.

Many thanks,

Shabaz.

    _        _   _             
   / \   ___| |_(_)_ __   __ _ 
  / _ \ / _ \ __| | '_ \ / _` |
 / ___ \  __/ |_| | | | | (_| |
/_/   \_\___|\__|_|_| |_|\__,_|

Welcome to Ubuntu 22.04.5 LTS (GNU/Linux 5.10.110 aarch64)

 * Documentation:  http://wiki.t-firefly.com
 * Management:     http://www.t-firefly.com

System information as of Wed Sep 10 12:14:47 UTC 2025

System load:   1.36 0.30 0.10   Up time:       0 min
Memory usage:  4 % of 15952MB   IP:            192.168.1.89
Usage of /:    57% of 14G       storage/:      2% of 469G   

Last login: Wed Sep 10 12:14:39 2025
root@aetina:~# su - aetina
aetina@aetina:~$ cd /axelera/voyager-sdk/
aetina@aetina:/axelera/voyager-sdk$ source venv/bin/activate
(venv) aetina@aetina:/axelera/voyager-sdk$ make help | grep -e vehicle-plates-reference-design
|   vehicle-plates-reference-design  vehicle then plates                                                               |
(venv) aetina@aetina:/axelera/voyager-sdk$ ./deploy.py vehicle-plates-reference-design
INFO    : Detected Metis type as m2
INFO    : Compiling network vehicle-plates-reference-design /axelera/voyager-sdk/customers/lpr_community/vehicles-then-plates.yaml
INFO    : Compile model: yolov5m
INFO    : Using representative images from /mnt/m2ssd/aetina_cache_axelera/data/coco2017_repr400 with backend ImageReader.PIL,pipeline input color format ColorFormat.RGB
INFO    : Prequantizing vehicle-plates-reference-design: yolov5m
INFO    : Quantizing network vehicle-plates-reference-design /axelera/voyager-sdk/customers/lpr_community/vehicles-then-plates.yaml yolov5m
INFO    : Compile model: yolov5m
INFO    : Using representative images from /mnt/m2ssd/aetina_cache_axelera/data/coco2017_repr400 with backend ImageReader.PIL,pipeline input color format ColorFormat.RGB
Calibrating... ✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨ | 100% |  2.74s/it | 200it |
/home/aetina/.cache/axelera/venvs/0ce26488/lib/python3.10/site-packages/torch/nn/functional.py:3734: UserWarning: nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.
  warnings.warn("nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.")
/home/aetina/.cache/axelera/venvs/0ce26488/lib/python3.10/site-packages/torch/jit/_trace.py:976: TracerWarning: Encountering a list at the output of the tracer might cause the trace to be incorrect, this is only valid if the container structure does not change based on the module's inputs. Consider using a constant container instead (e.g. for `list`, use a `tuple` instead. for `dict`, use a `NamedTuple` instead). If you absolutely need this and know the side effects, pass strict=False to trace() to allow this behavior.
  module._c._create_method_from_trace(
INFO    : Finished quantizing network vehicle-plates-reference-design: model 'yolov5m'
INFO    : Quantizing vehicle-plates-reference-design: yolov5m took 652.430 seconds
INFO    : Successfully prequantized vehicle-plates-reference-design: yolov5m
INFO    : Using representative images from /mnt/m2ssd/aetina_cache_axelera/data/coco2017_repr400 with backend ImageReader.PIL,pipeline input color format ColorFormat.RGB
LowerTIR failed to fit buffers into memory after iteration 0/4.
  Pool usage: {L1: alloc:4,865,792B avail:4,194,304B over:671,488B util:116.01%, L2: alloc:6,656,896B avail:8,077,312B over:0B util:82.41%, DDR: alloc:20,566,272B avail:260,046,848B over:0B util:7.91%}
  Overflowing buffer IDs: {1536, 1538, 133, 134, 137, 138, 139, 140, 1549, 1550, 1551, 1552, 1553, 1810, 1811, 1555, 1554, 1814, 1815, 151, 153, 1818, 154, 1817, 150, 149, 1816, 1812, 1826, 1813, 168, 169, 175, 177, 1477, 1478, 1481, 94, 116, 1533, 1535}
LowerTIR failed to fit buffers into memory after iteration 1/4.
  Pool usage: {L1: alloc:4,204,288B avail:4,194,304B over:9,984B util:100.24%, L2: alloc:6,656,896B avail:8,077,312B over:0B util:82.41%, DDR: alloc:20,566,272B avail:260,046,848B over:0B util:7.91%}
  Overflowing buffer IDs: {1477, 1478, 1481, 1852, 94}
WARNING : LowerTIR failed to fit buffers into memory after iteration 0/4.
WARNING :   Pool usage: {L1: alloc:4,865,792B avail:4,194,304B over:671,488B util:116.01%, L2: alloc:6,656,896B avail:8,077,312B over:0B util:82.41%, DDR: alloc:20,566,272B avail:260,046,848B over:0B util:7.91%}
WARNING :   Overflowing buffer IDs: {1536, 1538, 133, 134, 137, 138, 139, 140, 1549, 1550, 1551, 1552, 1553, 1810, 1811, 1555, 1554, 1814, 1815, 151, 153, 1818, 154, 1817, 150, 149, 1816, 1812, 1826, 1813, 168, 169, 175, 177, 1477, 1478, 1481, 94, 116, 1533, 1535}
WARNING : LowerTIR failed to fit buffers into memory after iteration 1/4.
WARNING :   Pool usage: {L1: alloc:4,204,288B avail:4,194,304B over:9,984B util:100.24%, L2: alloc:6,656,896B avail:8,077,312B over:0B util:82.41%, DDR: alloc:20,566,272B avail:260,046,848B over:0B util:7.91%}
WARNING :   Overflowing buffer IDs: {1477, 1478, 1481, 1852, 94}
INFO    : Allocate L1 memory pool pool_l1_0
INFO    : Allocating pool_l1_0 at 0x01801D900 (size: 4073216 bytes, alignment: 64 bytes)
INFO    : Allocate L1 memory pool pool_l1_1
INFO    : Allocating pool_l1_1 at 0x01801D900 (size: 4073216 bytes, alignment: 64 bytes)
|████████████████████████████████████████| 15:36.7 
INFO    : Compile model: oc_sort
INFO    : Compile model: lp_yolov8n
INFO    : Using representative images from /mnt/m2ssd/aetina_cache_axelera/data/coco2017_repr400 with backend ImageReader.PIL,pipeline input color format ColorFormat.RGB
INFO    : Prequantizing vehicle-plates-reference-design: lp_yolov8n
INFO    : Quantizing network vehicle-plates-reference-design /axelera/voyager-sdk/customers/lpr_community/vehicles-then-plates.yaml lp_yolov8n
INFO    : Compile model: lp_yolov8n
INFO    : Using representative images from /mnt/m2ssd/aetina_cache_axelera/data/coco2017_repr400 with backend ImageReader.PIL,pipeline input color format ColorFormat.RGB
Calibrating... ✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨ | 100% |  4.84s/it | 200it |
/home/aetina/.cache/axelera/venvs/0ce26488/lib/python3.10/site-packages/torch/nn/functional.py:3734: UserWarning: nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.
  warnings.warn("nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.")
/home/aetina/.cache/axelera/venvs/0ce26488/lib/python3.10/site-packages/torch/jit/_trace.py:976: TracerWarning: Encountering a list at the output of the tracer might cause the trace to be incorrect, this is only valid if the container structure does not change based on the module's inputs. Consider using a constant container instead (e.g. for `list`, use a `tuple` instead. for `dict`, use a `NamedTuple` instead). If you absolutely need this and know the side effects, pass strict=False to trace() to allow this behavior.
  module._c._create_method_from_trace(
INFO    : Finished quantizing network vehicle-plates-reference-design: model 'lp_yolov8n'
INFO    : Quantizing vehicle-plates-reference-design: lp_yolov8n took 1064.329 seconds
INFO    : Successfully prequantized vehicle-plates-reference-design: lp_yolov8n
INFO    : Using representative images from /mnt/m2ssd/aetina_cache_axelera/data/coco2017_repr400 with backend ImageReader.PIL,pipeline input color format ColorFormat.RGB
LowerTIR failed to fit buffers into memory after iteration 0/4.
  Pool usage: {L1: alloc:4,300,288B avail:4,194,304B over:105,984B util:102.53%, L2: alloc:7,229,184B avail:8,077,312B over:0B util:89.50%, DDR: alloc:11,457,664B avail:260,046,848B over:0B util:4.41%}
  Overflowing buffer IDs: {97, 99, 100, 1486, 1405, 1406, 1407}
WARNING : LowerTIR failed to fit buffers into memory after iteration 0/4.
WARNING :   Pool usage: {L1: alloc:4,300,288B avail:4,194,304B over:105,984B util:102.53%, L2: alloc:7,229,184B avail:8,077,312B over:0B util:89.50%, DDR: alloc:11,457,664B avail:260,046,848B over:0B util:4.41%}
WARNING :   Overflowing buffer IDs: {97, 99, 100, 1486, 1405, 1406, 1407}
INFO    : Allocate L1 memory pool pool_l1_0
INFO    : Allocating pool_l1_0 at 0x018006200 (size: 4169216 bytes, alignment: 64 bytes)
INFO    : Allocate L1 memory pool pool_l1_1
INFO    : Allocating pool_l1_1 at 0x018006200 (size: 4169216 bytes, alignment: 64 bytes)
|████████████████████████████████████████| 10:03.7 
INFO    : Compile vehicles-then-plates.yaml:pipeline
INFO    : Compiling vehicle-plates-reference-design took 3289.184 seconds
INFO    : Successfully deployed network
(venv) aetina@aetina:/axelera/voyager-sdk$ 

That’s awesome, nice work!

Not sure I’ve got the chops to answer your question, but ​@Habib might! 😄

​@shabaz 

I’d love to know a bit more about your pipeline architecture. Can you please provide some more details on the following aspects?

-- What is the overall processing flow you're aiming for?

-- What are the primary inputs and outputs of this pipeline?

-- Are there any specific requirements or constraints that need to be considered?

Any additional information you can share would be greatly appreciated. 

Thanks!

Hi Habib,

The overall flow is: /dev/video20 → vehicle detection (yolov5m-v7-coco-tracker) → license plate detection (yolov8n) → LPRNet (plate characters recognition)  (i.e. the output is to be the license plate text) but since I’ve not tried this before, I wasn’t sure if I may need to insert a stage to straighten the plates, and that stage may have been coded in C or Python. However, it’s optional, so I’m now thinking let’s simplify and just assume for now that the camera will be facing head-on with very little skew. If we make that assumption for now, is the plan sound? Should I go ahead and extend that yaml file to be a three-stage pipeline? Anything I need to consider in the Python code?

I have no other requirement or constraint, this is my first project with the Metis accelerator, so provided I can get to the stage of outputting license plate text, even if everything else has arbitrary or otherwise constraints is OK.

What will ultimately happen with that license plate text is already coded (I will simply run a Python function which will send it via RS-232 to a remote microcontroller device (which puts it onto an LCD screen and controls an external relay when it matches particular license plate strings).

Many thanks for all your help guys,

Shabaz.

Hi ​@Habib 

I am from DeGirum (a SW partner of Axelera), and we are working on a user guide to help your use case. We actually have a pipeline that does the license plate detection and OCR. Can you tell us why the first stage is needed? Our current license plate detection model is good enough to detect the plates in a single stage.

Hi Shashi,

My second stage is only trained on a couple of thousand images so it’s likely not as robust as yours. I was just keen to try a multi-stage pipeline including a custom second stage so I could examine the process to compile/deploy from start to finish of a functioning project. But I’m definitely keen to later strip out stages if possible and make it more robust in a phase 2 for sure. 

Many thanks,

Shabaz.

Hi ​@shabaz  and ​@shashi.chilappagari ,

I would suggest keeping the high-level YAML file with two models and a tracker as it is. For LPRNet, we could perhaps modify this example from https://github.com/axelera-ai-hub/voyager-sdk/blob/release/v1.4/examples/axruntime/axruntime_example.py to suit our needs. And if needed, we can also deskew the detections before feeding them into the LPRNet.

As for using a single stage for this task, ​@shashi.chilappagari, I think it depends on the use case. While a single stage might provide better results, but would require processing high-resolution images, which could be compute-intensive. But would be interesting to see results from both approaches.

Thanks!

​@Habib 

Thanks for the insights. We will share our code soon. Then anyone can evaluate and see which approach is better for their use case. Our code example will allow users to configure 2 or 3 stage pipelines with minimal changes.