Guidance on Running Lane Detection Model (culane_res18.onnx) on Metis M2

This sounds awesome! Great project to be building. How far along is it?

I don’t think it’s currently in the current Axelera model zoo, but you can absolutely experiment with it on Metis M.2 with the Voyager SDK.

You’ll just need to:

1. Clone and install the SDK from GitHub.

2. Copy a zoo YAML (like a ResNet or ONNX-based one) and swap in your model and dataset paths.

3. Deploy with ./deploy.py and test with ./inference.py.

Even though it’s not officially in the zoo, it looks like ResNet-18-based models like culane_res18.onnx are pretty lightweight, so we’re in with a good chance here. Would be great to see this running - lane detection is a perfect fit for edge AI!

Links to help you get started:

• Voyager SDK Getting Started Guide

• Deploying Custom Models

• Deploying ONNX with Custom Weights

• Advanced Model Deployment Guide

Keep us posted, and drop any replies in here for any extra help or ideas!

Hey,

Thanks for all the pointers! Here’s what I’ve done so far and where I’m stuck:

1. SDK setup

   • Cloned and installed the Voyager SDK v1.2.5 from GitHub

   • Verified I can run ./deploy.py and ./inference.py on the ResNet-50 example successfully

2. Custom model YAMLs

   • Created customers/myculane/culane_res18-lane-onnx.yaml by copying the ResNet-50 template and swapping in my ONNX model and CULane dataset paths

   • Modified the torch-imagenet.yaml in the pipeline-template folder so that its input/output definitions line up with my lane-detection model

3. ONNX export

   • Started from the pretrained culane_res18.pth from the Ultra-Fast-Lane-Detection-v2 repo

   • Used their pt2onnx.py script (modified to explicitly set opset_version=14) to export culane_res18_opset14.onnx

   • Confirmed with onnx.load that the new model’s opset is indeed version 14 (the original was opset 14 already)

4. Deployment error
   When I run:

   ./deploy.py customers/myculane/culane_res18-lane-onnx.yaml
   

   I get the quantization failure:

   (venv) aravind@aravind-H610M-H-V2:~/Desktop/voyager-sdk$ ./deploy.py customers/myculane/culane_res18-lane-onnx.yaml 
   INFO    : Using device metis-0:1:0
   INFO    : Detected Metis type as pcie
   INFO    : Compiling network culane_res18-lane-onnx /home/aravind/Desktop/voyager-sdk/customers/myculane/culane_res18-lane-onnx.yaml
   INFO    : Compile model: culane_res18-lane-onnx
   INFO    : Imported DataAdapter TorchvisionDataAdapter from /home/aravind/Desktop/voyager-sdk/ax_datasets/torchvision.py
   INFO    : Assuming it's a custom dataset with ImageFolder format.
   INFO    : Using representative images from /home/aravind/Desktop/voyager-sdk/data/CULANE/calib with backend ImageReader.PIL,pipeline input color format ColorFormat.RGB
   INFO    : Prequantizing c-culane_res18-lane-onnx: culane_res18-lane-onnx
   INFO    : Using device metis-0:1:0
   INFO    : Quantizing network culane_res18-lane-onnx /home/aravind/Desktop/voyager-sdk/customers/myculane/culane_res18-lane-onnx.yaml culane_res18-lane-onnx
   INFO    : Compile model: culane_res18-lane-onnx
   INFO    : Imported DataAdapter TorchvisionDataAdapter from /home/aravind/Desktop/voyager-sdk/ax_datasets/torchvision.py
   INFO    : Assuming it's a custom dataset with ImageFolder format.
   INFO    : Using representative images from /home/aravind/Desktop/voyager-sdk/data/CULANE/calib with backend ImageReader.PIL,pipeline input color format ColorFormat.RGB
   Calibrating... ✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨✨ | 100% |  4.06it/s | 200
   ERROR   : Traceback (most recent call last):
   ERROR   :   File "/home/aravind/Desktop/voyager-sdk/axelera/app/compile.py", line 429, in compile
   ERROR   :     the_manifest = top_level.compile(model, compilation_cfg, output_path)
   ERROR   :   File "<frozen compiler.top_level>", line 833, in compile
   ERROR   :   File "<frozen compiler.top_level>", line 550, in quantize
   ERROR   :   File "<frozen qtools_tvm_interface.graph_exporter_v2.graph_exporter>", line 120, in __init__
   ERROR   : RuntimeError: External op reshape_branching_point_to_cls_cls_dot_0_reduce_mean_dre found in the model (<class 'qtoolsv2.rewriter.operators.fqelements.Dequantize'> op). QTools may have issues quantizing this model.
   ERROR   : External op reshape_branching_point_to_cls_cls_dot_0_reduce_mean_dre found in the model (<class 'qtoolsv2.rewriter.operators.fqelements.Dequantize'> op). QTools may have issues quantizing this model.
   INFO    : Quantizing c-culane_res18-lane-onnx: culane_res18-lane-onnx took 114.248 seconds
   ERROR   : Failed to deploy network
   ERROR   : Failed to prequantize c-culane_res18-lane-onnx: culane_res18-lane-onnx
   INFO    : Compiling c-culane_res18-lane-onnx took 118.637 seconds

   I’ve attached both YAMLs for reference:

• culane_res18-lane-onnx.yaml - link
• torch-imagenet.yaml - link

Great work so far ​@ChinmayMulgund, and thanks for the detailed response.

I wonder if this is the key issue - I feel like it’s suggesting that there’s an ONNX operation that’s not supported:

External op reshape_branching_point_to_cls_cls_dot_0_reduce_mean_dre found in the model (<class 'qtoolsv2.rewriter.operators.fqelements.Dequantize'> op). QTools may have issues quantizing this model.

As a test, maybe it’s worth trying to skip QTools to see if it allows us to test the pipeline and decoder logic?

I think that means adding the following to your culane_res18-lane-onnx.yaml YAML:

quantization:
  skip: true

And then we’ll have a better idea if the issue is strictly with the quantisation step, or if there’s something deeper going on in the model or pipeline setup. Let me know how it goes! 👍

I tried adding the quantization: skip: true block as suggested, but I’m still hitting the same QTools error during deploy:

(venv) aravind@aravind-H610M-H-V2:~/Desktop/voyager-sdk$ ./deploy.py customers/myculane/culane_res18-lane-onnx.yaml 
INFO    : Using device metis-0:1:0
…  
INFO    : Prequantizing c-culane_res18-lane-onnx: culane_res18-lane-onnx
INFO    : Quantizing network culane_res18-lane-onnx /home/aravind/Desktop/voyager-sdk/customers/myculane/culane_res18-lane-onnx.yaml culane_res18-lane-onnx
…  
ERROR   : RuntimeError: External op reshape_branching_point_to_cls_cls_dot_0_reduce_mean_dre found in the model (<class 'qtoolsv2.rewriter.operators.fqelements.Dequantize'> op). QTools may have issues quantizing this model.
…
ERROR   : Failed to prequantize c-culane_res18-lane-onnx: culane_res18-lane-onnx

It looks like the quantization skip flag isn’t taking effect—Voyager is still invoking QTools. I’ve attached my full YAML (culane_res18-lane-onnx.yaml) below so you can verify placement:

axelera-model-format: 1.0.0

name: culane_res18-lane-onnx
description: Ultrafast lane detector (Res18 → ONNX) on CULane

pipeline:
  - culane_res18-lane-onnx:
      input:
        type: image
      template_path: $AXELERA_FRAMEWORK/pipeline-template/torch-imagenet.yaml
      postprocess: []

models:
  culane_res18-lane-onnx:
    class: AxONNXModel
    class_path: $AXELERA_FRAMEWORK/ax_models/base_onnx.py
    task_category: Classification
    weight_path: $AXELERA_FRAMEWORK/customers/myculane/culane_res18.onnx
    input_tensor_layout: NCHW
    input_tensor_shape: [1, 3, 320, 1600]
    input_color_format: RGB
    num_classes: 4
    dataset: CULANE
    extra_kwargs:
      max_compiler_cores: 4

# --------------------
quantization:
  skip: true
# --------------------

datasets:
  CULANE:
    class: TorchvisionDataAdapter
    class_path: $AXELERA_FRAMEWORK/ax_datasets/torchvision.py
    data_dir_name: CULANE
    images_dir: .
    masks_dir: labels/laneseg_label_w16
    # 200–400 representative images for quantization:
    repr_imgs_dir_path: $AXELERA_FRAMEWORK/data/CULANE/calib
    # Full train+GT list for calibration (image + mask + existence flags):
    cal_data: $AXELERA_FRAMEWORK/data/CULANE/list/train_gt.txt
    # Validation list for post‐deployment accuracy measurement:
    val_data: $AXELERA_FRAMEWORK/data/CULANE/list/val_gt.txt

Hmm…

Maybe try with the quantization skip inside the models block? Like…

models:
  culane_res18-lane-onnx:
    class: AxONNXModel
    class_path: $AXELERA_FRAMEWORK/ax_models/base_onnx.py
    task_category: Classification
    weight_path: $AXELERA_FRAMEWORK/customers/myculane/culane_res18.onnx
    input_tensor_layout: NCHW
    input_tensor_shape: [1, 3, 320, 1600]
    input_color_format: RGB
    num_classes: 4
    dataset: CULANE
    extra_kwargs:
      max_compiler_cores: 4
    quantization:
      skip: true

(venv) aravind@aravind-H610M-H-V2:~/Desktop/voyager-sdk$ ./deploy.py customers/myculane/culane_res18-lane-onnx.yaml 
INFO    : Using device metis-0:1:0
INFO    : Compiling c-culane_res18-lane-onnx took 226.936 mseconds
ERROR   : quantization is not a valid key for model culane_res18-lane-onnx

Also tried in the pipeline as well:
 

(venv) aravind@aravind-H610M-H-V2:~/Desktop/voyager-sdk$ ./deploy.py customers/myculane/culane_res18-lane-onnx.yaml 
INFO    : Using device metis-0:1:0
INFO    : Compiling c-culane_res18-lane-onnx took 239.255 mseconds
ERROR   : quantization is not a valid property of a Task

Ah, my mistake - look like the “quantization” command isn’t recognised at al, no matter where it is. Better strip that back out. Let me ask if anyone has any ideas on this 👍

Okay, unfortunately it sounds like it could be because model includes a reduce_mean op, which Voyager SDK doesn’t support right now. We touched on that as being the issue earlier, but I didn’t click that it’s because that particular operation isn’t supported.

Thanks for clarifying that the root cause is the reduce_mean operation.

Given that, what are my options for moving forward?
Since this operation blocks my current ResNet-18-based lane detector, do you have any recommendations for a lane detection model or architecture that’s already known to work with Voyager SDK on Metis? 

I’d love to try that instead to get lane detection up and running.

I wonder if something like YOLOv5 could do it, if there was a suitable dataset?

That's already in the model zoo, so it'd run really well - just needs training. 

Possibly there are some existing datasets that would work, it could be modified?

I’ll experiment with a YOLOv5-based lane detector and a suitable dataset, and I’ll keep you posted on how it goes. Appreciate your help!

I had a quick look around online, and the BDD100K dataset might be a decent starting point. It looks like it’s got vehicle and lane annotations, and could be adapted for YOLOv5 to spot lane changes. You’d likely need to tweak the labels and fine-tune the model, but with some format conversion it might be worth a look!

Sounds great—thanks for pointing me to BDD100K! I’ll explore its lane annotations and look into YOLOv5.