Hi PF05,

Thanks for reporting the issue. Just to confirm, are you using this AxInferenceNet example in our SDK, without any modifications?

Thanks

Hello might-be-david,

Thank you for your answer.

Yes I’m using this script and I only change the condition in the line 135 :

id > 0 into id >= 0

With this fix, it’s possible to detect person, because the people were detected as unknown.

But I still have the same problem.

Thanks in advance.

hi ​@PF05 

We used your image and the axinferencet example but cannot reproduce the issue. 
Good catch on the change on LoC 135 which is indeed needed to display the correct label. 
Are you using a single image or a video file? 

 

Hello Jonas,

Thank you for your feedback, I am using a video file, but I don’t understand how you can have this result.

I am testing with this command line :

examples/bin/axinferencenet_example build/yolov8s-coco/yolov8s-coco.axnet ‘video_file’

Is it wrong ?

And if in the file : axinferencenet_example.cpp, line 135, I don’t change this condition : 

id > 0 into id >= 0 the persons are recognize as unknown.

Thanks in advance.

hi ​@PF05 , 
I tested with a single image (this requires slight modification of the example script). 
Does the full rendered video look correct? How did you obtain the single image from the rendered video?

Be aware that AxInferenceNet is a pipeline rather than single-image runtime. 
So, multiple frames are being processed. I suspect the predictions bounding box is not from the same frame as you showing here. 

I am using this video : 

https://pixabay.com/videos/man-adult-road-walking-walk-76621/

The image is just a screenshot I took to show you a part of the result.

But in all the video, the results box are staggered.

​@PF05 I tried with this video and also got a good result
Can you please report all details of your system: 
 

• SDK version
• Card (pcie or m.2)
• Host system
• All modifications made to the example

SDK Version : v1.4

Card : Metis PCIe CARD

Host System : Ubuntu 22.04

The only thing I changed in the example is the condition in the file axinferencenet_example.cpp line 135 : id > 0 into id >= 0.

Thanks.

Are you able to test this with axinferencenet’s example traffic3_480p.mp4 example video perhaps, ​@PF05? Just as a test, to see if it’s anything to do with the video file itelf, or somewhere else in the pipeline.

Hi Spanner,

Yes I already tested with this video and the results are also staggered.

Result of 1 frame :

My testing process was :

axdownloadmodel yolov8s-coco

./inference.py yolov8s-coco fakevideo --frames=1 --no-display

examples/bin/axinferencenet_example build/yolov8s-coco/yolov8s-coco.axnet media/traffic3_480p.mp4

When I saw that the results were not good, I tested with a video with only 1 person (results in the picture on the first post). The results were again staggered and the person was detected has unknown.

I changed a condition to recognize the person but I still don’t have a good detection, whatever the model I use (yolov8s-coco, yolov8s-coco-onnx, yolov10s-coco-onnx, yolov5s-v7-coco, … ).

Okay, the team may have managed to reproduce it 😃 As a first step, could you run…

clinfo

… and post what you see? It sounds like it could be falling back to a non-OpenCL mode, and thereby misses the coordinate scaling logic.

Let us know what you see from that. 👍

$clinfo

Number of platforms                               1
  Platform Name                                   Portable Computing Language
  Platform Vendor                                 The pocl project
  Platform Version                                OpenCL 2.0 pocl 1.8  Linux, None+Asserts, RELOC, LLVM 11.1.0, SLEEF, DISTRO, POCL_DEBUG
  Platform Profile                                FULL_PROFILE
  Platform Extensions                             cl_khr_icd cl_pocl_content_size
  Platform Extensions function suffix             POCL

  Platform Name                                   Portable Computing Language
Number of devices                                 1
  Device Name                                     pthread-13th Gen Intel(R) Core(TM) i7-13620H
  Device Vendor                                   GenuineIntel
  Device Vendor ID                                0x8086
  Device Version                                  OpenCL 1.2 pocl HSTR: pthread-x86_64-pc-linux-gnu-goldmont
  Driver Version                                  1.8
  Device OpenCL C Version                         OpenCL C 1.2 pocl
  Device Type                                     CPU
  Device Profile                                  FULL_PROFILE
  Device Available                                Yes
  Compiler Available                              Yes
  Linker Available                                Yes
  Max compute units                               16
  Max clock frequency                             4700MHz
  Device Partition                                (core)
    Max number of sub-devices                     16
    Supported partition types                     equally, by counts
    Supported affinity domains                    (n/a)
  Max work item dimensions                        3
  Max work item sizes                             4096x4096x4096
  Max work group size                             4096
  Preferred work group size multiple (kernel)     8
  Preferred / native vector sizes                 
    char                                                16 / 16      
    short                                               16 / 16      
    int                                                  8 / 8       
    long                                                 4 / 4       
    half                                                 0 / 0        (n/a)
    float                                                8 / 8       
    double                                               4 / 4        (cl_khr_fp64)
  Half-precision Floating-point support           (n/a)
  Single-precision Floating-point support         (core)
    Denormals                                     Yes
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 Yes
    Round to infinity                             Yes
    IEEE754-2008 fused multiply-add               Yes
    Support is emulated in software               No
    Correctly-rounded divide and sqrt operations  Yes
  Double-precision Floating-point support         (cl_khr_fp64)
    Denormals                                     Yes
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 Yes
    Round to infinity                             Yes
    IEEE754-2008 fused multiply-add               Yes
    Support is emulated in software               No
  Address bits                                    64, Little-Endian
  Global memory size                              14186590208 (13.21GiB)
  Error Correction support                        No
  Max memory allocation                           4294967296 (4GiB)
  Unified memory for Host and Device              Yes
  Minimum alignment for any data type             128 bytes
  Alignment of base address                       1024 bits (128 bytes)
  Global Memory cache type                        Read/Write
  Global Memory cache size                        25165824 (24MiB)
  Global Memory cache line size                   64 bytes
  Image support                                   Yes
    Max number of samplers per kernel             16
    Max size for 1D images from buffer            268435456 pixels
    Max 1D or 2D image array size                 2048 images
    Max 2D image size                             16384x16384 pixels
    Max 3D image size                             2048x2048x2048 pixels
    Max number of read image args                 128
    Max number of write image args                128
  Local memory type                               Global
  Local memory size                               1310720 (1.25MiB)
  Max number of constant args                     8
  Max constant buffer size                        1310720 (1.25MiB)
  Max size of kernel argument                     1024
  Queue properties                                
    Out-of-order execution                        Yes
    Profiling                                     Yes
  Prefer user sync for interop                    Yes
  Profiling timer resolution                      1ns
  Execution capabilities                          
    Run OpenCL kernels                            Yes
    Run native kernels                            Yes
  printf() buffer size                            16777216 (16MiB)
  Built-in kernels                                (n/a)
  Device Extensions                               cl_khr_byte_addressable_store cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_3d_image_writes cl_khr_fp64 cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_fp64

NULL platform behavior
  clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...)  Portable Computing Language
  clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...)   Success [POCL]
  clCreateContext(NULL, ...) [default]            Success [POCL]
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_DEFAULT)  Success (1)
    Platform Name                                 Portable Computing Language
    Device Name                                   pthread-13th Gen Intel(R) Core(TM) i7-13620H
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU)  Success (1)
    Platform Name                                 Portable Computing Language
    Device Name                                   pthread-13th Gen Intel(R) Core(TM) i7-13620H
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL)  Success (1)
    Platform Name                                 Portable Computing Language
    Device Name                                   pthread-13th Gen Intel(R) Core(TM) i7-13620H

ICD loader properties
  ICD loader Name                                 OpenCL ICD Loader
  ICD loader Vendor                               OCL Icd free software
  ICD loader Version                              2.2.14
  ICD loader Profile                              OpenCL 3.0

Here is the output of the clinfo.

Thanks for the info ​@PF05 , it looks like we are not using the CPU based OpenCL (upcoming versions of the SDK will). The issue is fairly straightforward, when OpenCL is not being used we need to feed axinferencenet RGBA input and we are passing RGB.
The following patch should get you moving forward:
 

--- a/examples/axinferencenet/axinferencenet_example.cpp
+++ b/examples/axinferencenet/axinferencenet_example.cpp
@@ -138,8 +138,8 @@ main(int argc, char **argv)
       return;
     }
     auto frame_data = std::make_shared<Frame>();
-    frame_data->rgb = std::move(frame);
-    auto video = Ax::video_from_cvmat(frame_data->rgb, AxVideoFormat::RGB);
+    cv::cvtColor(frame, frame_data->rgb, cv::COLOR_RGB2RGBA);
+    auto video = Ax::video_from_cvmat(frame_data->rgb, AxVideoFormat::RGBA);
     net->push_new_frame(frame_data, video, frame_data->meta);
   };

Let us know how you get on.

Hi John Mullins, thank you for your return.

Is it possible to tell me what is the release of your code, because I am using the GitHub version and the line 138 is this one : 

cv::FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 0xff, 0xff), 2);

In the render function, and I already pull the latest changes.

Thank you in advance.

My apologies, I am on a slightly different version to you, this patch should do be fine for your version:
 

--- a/examples/axinferencenet/axinferencenet_example.cpp
+++ b/examples/axinferencenet/axinferencenet_example.cpp
@@ -116,7 +116,9 @@ reader_thread(cv::VideoCapture &input, Ax::InferenceNet &net)
       net.end_of_input();
       break;
     }
-    auto video = Ax::video_from_cvmat(frame->bgr, AxVideoFormat::BGR);
+    auto temp = std::move(frame->bgr);
+    cv::cvtColor(temp, frame->bgr, cv::COLOR_BGR2BGRA);
+    auto video = Ax::video_from_cvmat(frame->bgr, AxVideoFormat::BGRA);
     net.push_new_frame(frame, video, frame->meta);
   }
 }

​@John Mullins Thank you very much it works !

Excuse me for the inconvenience, thank to your team and all the person who helped me !

Have a nice day !

PF05