Hello,
I’m trying to test the AxInferenceNet C++ Integration.
When I launch the test with the Voyager SDK’s given code and the yolov8s-coco model, the results of the detection are totally staggered.
Display of the result :
I followed the AxInferenceNet C++ Integration Tutorial and I’m using Voyager SDK with Ubuntu 22.04.
Could someone help me to understand the results and give me a bunch of idea to solve it ?
Thanks in advance.
Best answer by John Mullins
My apologies, I am on a slightly different version to you, this patch should do be fine for your version:
--- a/examples/axinferencenet/axinferencenet_example.cpp
+++ b/examples/axinferencenet/axinferencenet_example.cpp
@@ -116,7 +116,9 @@ reader_thread(cv::VideoCapture &input, Ax::InferenceNet &net)
net.end_of_input();
break;
}
- auto video = Ax::video_from_cvmat(frame->bgr, AxVideoFormat::BGR);
+ auto temp = std::move(frame->bgr);
+ cv::cvtColor(temp, frame->bgr, cv::COLOR_BGR2BGRA);
+ auto video = Ax::video_from_cvmat(frame->bgr, AxVideoFormat::BGRA);
net.push_new_frame(frame, video, frame->meta);
}
}
Hi PF05,
Thanks for reporting the issue. Just to confirm, are you using this AxInferenceNet example in our SDK, without any modifications?
Thanks
Hello might-be-david,
Thank you for your answer.
Yes I’m using this script and I only change the condition in the line 135 :
id > 0 into id >= 0
With this fix, it’s possible to detect person, because the people were detected as unknown.
But I still have the same problem.
Thanks in advance.
hi
We used your image and the axinferencet example but cannot reproduce the issue.
Good catch on the change on LoC 135 which is indeed needed to display the correct label.
Are you using a single image or a video file?
Hello Jonas,
Thank you for your feedback, I am using a video file, but I don’t understand how you can have this result.
I am testing with this command line :
examples/bin/axinferencenet_example build/yolov8s-coco/yolov8s-coco.axnet ‘video_file’
Is it wrong ?
And if in the file : axinferencenet_example.cpp, line 135, I don’t change this condition :
id > 0 into id >= 0 the persons are recognize as unknown.
Thanks in advance.
hi
I tested with a single image (this requires slight modification of the example script).
Does the full rendered video look correct? How did you obtain the single image from the rendered video?
Be aware that AxInferenceNet is a pipeline rather than single-image runtime.
So, multiple frames are being processed. I suspect the predictions bounding box is not from the same frame as you showing here.
I am using this video :
https://pixabay.com/videos/man-adult-road-walking-walk-76621/
The image is just a screenshot I took to show you a part of the result.
But in all the video, the results box are staggered.
Can you please report all details of your system:
SDK Version : v1.4
Card : Metis PCIe CARD
Host System : Ubuntu 22.04
The only thing I changed in the example is the condition in the file axinferencenet_example.cpp line 135 : id > 0 into id >= 0.
Thanks.
+2Are you able to test this with axinferencenet’s example traffic3_480p.mp4 example video perhaps,
Hi Spanner,
Yes I already tested with this video and the results are also staggered.
Result of 1 frame :
My testing process was :
axdownloadmodel yolov8s-coco
./inference.py yolov8s-coco fakevideo --frames=1 --no-display
examples/bin/axinferencenet_example build/yolov8s-coco/yolov8s-coco.axnet media/traffic3_480p.mp4
When I saw that the results were not good, I tested with a video with only 1 person (results in the picture on the first post). The results were again staggered and the person was detected has unknown.
I changed a condition to recognize the person but I still don’t have a good detection, whatever the model I use (yolov8s-coco, yolov8s-coco-onnx, yolov10s-coco-onnx, yolov5s-v7-coco, … ).
+2Okay, the team may have managed to reproduce it 😃 As a first step, could you run…
clinfo… and post what you see? It sounds like it could be falling back to a non-OpenCL mode, and thereby misses the coordinate scaling logic.
Let us know what you see from that. 👍
$clinfo
Number of platforms 1
Platform Name Portable Computing Language
Platform Vendor The pocl project
Platform Version OpenCL 2.0 pocl 1.8 Linux, None+Asserts, RELOC, LLVM 11.1.0, SLEEF, DISTRO, POCL_DEBUG
Platform Profile FULL_PROFILE
Platform Extensions cl_khr_icd cl_pocl_content_size
Platform Extensions function suffix POCL
Platform Name Portable Computing Language
Number of devices 1
Device Name pthread-13th Gen Intel(R) Core(TM) i7-13620H
Device Vendor GenuineIntel
Device Vendor ID 0x8086
Device Version OpenCL 1.2 pocl HSTR: pthread-x86_64-pc-linux-gnu-goldmont
Driver Version 1.8
Device OpenCL C Version OpenCL C 1.2 pocl
Device Type CPU
Device Profile FULL_PROFILE
Device Available Yes
Compiler Available Yes
Linker Available Yes
Max compute units 16
Max clock frequency 4700MHz
Device Partition (core)
Max number of sub-devices 16
Supported partition types equally, by counts
Supported affinity domains (n/a)
Max work item dimensions 3
Max work item sizes 4096x4096x4096
Max work group size 4096
Preferred work group size multiple (kernel) 8
Preferred / native vector sizes
char 16 / 16
short 16 / 16
int 8 / 8
long 4 / 4
half 0 / 0 (n/a)
float 8 / 8
double 4 / 4 (cl_khr_fp64)
Half-precision Floating-point support (n/a)
Single-precision Floating-point support (core)
Denormals Yes
Infinity and NANs Yes
Round to nearest Yes
Round to zero Yes
Round to infinity Yes
IEEE754-2008 fused multiply-add Yes
Support is emulated in software No
Correctly-rounded divide and sqrt operations Yes
Double-precision Floating-point support (cl_khr_fp64)
Denormals Yes
Infinity and NANs Yes
Round to nearest Yes
Round to zero Yes
Round to infinity Yes
IEEE754-2008 fused multiply-add Yes
Support is emulated in software No
Address bits 64, Little-Endian
Global memory size 14186590208 (13.21GiB)
Error Correction support No
Max memory allocation 4294967296 (4GiB)
Unified memory for Host and Device Yes
Minimum alignment for any data type 128 bytes
Alignment of base address 1024 bits (128 bytes)
Global Memory cache type Read/Write
Global Memory cache size 25165824 (24MiB)
Global Memory cache line size 64 bytes
Image support Yes
Max number of samplers per kernel 16
Max size for 1D images from buffer 268435456 pixels
Max 1D or 2D image array size 2048 images
Max 2D image size 16384x16384 pixels
Max 3D image size 2048x2048x2048 pixels
Max number of read image args 128
Max number of write image args 128
Local memory type Global
Local memory size 1310720 (1.25MiB)
Max number of constant args 8
Max constant buffer size 1310720 (1.25MiB)
Max size of kernel argument 1024
Queue properties
Out-of-order execution Yes
Profiling Yes
Prefer user sync for interop Yes
Profiling timer resolution 1ns
Execution capabilities
Run OpenCL kernels Yes
Run native kernels Yes
printf() buffer size 16777216 (16MiB)
Built-in kernels (n/a)
Device Extensions cl_khr_byte_addressable_store cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_3d_image_writes cl_khr_fp64 cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_fp64
NULL platform behavior
clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...) Portable Computing Language
clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...) Success [POCL]
clCreateContext(NULL, ...) [default] Success [POCL]
clCreateContextFromType(NULL, CL_DEVICE_TYPE_DEFAULT) Success (1)
Platform Name Portable Computing Language
Device Name pthread-13th Gen Intel(R) Core(TM) i7-13620H
clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU) Success (1)
Platform Name Portable Computing Language
Device Name pthread-13th Gen Intel(R) Core(TM) i7-13620H
clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL) Success (1)
Platform Name Portable Computing Language
Device Name pthread-13th Gen Intel(R) Core(TM) i7-13620H
ICD loader properties
ICD loader Name OpenCL ICD Loader
ICD loader Vendor OCL Icd free software
ICD loader Version 2.2.14
ICD loader Profile OpenCL 3.0
Thanks for the info
The following patch should get you moving forward:
--- a/examples/axinferencenet/axinferencenet_example.cpp
+++ b/examples/axinferencenet/axinferencenet_example.cpp
@@ -138,8 +138,8 @@ main(int argc, char **argv)
return;
}
auto frame_data = std::make_shared<Frame>();
- frame_data->rgb = std::move(frame);
- auto video = Ax::video_from_cvmat(frame_data->rgb, AxVideoFormat::RGB);
+ cv::cvtColor(frame, frame_data->rgb, cv::COLOR_RGB2RGBA);
+ auto video = Ax::video_from_cvmat(frame_data->rgb, AxVideoFormat::RGBA);
net->push_new_frame(frame_data, video, frame_data->meta);
};
Let us know how you get on.
Hi John Mullins, thank you for your return.
Is it possible to tell me what is the release of your code, because I am using the GitHub version and the line 138 is this one :
cv::FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 0xff, 0xff), 2);In the render function, and I already pull the latest changes.
Thank you in advance.
My apologies, I am on a slightly different version to you, this patch should do be fine for your version:
--- a/examples/axinferencenet/axinferencenet_example.cpp
+++ b/examples/axinferencenet/axinferencenet_example.cpp
@@ -116,7 +116,9 @@ reader_thread(cv::VideoCapture &input, Ax::InferenceNet &net)
net.end_of_input();
break;
}
- auto video = Ax::video_from_cvmat(frame->bgr, AxVideoFormat::BGR);
+ auto temp = std::move(frame->bgr);
+ cv::cvtColor(temp, frame->bgr, cv::COLOR_BGR2BGRA);
+ auto video = Ax::video_from_cvmat(frame->bgr, AxVideoFormat::BGRA);
net.push_new_frame(frame, video, frame->meta);
}
}
Already have an account? Login
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.