running from DVP camera

[kervel]

Hello,

We tried to get the C code running on K210 maix bit, using the latest version of the standalone SDK.
I see your main.c contains code for DVP but its commented. so we tried to bring it to life but so far not succesfull.

I have some questions:

• we get 0.2 FPS on mobile V1 with dm=0.75 .. is this realistic ? the animated gif seemed much faster. we had to hack the latest standalone sdk to get it working (remove an assert in dvp.c !is_memory_cache)
• we don't get meaningful results (always the same rectangle). might be a bug in our trials.

i guess you used this repo mainly for creating a model rather than using it as a base for inference on K210. what would be the best practises for doing inference based on the models here ?

/cc @GillesKap

Greetings,
Frank

[krishnak]

Did you create your KModel with latest nncase? it is bound to have operations that are performed on CPU which might increase the inference time. I can offer you commercial consultancy if you are interested

[zhen8838]

0.2fps may be because you didn't rewrite region layer.c. the real-time demo detection code I showed was rewritten by me.

The output is the same, You should refer to the official real-time Yolo detection demo

[kervel]

Hello @zhen8838 and @krishnak, thanks for your reply ... i'll check but i'm not sure to which code you refer (official real-time demo). could you give me a pointer ?

as for commercial consultancy, thanks for the offer. i'm not sure if that will be possible in my project but at least i need to prove feasibility first.

[zhen8838]

demo link: https://github.com/kendryte/nncase/tree/master/examples/20classes_yolo

[kervel]

Hello @zhen8838, thanks for the link! we didn't look at that one because it seems it contains also a preloaded image (image.c) and no image recognition on the DVP signal. or do you want to indicate this is a good base for starting a conversion to DVP based image recognition ?

[zhen8838]

which board did you use ? my code and kendryte demo code only support kendryte KD233 board.

The DVP driver will auto convert rgb565 image to rbg888 format and put into kpu memory, so you just set kpu memory address when board init.

image.c just contain height, width information.

[kervel]

hi @zhen8838 sorry for the confusion. I didn't use it on a board yet, i just tried to find out if the demo is a real time demo (i'll probably use a maix bit in the first stage)

so i looked at https://github.com/kendryte/nncase/blob/master/examples/20classes_yolo/k210/kpu_20classes_example/main.c#L248 and i saw the kmodel uses a reference to gImage_image which is defined in https://raw.githubusercontent.com/kendryte/nncase/master/examples/20classes_yolo/k210/kpu_20classes_example/image.c as containing pixel data.

i compared this to https://github.com/kendryte/nncase/blob/master/examples/facedetect_landmark/k210/facedetect_landmark_example/main.c which does contain code to get image data from the camera. Should i try to combine the two ?

thanks!

[zhen8838]

yes.
The facedetect_landmark is realtime demo.
The yolo demo just infer once, you can custom image array.