Export yolov4-large models to ONNX and inference with TensorRT

[linghu8812]

Hello everyone, here is a code that can convert the ScaledYOLOv4 model to onnx model.

yolov4-large branch

For the branch of yolov4-large, the code is here: https://github.com/linghu8812/tensorrt_inference/tree/master/ScaledYOLOv4, now it can support the conversion of yolov4-p5, yolov4-p6, yolov4-p7 models. After the model exported, a TensorRT inference code was supplied, the TensorRT inference results has been shown below, which was consistent with the PyTorch results.

yolov4-csp and yolov4-tiny branch

As for the darknet supportted models, please refer this issue: AlexeyAB/darknet#7002, it shows how to convert darknet models to ONNX models, now it has supported yolov4, yolov4x-mish, yolov4-tiny models and so on.

[DaChaoXc]

yolov4-csp convert onnx is support? Me really looking forward!

[linghu8812]

yolov4-csp convert onnx is support? Me really looking forward!

@DaChaoXc Hello, yolov4-csp.weights can be exported to onnx model, using Yolov4/export_onnx.py and run:

python3 export_onnx.py --cfg_file cfg/yolov4-csp.cfg --weights_file yolov4-csp.weights --output_file yolov4-csp.onnx --sigmoid

inference can be done with:

./Yolov4_trt ../config-yolov4-csp.yaml ../samples/

[DaChaoXc]

@linghu8812 Hello ,I use https://github.com/CaoWGG/TensorRT-YOLOv4, the box is wrong, it seems the diffenence is yolo layer convert need add function of sigmoid?

[DaChaoXc]

@linghu8812 something wrong
Traceback (most recent call last):
File "/home/xc/xc/code/obj/YOLO/scaleyolov4-onnx/tensorrt_inference/Yolov4/export_onnx.py", line 1114, in
neck=args.neck, sigmoid=args.sigmoid)
File "/home/xc/xc/code/obj/YOLO/scaleyolov4-onnx/tensorrt_inference/Yolov4/export_onnx.py", line 1096, in main
onnx.checker.check_model(yolo_model_def)
File "/home/xc/xc/softwares/anaconda3/lib/python3.6/site-packages/onnx/checker.py", line 93, in check_model
C.check_model(model.SerializeToString())
onnx.onnx_cpp2py_export.checker.ValidationError: Node (116_upsample) has input size 2 not in range [min=3, max=4].

==> Context: Bad node spec: input: "115_convolutional_mish" input: "116_upsample_scale" output: "116_upsample" name: "116_upsample" op_type: "Resize" attribute { name: "mode" s: "nearest" type: STRING }

[DaChaoXc]

@linghu8812 something wrong
Traceback (most recent call last):
File "/home/xc/xc/code/obj/YOLO/scaleyolov4-onnx/tensorrt_inference/Yolov4/export_onnx.py", line 1114, in
neck=args.neck, sigmoid=args.sigmoid)
File "/home/xc/xc/code/obj/YOLO/scaleyolov4-onnx/tensorrt_inference/Yolov4/export_onnx.py", line 1096, in main
onnx.checker.check_model(yolo_model_def)
File "/home/xc/xc/softwares/anaconda3/lib/python3.6/site-packages/onnx/checker.py", line 93, in check_model
C.check_model(model.SerializeToString())
onnx.onnx_cpp2py_export.checker.ValidationError: Node (116_upsample) has input size 2 not in range [min=3, max=4].

==> Context: Bad node spec: input: "115_convolutional_mish" input: "116_upsample_scale" output: "116_upsample" name: "116_upsample" op_type: "Resize" attribute { name: "mode" s: "nearest" type: STRING }

solved ,onnx must 1.5.0

[linghu8812]

@DaChaoXc Hello,
for the new feature [yolo] new_coords=1 in yolov4-csp and yolov4x-mish, the decode boxes function has been changed, I add sigmoid at the end just for the convience of postprocess. For more information, please refer this: AlexeyAB/darknet#6987 (comment)

[DaChaoXc]

i changed the compute way, it works!
// x1,y1,x2,y2,cls,conf,batch_id
//data[0] = (row + sigmoid(input[begin_id])) * down_stride;
//data[1] = (col + sigmoid(input[begin_id+stride])) * down_stride;
//data[2] = expf(input[begin_id+2stride]) * (float)shared_anchors[2 * anchor_id];
//data[3] = expf(input[begin_id+3stride]) * (float)shared_anchors[2 * anchor_id + 1];

data[0] = (row + sigmoid(input[begin_id]) * 2 - 0.5) * down_stride;
data[1] = (col + sigmoid(input[begin_id+stride]) * 2 - 0.5) * down_stride;
data[2] = pow(sigmoid(input[begin_id+2stride]) * 2, 2) * (float)shared_anchors[2 * anchor_id];
data[3] = pow(sigmoid(input[begin_id+3stride]) * 2, 2) * (float)shared_anchors[2 * anchor_id + 1];

[linghu8812]

@DaChaoXc yes, if add sigmoid op to onnx model, the sigmoid op will run with tensorrt, the code can be simplified as:

data[0] = (row + input[begin_id] * 2 - 0.5)down_stride;
data[1] = (col + input[begin_id+stride] * 2 - 0.5)down_stride;
data[2] = pow(input[begin_id+2stride] * 2, 2) (float)shared_anchors[2anchor_id];
data[3] = pow(input[begin_id+3stride] * 2, 2) * (float)shared_anchors[2*anchor_id + 1];

[linghu8812]

@linghu8812 something wrong
Traceback (most recent call last):
File "/home/xc/xc/code/obj/YOLO/scaleyolov4-onnx/tensorrt_inference/Yolov4/export_onnx.py", line 1114, in
neck=args.neck, sigmoid=args.sigmoid)
File "/home/xc/xc/code/obj/YOLO/scaleyolov4-onnx/tensorrt_inference/Yolov4/export_onnx.py", line 1096, in main
onnx.checker.check_model(yolo_model_def)
File "/home/xc/xc/softwares/anaconda3/lib/python3.6/site-packages/onnx/checker.py", line 93, in check_model
C.check_model(model.SerializeToString())
onnx.onnx_cpp2py_export.checker.ValidationError: Node (116_upsample) has input size 2 not in range [min=3, max=4].
==> Context: Bad node spec: input: "115_convolutional_mish" input: "116_upsample_scale" output: "116_upsample" name: "116_upsample" op_type: "Resize" attribute { name: "mode" s: "nearest" type: STRING }

solved ,onnx must 1.5.0

@DaChaoXc I have add export onnx model with opset 10 code, onnx version has been no longer a restriction.

[tjuskyzhang]

A yolov4-csp/p5/p6/p7-tensorrt project
https://github.com/tjuskyzhang/Scaled-YOLOv4-TensorRT

[huyhoangle86]

hey do you have code to export Scaled Yolov4 CSP version to Onnx ? @linghu8812

[lantudou]

#371

Hi, Could you update your code for the newest branch ?