Gaussian YOLOv3 (+3.1% [email protected] on COCO) , (+3.0% [email protected] on KITTI) , (+3.5% [email protected] on BDD)

[phucnhs]

Have you tried the Gaussian object detection method?

• cfg files: https://github.com/jwchoi384/Gaussian_YOLOv3/tree/master/cfg
• BDD100k weights file: https://drive.google.com/open?id=1Eutnens-3z6o4LYe0PZXJ1VYNwcZ6-2Y
• paper: https://arxiv.org/abs/1904.04620v2
• source code: https://github.com/jwchoi384/Gaussian_YOLOv3

[AlexeyAB]

What do you mean?
Can you provide URL to article/paper?

[phucnhs]

This is object detection by new method
this is paper : https://arxiv.org/pdf/1904.04620.pdf
Here is the code, you can refer to it because I am not confident about this code : https://github.com/jwchoi384/Gaussian_YOLOv3

[lsd1994]

Furthermore, on the COCO dataset [14], the AP of Gaussian YOLOv3 is 36.1, which is 3.1 higher than YOLOv3. In particular, the AP75 (i.e., strict metric) of Gaussian YOLOv3 is 39.0, which is 4.6 higher than that of YOLOv3.

It seems Gaussian YOLOv3 is better than YOLOv3 on COCO dataset, especially in strict metric.

[AlexeyAB]

So

• Yolov3 default 608x608 - 33.0% [email protected] and 34.4% [email protected] https://arxiv.org/pdf/1804.02767v1.pdf

• Yolov3 Gaussian 608x608 - 36.1% [email protected] and 39.0% [email protected] https://arxiv.org/pdf/1904.04620v2.pdf

---

https://arxiv.org/pdf/1902.09630v2.pdf

• Yolov3 default MSE 608x608 - 31.4% [email protected] and 33.3% [email protected] - testdev 2018
• Yolov3 GIoU 608x608- 33.3% [email protected] and 36.2% [email protected] - testdev 2018
• Yolov3 GIoU 608x608- 33.5% [email protected] and 35.9% [email protected] - val 2014

GIoU: #3249

[sctrueew]

@AlexeyAB Hi,

Have you tried to combine Gaussian and GIoU? If yes, Could you share the result?

Thanks

[tuteming]

where I can get the Gaussian YOLOv3 config file?

[AlexeyAB]

@tuteming

cfg files: https://github.com/jwchoi384/Gaussian_YOLOv3/tree/master/cfg
BDD100k weights file: https://drive.google.com/open?id=1Eutnens-3z6o4LYe0PZXJ1VYNwcZ6-2Y

[AlexeyAB]

I added yolo_v3_tiny_pan3 matrix_gaussian aa_ae_mixup.cfg.txt model.

More: #3114 (comment)

---

Model (cfg & weights) network size = 544x544	Training chart	Validation video	BFlops	Inference time RTX2070, ms	mAP, %
yolo_v3_tiny_pan3 matrix_gaussian aa_ae_mixup.cfg.txt and weights-file		video	13	19.0 ms	57.2%

[Kyuuki93]

@AlexeyAB Hi, I trained yolov3-spp.cfg and yolov3-spp-gs.cfg in my Custom Data, the yolov3-spp-gs.cfg just replaced [yolo] to [Gaussian yolo]. My data was one-class dataset with 130k images, but for some reason there no validation data, so I can't report the mAP value.
There are some observation:

1. Gaussian yolo got high avgloss in the train,
2. Network with Gaussian yolo layer get lower confidence of object,
3. Network with Gaussian yolo layer decrease the wrong detected boxes but aggravated blinking issue in video

I think maybe in the test, * (1.0 - uc_aver)in line 454 of gaussian_yolo_layer.c was not necessary .

Three some video results, extract code :xqxu
all results in the -thresh 0.5,
spp.mp4 was the result of yolov3-spp.cfg
spp-gs.mp4 was the result of yolov3-spp-gs.cfg
spp-gs2.mp4 was the result of yolov3-spp-gs.cfg and removed * (1.0 - uc_aver)in [gaussian_yolo_layer.c]

[AlexeyAB]

My data was one-class dataset with 130k images, but for some reason there no validation data, so I can't report the mAP value.

Just set valid=train.txt in obj.data file

Gaussian yolo got high avgloss in the train,
Network with Gaussian yolo layer get lower confidence of object,
Network with Gaussian yolo layer decrease the wrong detected boxes but aggravated blinking issue in video

You should use lower -threshold 0.2 or 0.15` for [Gaussian yolo] instead of default 0.24

I think maybe in the test, * (1.0 - uc_aver)in line 454 of gaussian_yolo_layer.c was not necessary .

It is necessary if you want to have high [email protected]

[AlexeyAB]

@zpmmehrdad

I added GIoU to the [Gaussian_yolo] layer.

So now you can use for training:

[Gaussian_yolo]
mask = 0,1,2
anchors = 7,10, 14,24, 27,43, 32,97, 57,64, 92,109, 73,175, 141,178, 144,291
classes=10
num=9
jitter=.3
ignore_thresh = .5
truth_thresh = 1
iou_thresh=0.213
uc_normalizer=1.0
cls_normalizer=1.0
iou_normalizer=0.5
iou_loss=giou
scale_x_y=1.1
random=1

[yrc08]

@AlexeyAB
Hi!
I want to use the Gaussian_yolov3_BDD.cfg training to use darknet53.conv.74 or Gaussian_yolov3_BDD.weights.

[AlexeyAB]

@yrc08
Use darknet53.conv.74 for training and Gaussian_yolov3_BDD.weights for detection.

[yrc08]

@AlexeyAB
Hi!
In other words, I want to use darknet53.conv.74 when I want to train my own data set. Is my understanding correct?

[LukeAI]

Just to share, I found the gaussian yolo gave me a small boost in mAP, around that claimed in the paper (which is unusual in my experience!)

Gaussian YOLO

Yolov3_spp_swish_scale

I'm currently trying out Gaussian_yolo_spp_swish...

[AlexeyAB]

I'm currently trying out Gaussian_yolo_spp_swish...

Also try then to add params to each of [Gaussian_yolo] layer and train. It requires the latest version of Darknet

iou_thresh=0.3
scale_x_y=1.1

So compare Gaussian_yolo_spp_swish... vs Gaussian_yolo_spp_swish_scale_iou|_thresh...

[LukeAI]

I'm currently trying out Gaussian_yolo_spp_swish...

Also try then to add params to each of [Gaussian_yolo] layer and train. It requires the latest version of Darknet

iou_thresh=0.3
scale_x_y=1.1

So compare Gaussian_yolo_spp_swish... vs Gaussian_yolo_spp_swish_scale_iou|_thresh...

So add the same to every gaussian_yolo? As opposed to scale_x_y=1.05 , 1.1, 1.2 in the different layers?

[LukeAI]

what is the iou_thresh?

[AlexeyAB]

As opposed to scale_x_y=1.05 , 1.1, 1.2 in the different layers?

Or you can add different values for different yolo-layers: scale_x_y=1.05 (for 17x17) , 1.1 (34x34) 1.2 (68x68).

what is the iou_thresh?

Read about iou_thresh=0.213:

Our loss initially only paired the best anchor (out of all 9) to each target. We changed this to pair a target to any anchor above iou_thres = 0.213 (all paired anchors generate losses). So for example all 9 anchors might be paired or none at all. This change boosted mAP by about 0.07 to 0.47.

#3114 (comment)

[AlexeyAB]

@nyj-ocean

• Did you get Recall by using ./darknet detector map. ... command? You should compare Recall only if the same Precision (so you should play with -thresh 0.x flag).

I want to know these improvement in mAP and Recall of yolov3+Gaussian+CIoU are just for random or for the added module Gaussian+CIoU?

Just train again yolov3 and yolov3+Gaussian+CIoU

Also show chart.png of both models.

[AlexeyAB]

@LukeAI

iou_normalizer= will affect only for losses C/D/GIoU:

I fixed it.

Now iou_normalizer=0.1 and uc_normalizer=0.1 in [Gaussian_yolo] affect even without C/D/GIoU.

[AlexeyAB]

@WongKinYiu

It seems that we should use

1. iou_loss=1 for [yolo]
2. iou_loss=0.01 for [yolo] + C/D/GIoU (may be 0.07 )
3. iou_loss=0.1 and uc_loss=0.1 for [Gaussian_yolo] (may be 0.3 )
4. iou_loss=0.01 and uc_loss=0.01 for [Gaussian_yolo] + C/D/GIoU (may be 0.07 )

I won’t be surprised if the main effect of improving the AP75 (and decreasing AP50) is not from the good GIoU algorithm, but just from higher values ​​of iou_loss. I.e. the same effect we can achieve by using default [yolo] with default mse-loss but with iou_normalizer=10 or 100

---

1. [yolo] has class_loss and iou_loss ~= 10

---

2. [yolo]+GIoU has class_loss and iou_loss ~= 1000

---

2. [Gaussian_yolo] has high iou_loss and uc_loss ~100

---

3. [Gaussian_yolo] + GIoU has much higher iou_loss and uc_loss ~1000

---

[HagegeR]

so just multiplying the error is what make the network learn faster, but I think the way the loss is calculated result in better ap75

[AlexeyAB]

@HagegeR
A higher iou_loss shifts the priorities to more accurate IoU (coordinates and sizes) of the object (increases AP75), but to a less accurate prediction of the class_id (decreases AP50).

C/D/GIoU increases AP75, but decreases AP50.

[glenn-jocher]

Yes, normalizers have a tremendous effect on performance. In our repo we've spent thousands of GPU hours evolving these normalizers on COCO along with all of our other hyperparameters. See ultralytics/yolov3#392

Our 3 main balancing hyperparameters are here, though they are changing every few weeks as new evolution results come in. They must change depending on image size, as well as class count, and occupancy (how many objects per image).

hyp = {'giou': 3.31,  # giou loss gain
       'cls': 42.4,  # cls loss gain
       'obj': 52.0,  # obj loss gain (*=img_size/416 if img_size != 416)}

[glenn-jocher]

I have updated mAPs now from https://github.com/ultralytics/yolov3#map using the default hyperparameters and default training settings on COCO2014, starting yolov3-spp.cfg from scratch.

	Size	COCO mAP @0.5...0.95	COCO mAP @0.5
YOLOv3-tiny YOLOv3 YOLOv3-SPP YOLOv3-SPP ultralytics	320	14.0 28.7 30.5 35.4	29.1 51.8 52.3 54.3
YOLOv3-tiny YOLOv3 YOLOv3-SPP YOLOv3-SPP ultralytics	416	16.0 31.2 33.9 39.0	33.0 55.4 56.9 59.2
YOLOv3-tiny YOLOv3 YOLOv3-SPP YOLOv3-SPP ultralytics	512	16.6 32.7 35.6 40.3	34.9 57.7 59.5 60.6
YOLOv3-tiny YOLOv3 YOLOv3-SPP YOLOv3-SPP ultralytics	608	16.6 33.1 37.0 40.9	35.4 58.2 60.7 60.9

[AlexeyAB]

@glenn-jocher

is YOLOv3-SPP with default hyperparameters giou=1 cls=1 obj=1, while YOLOv3-SPP ultralytics with these hyperparameters?

hyp = {'giou': 3.31, # giou loss gain
'cls': 42.4, # cls loss gain
'obj': 52.0, # obj loss gain (*=img_size/416 if img_size != 416)}

So good hyper parameters for GIoU:

[yolo]
iou_normalizer=0.07    # giou hyperparameter: 3.31 / ((52+42)/2) = 0.07
cls_normalizer=1.0     # obj hyperparameter
iou_loss=giou

---

img_size/416 is it image size or network size? Since every image will be resized to the network size.

[glenn-jocher]

@AlexeyAB 416 is the network size. So for example if the network is trained at 320, the objectness hyperparameter would be multiplied by 320/416. I found this helped keep the balance when network size changed.

Yes, roughly speaking we found an optimal GIoU gain is about 10 times smaller than cls and obj, and that obj and cls seem to optimally be of similar magnitude. I average the elements in each yolo layer, and sum the 3 values for the 3 yolo layers. So for example obj loss = mean(obj_loss_yolo_layer1) + mean(obj_loss_yolo_layer2) + mean(obj_loss_yolo_layer3). The chart below shows the loss after multiplying by the hyps:

[glenn-jocher]

I think the fundamental concept is that if each of the 3 loss components are equally important to the solution (and I believe they are), then they should roughly each be represented equally in the total loss (i.e. 1/3, 1/3, 1/3). This is probably the best place to start when in doubt, and then fine tune from there if you can.

[AlexeyAB]

@glenn-jocher

hyp = {'giou': 3.31, # giou loss gain
'cls': 42.4, # cls loss gain
'obj': 52.0, # obj loss gain (*=img_size/416 if img_size != 416)}

Are these hyperparameters good for C/D/GIoU only, or also the same values for the default MSE-loss?

[glenn-jocher]

@AlexeyAB these values are only for C/D/GIoU. I abandoned the MSE loss a while ago, as I had problems balancing the x-y losses with the width-height losses, since their loss functions are quite different. Also the MSE w-h loss tended to be unstable, which is solved by GIoU. So the ultralytics/yolov3 repo does not have MSE loss capability anymore, GIoU is the new default for box losses.

[WongKinYiu]

@AlexeyAB Hello,

1. iou_loss=1 for [yolo]
2. iou_loss=0.01 for [yolo] + C/D/GIoU (may be 0.07 )
3. iou_loss=0.1 and uc_loss=0.1 for [Gaussian_yolo] (may be 0.3 )
4. iou_loss=0.01 and uc_loss=0.01 for [Gaussian_yolo] + C/D/GIoU (may be 0.07 )

Do you mean:

• set iou_normalizer=1 for [yolo]
• set iou_normalizer=0.01 for [yolo] + C/D/GIoU
• set iou_normalizer=0.1 and uc_normalizer=0.1 for [Gaussian_yolo]
• and set iou_normalizer=0.01 and uc_normalizer=0.01 for [Gaussian_yolo] + C/D/GIoU
  of [yolo]/[Gaussian_yolo] layers in cfg?

@glenn-jocher Hello,

Do you have an automatic or systematical method to find optimal value of hyper-parameters? #4147 (comment)

[glenn-jocher]

@WongKinYiu yes we have an automatic hyperparameter search method. It's based on a genetic algorithm, which evolves the hyperparameters from an initial point to a minima. It does not compute a gradient, it simply mutates successive generations based on combinations of the most successful parents. Hyperparameter evolution is run the same as the training command, except with the --evolve flag included. We use the mAP after 10% of full training as the evolutionary fitness metric. This command evolves an unlimited number of generations using https://github.com/ultralytics/yolov3 (starting from the current hyps):

while true
do
  python3 train.py --data data/coco.data --epochs 27 --weights '' --evolve
done

See ultralytics/yolov3#392 for full details.

[glenn-jocher]

An example snapshot of the results are here. Mutation fitness (weighted mAP and F1 combination) is on the y axis (higher is better), hyperparameter value is on the x axis. Each orange point is a random genetic mutation. The highest fitness offspring is in blue. These are dynamically changing results, with better fitness evolving week to week etc.

from utils.utils import *; plot_evolution_results(hyp)

[glenn-jocher]

BTW, all of this does take a long time on large datasets. If you assume you can train COCO on 1 GPU in 10 days, then training to 10% of the full epochs (each orange point above) requires 1 GPU-day. To get good results you want to train for at least 100 generations, which on COCO would be 100 GPU-days, though on smaller datasets would obviously work much faster.

I would say 100 generations is a minimum evolution time (starting from a good, manually selected starting point). 200 to 300 generations is more ideal, if you have the hardware available.

[WongKinYiu]

@glenn-jocher

I have only few gpus, I think I can not afford the computation of training hyper-parameter searching.

[glenn-jocher]

@WongKinYiu yes, unfortunately it requires significant resources. If you leave something like this (https://lambdalabs.com/deep-learning/workstations/4-gpu/basic/customize) running 24/7 you can produce about 50 orange points a week on COCO. We tried using shortcuts, like evolving from the result after just 1% of epochs:

 python3 train.py --data data/coco.data --epochs 3 --weights '' --evolve

but it did not work well. It evolved hyps for best results after 3 epochs, but they did not generalize well to 273 epochs.

[AlexeyAB]

@WongKinYiu

Do you mean:

set iou_normalizer=1 for [yolo]
set iou_normalizer=0.01 for [yolo] + C/D/GIoU
set iou_normalizer=0.1 and uc_normalizer=0.1 for [Gaussian_yolo]
and set iou_normalizer=0.01 and uc_normalizer=0.01 for [Gaussian_yolo] + C/D/GIoU
of [yolo]/[Gaussian_yolo] layers in cfg?

Yes, but @glenn-jocher found the best 0.07 multiplier for GIoU:

• set iou_normalizer=1 for [yolo]
• set iou_normalizer=0.07 for [yolo] + C/D/GIoU
• set iou_normalizer=0.1 and uc_normalizer=0.1 for [Gaussian_yolo]
• and set iou_normalizer=0.07 and uc_normalizer=0.07 for [Gaussian_yolo] + C/D/GIoU

[AlexeyAB]

@WongKinYiu

Use: #4430

[net]

learning_rate=0.00261
momentum=0.949

...

[yolo]
iou_normalizer=0.07
cls_normalizer=1.0
iou_loss=giou    # or ciou
nms_kind=greedynms
beta1=0.6

[nyj-ocean]

@AlexeyAB
Since I found that nms_kind=greedynms beta1=0.6 is not in Gaussian_yolov3_BDD.cfg(this repo),but in coco-ciou.cfg(DIoU-darknet repo)
Should I add the two following hyperparameters to my yolov3+Gaussian+CIoU.cfg when training ?

nms_kind=greedynms
beta1=0.6

[AlexeyAB]

@nyj-ocean Yes you can. Since it doesn't affect during training, you can leave or comment these 2 lines after training, and check the mAP.

[WongKinYiu]

@AlexeyAB Thanks.

Use: #4430

[net]

learning_rate=0.00261
momentum=0.949

...

[yolo]
iou_normalizer=0.07
cls_normalizer=1.0
iou_loss=giou    # or ciou
nms_kind=greedynms
beta1=0.6