mAP Update

[glenn-jocher]

Aligned repository mAP computation to pycocotools, and improved mAP computation in general, which now exceeds darknet (using pycocotools to compare).

	ultralytics/yolov3 with pycocotools	darknet/yolov3
YOLOv3-320	51.8	51.5
YOLOv3-416	55.4	55.3
YOLOv3-608	58.2	57.9

sudo rm -rf yolov3 && git clone https://github.com/ultralytics/yolov3
# bash yolov3/data/get_coco_dataset.sh
sudo rm -rf cocoapi && git clone https://github.com/cocodataset/cocoapi && cd cocoapi/PythonAPI && make && cd ../.. && cp -r cocoapi/PythonAPI/pycocotools yolov3
cd yolov3

python3 test.py --save-json --conf-thres 0.001 --img-size 416
Namespace(batch_size=32, cfg='cfg/yolov3.cfg', conf_thres=0.001, data_cfg='cfg/coco.data', img_size=416, iou_thres=0.5, nms_thres=0.5, save_json=True, weights='weights/yolov3.weights')
Using cuda _CudaDeviceProperties(name='Tesla V100-SXM2-16GB', major=7, minor=0, total_memory=16130MB, multi_processor_count=80)
      Image      Total          P          R        mAP
Calculating mAP: 100%|█████████████████████████████████| 157/157 [08:34<00:00,  2.53s/it]
       5000       5000     0.0896      0.756      0.555
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.312
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.554
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.317
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.145
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.343
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.452
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.268
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.411
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.435
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.244
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.477
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.587
 
python3 test.py --save-json --conf-thres 0.001 --img-size 608 --batch-size 16
Namespace(batch_size=16, cfg='cfg/yolov3.cfg', conf_thres=0.001, data_cfg='cfg/coco.data', img_size=608, iou_thres=0.5, nms_thres=0.5, save_json=True, weights='weights/yolov3.weights')
Using cuda _CudaDeviceProperties(name='Tesla V100-SXM2-16GB', major=7, minor=0, total_memory=16130MB, multi_processor_count=80)
      Image      Total          P          R        mAP
Calculating mAP: 100%|█████████████████████████████████| 313/313 [08:54<00:00,  1.55s/it]
       5000       5000     0.0966      0.786      0.579
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.331
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.582
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.344
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.198
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.362
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.427
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.281
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.437
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.463
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.309
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.494
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.577

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

Enhancements to YOLOv3 object detection model, including expanded GPU support and updated model comparison metrics.

📊 Key Changes

• Added Google Cloud Platform (GCP) Quickstart guide to tutorials.
• Increased range of supported GPUs in GCP setup, including K80, P4, and T4, along with their respective pricing and performance metrics.
• Updated model comparison metrics (mAP, Precision, Recall) with darknet-based results.
• Changed Non-Maximum Suppression (NMS) IoU threshold from 0.45 to 0.5 in detection script to potentially reduce false positives.
• Updated test script to align NMS threshold and use pycocotools for mAP calculation during testing phase.
• Adjusted model's forward method to directly apply sigmoid function to class predictions.
• Optimizer’s momentum parameter description corrected to align with standard float notation.
• Revised GCP script to align it with the recent changes in the repository.
• Added more robust error checks in utilities functions.
• Included control code to switch the width-height multiplication method for bounding box processing.
• Minor adjustments throughout for code optimization and cleanup.

🎯 Purpose & Impact

• 🚀 Allow training on a wider range of GPUs on GCP, making the model more accessible and cost-effective for different users.
• 📈 Improve model performance comparability by presenting updated benchmarking stats against the darknet implementation.
• 🛠 Enhance model accuracy and evaluation precision with adjusted NMS threshold and mAP calculation alignment.
• 🌐 Broaden model utility with more robust error handling and cleaner codebase, supporting smoother future developments and maintenance.
• 💰 Provide better cost-to-performance insights for users considering GCP resources for training and inferencing.

These changes collectively aim to maintain an up-to-date and efficient YOLOv3 implementation, encouraging wider adoption and facilitating user understanding of model performance and costs.