I often use this project, but I saw it abandoned and without a public repository on github. Also, part of the project remained unfinished for a long time. I implemented some of the author's ideas and decided to make the results publicly available.
This package wraps a facebook C++ implementation of COCO-eval operations found in the pycocotools package. This implementation greatly speeds up the evaluation time for coco's AP metrics, especially when dealing with a high number of instances in an image.
For our use case with a test dataset of 5000 images from the coco val dataset. Testing was carried out using the mmdetection framework and the eval_metric.py script. The indicators are presented below.
Visualization of testing comparison.ipynb available in directory examples/comparison Tested with yolo3 model (bbox eval) and yoloact model (segm eval)
| Type | COCOeval | COCOeval_faster | Profit |
|---|---|---|---|
| bbox | 18.477 sec. | 7.345 sec. | 2.5x faster |
| segm | 29.819 sec. | 15.840 sec. | 2x faster |
This package contains a faster implementation of the
pycocotools COCOEval class.
To import and use COCOeval_faster type:
from faster_coco_eval import COCO, COCOeval_faster
....For usage, look at the original COCOEval class documentation.
from faster_coco_eval import COCO
from faster_coco_eval.extra import Curves
cocoGt = COCO(....)
cocoDt = cocoGt.loadRes(....)
cur = Curves(cocoGt, cocoDt, iou_tresh=0.5, iouType='segm')
cur.plot_pre_rec(plotly_backend=False)- numpy
- plotly (optional if extra.Curve usage)
- fix by ViTrox https://github.com/vitrox-technologies/faster_coco_eval
- missing file issue
- issue discovered by torchmetric
- fstring for python3.7
- Windows compilation
- rework math_matches function. moved to faster_eval_api
- Moved calculations from python to c++
- Separated extra classes
- Added new sample data
- append mIoU based on TP pred.
- append mAUC based on Coco pre/rec.
- rework mask code
- change np.float to float (numpy deprecations)
- remove pycocotools dependencies
- clean c/c++ code
- Implemented of mean IoU for TP
- set FP-red FN-blue
- Removed own implementation of pre-rec
- Switched to the implementation of pre-rec calculation from COCO eval
- Lost backward compatibility
- Implemented output fp/fn/tp + gt to pictures
- bug fix with pre-rec curve
- rework error calc (tp/fp/fn)
- change image plot to plotly
- append docker auto builder
- append native iou calc (slow but accurate)
- rework auc calc with link
- rebuild plotly backend
- Segm bug-fix
- Append fp fn error analysis via curves
- Append confusion matrix
- Append plotly backend support for ROC / AUC
- Redesigned curves
- Reworked data preload
- Append csrc to setup
- Build sdist Package
- Wrap c++ code
- Get it to compile
- Add COCOEval class wraper
- Remove detectron2 dependencies
- Remove torch dependencies
- Append unittest
- Append ROC / AUC curves
- Check if it works on windows
- Remove pycocotools dependencies
- Remove matplotlib dependencies
The original module was licensed with apache 2, I will continue with the same license. Distributed under the apache version 2.0 license, see license for more information.