Obviously inaccurate detection on Apple Silicon with MPS acceleration

[TANG617]

Search before asking

• I have searched the YOLOv5 issues and found no similar bug report.

YOLOv5 Component

Detection

Bug

I've read the tutorial from #7878 and solve the problem 'The operator 'aten::nonzero' is not currently supported on the MPS backend' from #8508

And the setup command is just like the following:

(Torch-GPU-Latest) timli@MacBook-Pro-by-Tang yolov5-master % PYTORCH_ENABLE_MPS_FALLBACK=1 python detect.py --device mps --source 0
detect: weights=yolov5s.pt, source=0, data=data/coco128.yaml, imgsz=[640, 640], conf_thres=0.25, iou_thres=0.45, max_det=1000, device=mps, view_img=False, save_txt=False, save_conf=False, save_crop=False, nosave=False, classes=None, agnostic_nms=False, augment=False, visualize=False, update=False, project=runs/detect, name=exp, exist_ok=False, line_thickness=3, hide_labels=False, hide_conf=False, half=False, dnn=False, vid_stride=1
YOLOv5 🚀 2022-9-25 Python-3.10.6 torch-1.14.0.dev20221023 MPS

Fusing layers... 
YOLOv5s summary: 213 layers, 7225885 parameters, 0 gradients
1/1: 0...  Success (inf frames 1920x1080 at 30.00 FPS)

It seems to work, but the result is obviously wrong.

And it crashes.

0: 384x640 2 persons, 10.5ms
Traceback (most recent call last):
  File "/Users/timli/MyLibrary/yolov5-master/detect.py", line 259, in <module>
    main(opt)
  File "/Users/timli/MyLibrary/yolov5-master/detect.py", line 254, in main
    run(**vars(opt))
  File "/Users/timli/opt/anaconda3/envs/Torch-GPU-Latest/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/Users/timli/MyLibrary/yolov5-master/detect.py", line 170, in run
    label = None if hide_labels else (names[c] if hide_conf else f'{names[c]} {conf:.2f}')
KeyError: 168

If I don't enable '--device mps', it works perfectly.
BTW, it seems that 'mps' boost about 5 times over 'cpu', which is exciting.

Environment

• YOLOv5 🚀 2022-9-25 Python-3.10.6 torch-1.14.0.dev20221023 MPS
• macOS 13.0 (22A379)
• Apple M1 Pro on MBP 14 2021 with 16GB Memory

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[github-actions]

👋 Hello @TANG617, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://ultralytics.com or email [email protected].

Requirements

Python>=3.7.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training, validation, inference, export and benchmarks on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

@TANG617 yes MPS seems to produce issues on detection models, but classification models work correctly. I don't know exactly where the problem lies.

[TANG617]

Thanks for the reply. I don't think I am able to solve it by myself 😢

[jgoo9410]

+1

[25benjaminli]

This needs more attention.

[jgoo9410]

#10178 (comment)

[glenn-jocher]

@lonnylundsten hello! Thank you for reaching out to us and bringing this up.

I understand your concern regarding using MPS backend with the YOLOv5 detection model. It seems that MPS produces issues on detection models, as mentioned by member of the YOLOv5 community. We appreciate your patience with this issue.

We are continuously working on improving the YOLOv5 model, the bug reports we credit to our community and try to solve them as efficiently as possible. Unfortunately, due to the limitations of this issue, we are unable to provide you with a more detailed response at the moment.

Nonetheless, please be assured that we are aware of this matter, and we will provide an update as soon as we have a fix for it. We encourage you to keep an eye on our updates, YOLOv5 documentation, and tutorials for more information.

If you have any other questions or concerns, please don't hesitate to keep asking. We're here to help.