The operator 'aten::nonzero' is not currently supported on the MPS backend

[abhimanyuchadha96]

Search before asking

• I have searched the YOLOv5 issues and found no similar bug report.

YOLOv5 Component

Detection

Bug

Environment

No response

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[glenn-jocher]

@abhimanyuchadha96 thanks for the bug report! This is interesting, are you running on Apple M1 hardware?

[glenn-jocher]

@abhimanyuchadha96 have you tried setting environment variable PYTORCH_ENABLE_MPS_FALLBACK=1 as the error message says?

[abhimanyuchadha96]

Yes, I have a Mac Studio which has the M1 Ultra Chip.
Also, PyTorch has opened a Github issue where it is asking people to vote for which op they want.

[abhimanyuchadha96]

@glenn-jocher where should I set up that flag though? I couldn't find the exact place.

[glenn-jocher]

i.e. to set environment variable before running detect.py just do this:

PYTORCH_ENABLE_MPS_FALLBACK=1 python detect.py

[glenn-jocher]

@abhimanyuchadha96 I don't have access to M1 hardware today, but you could debug this a bit to see if there's some alternative indexing implementation that works. If you find a solution please let us know or also submit a PR to help future users.

The indexing is right here on L820 that is causing the error. xc is a boolean tensor to indicate which anchors are above confidence threshold, xi is just the current image index, i.e. 0, 1, 2 etc.

yolov5/utils/general.py

Lines 800 to 821 in 27d831b

	xc = prediction[..., 4] > conf_thres # candidates
	# Checks
	assert 0 <= conf_thres <= 1, f'Invalid Confidence threshold {conf_thres}, valid values are between 0.0 and 1.0'
	assert 0 <= iou_thres <= 1, f'Invalid IoU {iou_thres}, valid values are between 0.0 and 1.0'
	# Settings
	# min_wh = 2 # (pixels) minimum box width and height
	max_wh = 7680 # (pixels) maximum box width and height
	max_nms = 30000 # maximum number of boxes into torchvision.ops.nms()
	time_limit = 0.3 + 0.03 * bs # seconds to quit after
	redundant = True # require redundant detections
	multi_label &= nc > 1 # multiple labels per box (adds 0.5ms/img)
	merge = False # use merge-NMS
	t = time.time()
	output = [torch.zeros((0, 6), device=prediction.device)] * bs
	for xi, x in enumerate(prediction): # image index, image inference
	# Apply constraints
	# x[((x[..., 2:4] < min_wh) | (x[..., 2:4] > max_wh)).any(1), 4] = 0 # width-height
	x = x[xc[xi]] # confidence

[glenn-jocher]

@abhimanyuchadha96 left a comment in pytorch/pytorch#77764

[abhimanyuchadha96]

Yeah it just pointed me to other ops that aren't implemented and it looks like some part of the tensor is on the CPU and some aren't.

[glenn-jocher]

@abhimanyuchadha96 well then you might just want to move the predictions to CPU at the beginning of the NMS function.

[abhimanyuchadha96]

@glenn-jocher While it did run successfully after doing that, the results that were saved were blank ie without any bounding boxes. Even with the confidence as low as 0.01

[glenn-jocher]

@abhimanyuchadha96 I tried this here on GPU and it seems to work well. If MPS isn't working I'm not sure what the problem is, but I'll stick a TODO here to investigate.

[tinosai]

Any updates on this? I am getting the same error on the Stable version 1.13.1

[ezam0000]

Any updates? on UserWarning: The operator 'aten::nonzero' is not currently supported on the MPS backend??

[Chase-Xuu]

Any updates on this? I have got the same error on my M2 device, but I can run it through the CPU.

[glenn-jocher]

@Chi-XU-Sean thanks for bringing this to our attention. We understand that you are experiencing an issue with the 'aten::nonzero' operator not being supported on the MPS backend, specifically on your M2 device.

At the moment, we don't have any specific updates on this issue. However, we recommend trying to run the code through the CPU as a workaround, if that is working for you.

Please note that the YOLOv5 repo is maintained by volunteers and the Ultralytics team, and we rely on community contributions and feedback to address issues and improve the codebase. Therefore, any further contributions or insights from the community on this issue would be greatly appreciated.

If you have any further questions or need additional assistance, please don't hesitate to ask. Thank you for your patience and support!