[Bug]: torch frontend inconsistent gradient behaviour of torch.nn.relu6

[MahmoudAshraf97]

Bug Explanation

the implementation of torch.nn.relu6 in Pytorch Frontend uses ivy.minimum and ivy.maximum, this implementation is identical to ivy.clip which uses torch.clamp when using torch backend.

The problem arises when calculating gradients because torch.clamp has a gradient of 1 when the input is 0 or 6 while torch.nn.relu6 has a gradient of 0 when the input is 0 or 6 (at the non-differentiable point)

Steps to Reproduce Bug

import ivy
import torch
import ivy.functional.frontends.torch as ivy_torch

x = torch.linspace(-3,8,12)
x.requires_grad = True
y = ivy_torch.nn.functional.relu6(x)  #ivy.minimum(ivy.maximum(x, 0), 6)
(y - 1.0).pow(2).sum().backward()
x_grad = x.grad


x_torch = torch.linspace(-3,8,12)
x_torch.requires_grad = True
y_torch = torch.nn.functional.relu6(x_torch)
(y_torch - 1.0).pow(2).sum().backward()
x_torch_grad = x_torch.grad

print(ivy.allclose(y, y_torch)) #identical results
>>> ivy.array(True)

print(ivy.allclose(x_grad, x_torch_grad)) #inconsistent gradients
>>> ivy.array(False)

Environment

Windows 11 using Docker and VS Code

Ivy Version

v1.1.9

Backend

• NumPy
• TensorFlow
• PyTorch
• JAX

Device

No response