Request to add PairNorm: Tackling Oversmoothing in GNNs

[WMF1997]

🚀 Feature

hello @rusty1s
After the discussion in #705 , and I mentioned something about "Normalization on Graph" here

I find a paper here. PAIRNORM: TACKLING OVERSMOOTHING IN GNNS

Motivation

Normalization on Graphs is a good idea, especially when it is used in node classification, as mentioned in the paper above. What's more, comparing with CNNs, we may neglect the Normalization, or we use the normalization in regular grids and find it okay in graph data.
in conclusion, normalization "specially designed" for graphs is seldom mentioned before.

Additional context

no.

yours sincerely,
@WMF1997

[rusty1s]

Hi, thank you for this feature request. This is an interesting method, and we should definitively support it. Eq. (10) and (11) can be easily implemented within PyG (untested):

x_c = x - scatter_mean(x, batch, dim=0, dim_size=num_graphs)[batch]
out = s * x_c / scatter_mean((x_c * x_c).sum(dim=-1, keepdim=True),
                             batch, dim=0, dim_size=num_graphs).sqrt()[batch]

Feel free to submit a PR if you like. Otherwise, I will try to add it to the repository as soon as I am able to. We should add nn.norm to the repository (and also add implementations for BatchNorm and InstanceNorm to it).

[rusty1s]

PairNorm is now integrated into PyG.