Neighbourhood definition of the GATConv layer

[Niklas2501]

I currently trying to figure out, how the node neighbourhood is definied for the GATConv layer,
sadly the documentation does not does not specify it in detail and i wasn't able to derive it from the code.

Primarily, I am interested in the case of a directed graphs, i.e. a non-symmetric A.

Based on the following minimum working example, I would assume that the calculation of a node incorporates only those nodes to which it has an outgoing edge.

Is this interpretation correct or am i missing something?

Thanks!

import numpy as np
import spektral
import tensorflow as tf

A = np.array([

    # No connection from or to A
    [
        # A, B, C, D
        [1, 0, 0, 0],  # A
        [0, 1, 0, 0],  # B
        [0, 0, 1, 0],  # C
        [0, 0, 0, 1]  # D
    ],

    # Connection A -> C
    [
        # A, B, C, D
        [1, 0, 1, 0],  # A
        [0, 1, 0, 0],  # B
        [0, 0, 1, 0],  # C
        [0, 0, 0, 1]  # D
    ],

    # Connection C -> A
    [
        # A, B, C, D
        [1, 0, 0, 0],  # A
        [0, 1, 0, 0],  # B
        [1, 0, 1, 0],  # C
        [0, 0, 0, 1]  # D
    ],

    # Connection A -> C and C -> A
    [
        # A, B, C, D
        [1, 0, 1, 0],  # A
        [0, 1, 0, 0],  # B
        [1, 0, 1, 0],  # C
        [0, 0, 0, 1]  # D
    ],
])

X = np.array([
    [
        [2, 1, 0],
        [0.5, -0.5, 1],
        [1, 2, 1.5],
        [1, 1, 0]
    ],
    [
        [2, 1, 0],
        [0.5, -0.5, 1],
        [1, 2, 1.5],
        [1, 1, 0]
    ],
    [
        [2, 1, 0],
        [0.5, -0.5, 1],
        [1, 2, 1.5],
        [1, 1, 0]
    ],
    [
        [2, 1, 0],
        [0.5, -0.5, 1],
        [1, 2, 1.5],
        [1, 1, 0]
    ],
])

W = np.array([
    [[1, 1]],
    [[1, 1]],
    [[1, 1]],
])

tf.random.set_seed(1)
n_nodes = X.shape[1]
n_features = X.shape[2]
n_channels = 2

x_input = tf.keras.Input(shape=(n_nodes, n_features))
a_input = tf.keras.Input(shape=(n_nodes, n_nodes), sparse=True)
layer = spektral.layers.GATConv(channels=n_channels, activation=None, use_bias=False,
                                kernel_initializer=tf.keras.initializers.Constant(value=1), attn_heads=1,
                                concat_heads=True, )

output = layer([x_input, a_input])
graph_model = tf.keras.Model([x_input, a_input], output, name=f'gat')

graph_model.compile(
    optimizer='adam',
    loss=tf.keras.losses.CategoricalCrossentropy()
)

out = graph_model([X, A]).numpy()

print('No connection from or to A')
print(out[0])
print('Connection A -> C')
print(out[1])
print('Connection C -> A')
print(out[2])
print('Connection A -> C and C -> A')
print(out[3])

Output:

No connection from or to A
[[3.  3. ]
 [1.  1. ]
 [4.5 4.5]
 [2.  2. ]]

Connection A -> C
[[3.6926923 3.6926923]
 [1.        1.       ]
 [4.5       4.5      ]
 [2.        2.       ]]

Connection C -> A
[[3.        3.       ]
 [1.        1.       ]
 [3.6862757 3.6862757]
 [2.        2.       ]]

Connection A -> C and C -> A
[[3.6926923 3.6926923]
 [1.        1.       ]
 [3.6862757 3.6862757]
 [2.        2.       ]]

The values of Node A (index 0) only change if there is an edge (A,C) (with the index of node C beeing 2), but are not influcenced by an edge (C, A)

[danielegrattarola]

Sorry, completely forgot about this issue.

Yes, GAT is particular because It has different behaviour for dense and sparse adjacency matrices.
In your case, A is dense and the adjacency matrix does not get transposed (I am working to implement directed edge support in all methods, I'll fix it eventually). If you have directed edges, I suggest you use a sparse A so that the information flow is right (neighbours to self).

Cheers