Link to paper
The full paper is available here.
You can also find the paper on PapersWithCode here.
Abstract
- Graphs are a main form of data from nature.
- Graphs are used to represent patterns in natural and artificial systems.
- Graphs are used in traffic forecasting, drug discovery, social network analysis and recommender systems.
- Graphs are related to image, text and speech processing.
Paper Content
The fundamentals: permutation equivariance and invariance
- Studying data that lives on graphs is a good idea
- Graph-structured inputs have a set of edges and a set of nodes
- Each node has a feature vector
- Node feature matrix is prepared by stacking the features
- Adjacency matrix is used to represent edges
- Permuting nodes and edges should not change outputs
- GNNs can be classified into three spatial flavours
- Node classification, graph classification and link prediction are three principal tasks
- GNNs may be limited in terms of problems they can solve
- GNNs in Equation 10 are Turing universal
Gnns without a graph: deep sets and transformers
- Assumption that input graph is given is often not true
- Optimal computation graph may not be given
- GNNs can modulate input graph structure
- Deep Sets model assumes no edges
- Lazy option is to assume fully connected graph
- Equation 9 reduces to Transformer
- NLP perspective: words interact, optimal graph task-dependent
- Transformer rederived
- Transformer computations align with hardware
- Storage complexity better than message passing
- Third option: infer graph structure for GNN
- Latent graph inference is challenging
- Various paradigms proposed to overcome challenge
Gnns beyond permutation equivariance: geometric graphs
- Graphs can be endowed with spatial geometry
- Features and coordinates of nodes can be transformed by 3D rotations, translations and reflections
- Model proposed by Satorras et al. (2021) only works for scalar features
- Vector features need to be updated explicitly
- Tensor Field Networks (Thomas et al., 2018) and SE(3)-Transformers (Fuchs et al., 2020) provide generic equations for roto-translation equivariant models
- Geometric GNNs have been used for protein folding, protein design and protein binding prediction