Link to paper

The full paper is available here.

You can also find the paper on PapersWithCode here.

Abstract

  • Graphs are a main form of data from nature.
  • Graphs are used to represent patterns in natural and artificial systems.
  • Graphs are used in traffic forecasting, drug discovery, social network analysis and recommender systems.
  • Graphs are related to image, text and speech processing.

Paper Content

The fundamentals: permutation equivariance and invariance

  • Studying data that lives on graphs is a good idea
  • Graph-structured inputs have a set of edges and a set of nodes
  • Each node has a feature vector
  • Node feature matrix is prepared by stacking the features
  • Adjacency matrix is used to represent edges
  • Permuting nodes and edges should not change outputs
  • GNNs can be classified into three spatial flavours
  • Node classification, graph classification and link prediction are three principal tasks
  • GNNs may be limited in terms of problems they can solve
  • GNNs in Equation 10 are Turing universal

Gnns without a graph: deep sets and transformers

  • Assumption that input graph is given is often not true
  • Optimal computation graph may not be given
  • GNNs can modulate input graph structure
  • Deep Sets model assumes no edges
  • Lazy option is to assume fully connected graph
  • Equation 9 reduces to Transformer
  • NLP perspective: words interact, optimal graph task-dependent
  • Transformer rederived
  • Transformer computations align with hardware
  • Storage complexity better than message passing
  • Third option: infer graph structure for GNN
  • Latent graph inference is challenging
  • Various paradigms proposed to overcome challenge

Gnns beyond permutation equivariance: geometric graphs

  • Graphs can be endowed with spatial geometry
  • Features and coordinates of nodes can be transformed by 3D rotations, translations and reflections
  • Model proposed by Satorras et al. (2021) only works for scalar features
  • Vector features need to be updated explicitly
  • Tensor Field Networks (Thomas et al., 2018) and SE(3)-Transformers (Fuchs et al., 2020) provide generic equations for roto-translation equivariant models
  • Geometric GNNs have been used for protein folding, protein design and protein binding prediction