A curated list for awesome self-supervised graph representation learning resources. Inspired by awesome-deep-vision, awesome-adversarial-machine-learning, awesome-deep-learning-papers, awesome-architecture-search, and awesome-self-supervised-learning.
Self-Supervised Learning has become an exciting direction in AI community.
- Jitendra Malik: "Supervision is the opium of the AI researcher"
- Alyosha Efros: "The AI revolution will not be supervised"
- Yann LeCun: "self-supervised learning is the cake, supervised learning is the icing on the cake, reinforcement learning is the cherry on the cake"
- Overview
- Training Strategy
- Contrastive Learning
- Generative Learning
- Predictive Learning
- A Summary of Methodology Details
- A Summary of Implementation Details
- A Summary of Common Graph Datasets
- A Summary of Open-source Codes
We extend the concept of self-supervised learning, which first emerged in the fields of computer vision and natural language processing, to present a timely and comprehensive review of the existing SSL techniques for graph data. Specifically, we divide existing graph SSL methods into three categories: contrastive, generative, and predictive as shown below.
- Contrastive Learning: it contrasts the views generated by different data augmentation methods. The information about the differences and sameness between data-data pairs (inter-data) is used as self-supervision signals.
- Generative Learning: it focuses on the (intra-data) information embedded in the data, generally based on prtext tasks such as reconstruction, which exploit the attributes and structure of the data itself as self-supervision signals.
- Predictive Learning: it generally self-generates labels from graph data through some simple statistical analysis, or expert knowledge, and designs prediction-based pretext tasks based on the self-generated labels to handle the data-label relationship.
Considering the relationship among bottleneck encoders, self-supervised pretext tasks, and downstream tasks, the training strategies can be divided into three categories: Pre-training and Fine-tuning (P&F), Joint Learning (JL), and Unsupervised Representation Learning (URL), with their detailed workflow shown below.
- Pre-train&Fine-tune (P&F): it first pre-trains the encoder with unlabeled nodes by the self-supervised pretext tasks. The pre-trained encoder’s parameters are then used as the initialization of the encoder used in supervised fine-tuning for downstream tasks.
- Joint Learning (JL): an auxiliary pretext task with self-supervision is included to help learn the supervised downstream task. The encoder is trained through both the pretext task and the downstream task simultaneously.
- Unsupervised Representation Learning (URL): it first pre-trains the encoder with unlabeled nodes by the self-supervised pretext tasks. The pre-trained encoder’s parameters are then frozen and used in the supervised downstream task with additional labels.
A general framework for contrastive learning is shown below. The two contrasting components may be local, contextual, or global, corresponding to node-level (marked in red), subgraph-level (marked in green), or graph-level (marked in yellow) information in the graph. The contrastive learning can thus contrast two views (at the same or different scales), which leads to two categories of algorithm: (1) same-scale contrasting, including Local-Local (L-L) contrasting, Context-Context (C-C) contrasting, and Global-Global (G-G) contrasting; and (2) cross-scale contrasting, including Local-Context (L-C) contrasting, Local-Global (L-G) contrasting, and Context-Global (C-G) contrasting.
- GraphCL: Graph Contrastive Learning with Augmentations.
- IGSD: Iterative Graph Self-Distillation.
- H. Zhang, S. Lin, W. Liu, P. Zhou, J. Tang, X. Arxiv 2020. [pdf]
- DACL: Towards Domain-Agnostic Contrastive Learning.
- V. Verma, M.-T. Luong, K. Kawaguchi, H. Pham, andQ. V. Le. Arxiv 2020. [pdf]
- LCC: Label Contrastive Coding Based Graph Neural Network for Graph Classification.
- CCGL: Contrastive Cascade Graph Learning.
- CSSL: Contrastive Self-Supervised Learning for Graph Classification.
- J. Zeng and P. Xie. Arxiv 2020. [pdf]
- GCC: Graph Contrastive Coding for Graph Neural Network Pre-training.
- CDNMF: Contrastive Deep Nonnegative Matrix Factorization for Community Detection.
- GRACE: Deep Graph Contrastive Representation Learning.
- GCA: Graph Contrastive Learning with Adaptive Augmentation.
- GROC: Towards Robust Graph Contrastive Learning.
- N. Jovanovi´c, Z. Meng, L. Faber, and R. Wattenhofer. Arxiv 2021. [pdf]
- SEPT: Socially-Aware Self-Supervised Tri-Training for Recommendation.
- STDGI: Spatio-Temporal Deep Graph Infomax.
- F. L. Opolka, A. Solomon, C. Cangea, P. Veliˇckovi´c, P. Li` o, and R. D. Hjelm. Arxiv 2019. [pdf]
- GMI: Graph Representation Learning via Graphical Mutual Information Maximization.
- KS2L: Self-Supervised Smoothing Graph Neural Networks.
- L. Yu, S. Pei, C. Zhang, L. Ding, J. Zhou, L. Li, and X. Zhang. Arxiv 2020. [pdf]
- CG3: Contrastive and Generative Graph Convolutional Networks for Graph-based Semi-Supervised Learning.
- S. Wan, S. Pan, J. Yang, and C. Gong. Arxiv 2020. [pdf]
- BGRL: Bootstrapped Representation Learning on Graphs.
- SelfGNN: Self-supervised Graph Neural Networks without Explicit Negative Sampling.
- HeCo: Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning.
- PT-DGNN: Pre-training on Dynamic Graph Neural Networks.
- COAD: Coad: Contrastive Pretraining with Adversarial Fine-tuning for Zero-shot Expert Linking.
- Contrast-Reg: Improving Graph Representation Learning by Contrastive Regularization.
- K. Ma, H. Yang, H. Yang, T. Jin, P. Chen, Y. Chen, B. F. Kamhoua, and J. Cheng. Arxiv 2021. [pdf]
- C-SWM: Contrastive Learning of Structured World Models.
- DGI: Deep Graph Infomax.
- HDMI: Hdmi: High-order Deep Multiplex Infomax.
- B. Jing, C. Park, and H. Tong. Arxiv 2021. [pdf]
- DMGI: Unsupervised Attributed Multiplex Network Embedding.
- MVGRL: Contrastive Multi-View Representation Learning on Graphs.
- HDGI: Heterogeneous Deep Graph Infomax.
- CDNMF: Contrastive Deep Nonnegative Matrix Factorization for Community Detection.
- Subg-Con: Sub-graph Contrast for Scalable Self-Supervised Graph Representation Learning.
- Cotext Prediction: Strategies for Pre-training Graph Neural Networks.
- GIC: Leveraging Cluster-level Node Information for Unsupervised Graph Representation Learning.
- GraphLoG: Self-Supervised Graph-level Representation Learning with Local and Global Structure.
- MHCN: Self-Supervised Multi-channel Hypergraph Convolutional Network for Social Recommendation.
- EGI: Transfer Learning of Graph Neural Networks with Ego-graph Information Maximization.
- MICRO-Graph: Motif-Driven Contrastive Learning of Graph Representations.
- InfoGraph: Unsupervised and Semi-Supervised Graph-level Representation Learning via Mutual Information Maximization.
- SUGAR: Subgraph Neural Network with Reinforcement Pooling and Self-Supervised Mutual Information Mechanism.
- BiGI: Bipartite Graph Embedding via Mutual Information Maximization.
- HTC: Graph Representation Learning by Ensemble Aggregating Subgraphs via Mutual Information Maximization.
- C. Wang and Z. Liu. Arxiv 2021. [pdf]
- DITNet: Drug Target Prediction using Graph Representation Learning via Substructures Contrast.
- CDNMF: Contrastive Deep Nonnegative Matrix Factorization for Community Detection.
- GraphMAE: Self-supervised Masked Graph Autoencoders
- Graph Completion: When Does Self-Supervision Help Graph Convolutional Networks?
- Node Attribute Masking: Self-Supervised Learning on Graphs: Deep Insights and New Direction.
- Edge Attribute Masking: Strategies for Pre-training Graph Neural Networks.
- Node Attribute and Embedding Denoising: Graph-based Neural Network Models with Multiple Self-Supervised Auxiliary Tasks.
- F. Manessi and A. Rozza. Arxiv 2020. [pdf]
- Adjacency Matrix Reconstruction: Self-Supervised Training of Graph Convolutional Networks.
- Q. Zhu, B. Du, and P. Yan. Arxiv 2020. [pdf]
- Graph Bert: Only Attention is Needed for Learning Graph Representations.
- Pretrain-Recsys: Pretraining Graph Neural Networks for Cold-start Users and Items Representation.
- SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networks.
- G-BERT: Pre-Training of Graph Augmented Transformers for Medication Recommendation.
- GPT-GNN: Generative Pre-training of Graph Neural Networks.
A comparison of the predictive learning is shown below. The predictive method generally self-generates labels from graph data and then designs prediction-based pretext tasks based on the self-generated labels. Categorized by how the labels areobtained, we summarize predictive learning methods forgraph data into four categories:
- Node Property Prediction: it pre-calculates the node properties, such as node degree and used them as self-supervised labels.
- Context-based Prediction: the local or global contextual information in the graph, such as the shortest path length between nodes can be extracted as labels to help with self-supervised learning.
- Self-Training: it applies algorithms such as unsupervised clustering to obtain pseudo-labels and then updates the pseudo-label set of the previous stage based on the prediction results or losses.
- Domain Knowledge-based Prediction: the domain knowledge, such as expert knowledge or specialized tools, can be used in advance to obtain informative labels.
- Node Property Prediction: Self-Supervised Learning on Graphs: Deep Insights and New Direction.
- S2GRL: Self-Supervised Graph Representation Learning via Global Context Prediction.
- Z. Peng, Y. Dong, M. Luo, X.-M. Wu, and Q. Zheng. Arxiv 2020. [pdf]
- PairwiseDistance: Self-Supervised Learning on Graphs: Deep Insights and New Direction.
- PairwiseAttsim: Self-Supervised Learning on Graphs: Deep Insights and New Direction.
- Distance2Cluster: Self-Supervised Learning on Graphs: Deep Insights and New Direction.
- EdgeMask: Self-Supervised Learning on Graphs: Deep Insights and New Direction.
- TopoTER: Unsupervised Learning of Topology Transformation Equivariant Representations.
- X. Gao, W. Hu, and G.-J. Qi. OpenReview 2021. [pdf]
- Centrality Score Ranking: Pretraining Graph Neural Networks for Generic Structural Feature Extraction.
- Z. Hu, C. Fan, T. Chen, K.-W. Chang, and Y. Sun. Arxiv 2019. [pdf]
- Meta-path prediction: Self-supervised Auxiliary Learning with Meta-paths for Heterogeneous Graphs.
- SLiCE: Self-Supervised Learning of Contextual Embeddings for Link Prediction in Heterogeneous Networks.
- Distance2Labeled: Self-Supervised Learning on Graphs: Deep Insights and New Direction.
- Distance2Labeled: Self-Supervised Learning on Graphs: Deep Insights and New Direction.
- HTM: Hop-count based Self-Supervised Anomaly Detection on Attributed Networks.
- T. Huang, Y. Pei, V. Menkovski, and M. Pechenizkiy. Arxiv 2021. [pdf]
- Multi-stage Self-training: Deeper insights into Graph Convolutional Networks for Semi-Supervised Learning.
- Node Clustering and Partitioning: When Does Self-Supervision Help Graph Convolutional Networks.
- CAGAN: Cluster-Aware Graph Neural Networks for Unsupervised Graph Representation Learning.
- Y. Zhu, Y. Xu, F. Yu, S. Wu, and L. Wang. Arxiv 2020. [pdf]
- M3S: Multi-stage Self-Supervised Learning for Graph Convolutional Networks on Graphs with Few Labeled Nodes.
- Cluster Preserving: Pretraining Graph Neural Networks for Generic Structural Feature Extraction.
- Z. Hu, C. Fan, T. Chen, K.-W. Chang, and Y. Sun. Arxiv 2019. [pdf]
- SEF: Self-Supervised Edge Features for Improved Graph Neural Network Training.
- Contextual Molecular Property Prediction: Self-Supervised Graph Transformer on Large-Scale Molecular Data.
- Graph-level Motif Prediction: Self-Supervised Graph Transformer on Large-scale Molecular Data.
- DrRepair: Graph-based, Self-Supervised Program Repair from Diagnostic Feedback.
A summary of all the surveyed works is presented below.
About Graph Property, Pretext Task, Data Augmentation, Objective Function, Training Strategy, and Year of publication.
| Methods | Graph Property | Pretext-Task | Data Augmentation | Objective Function | Training Strategy | Year |
|---|---|---|---|---|---|---|
| CDNMF | Attributed | Contrastive/L-C + Generative/AE | None | InfoNCE + AE | URL | 2024 |
| Graph Completion | Attributed | Generative/AE | Attribute Masking | MAE | P&F/JL | 2020 |
| Node Attribute Masking | Attributed | Generative/AE | Attribute Masking | MAE | P&F/JL | 2020 |
| Edge Attribute Masking | Attributed | Generative/AE | Attribute Masking | MAE | P&F | 2019 |
| Node Attribute and Embedding Denoising |
Attributed | Generative/AE | Attribute Masking | MAE | JL | 2020 |
| Adjacency Matrix Reconstruction |
Attributed | Generative/AE | Attribute Masking Edge Perturbation |
MAE | JL | 2020 |
| Graph Bert | Attributed | Generative/AE | Attribute Masking Edge Perturbation |
MAE | P&F | 2020 |
| Pretrain-Recsys | Attributed | Generative/AE | Edge Perturbation | MAE | P&F | 2021 |
| GPT-GNN | Heterogeneous | Generative/AR | Attribute Masking Edge Perturbation |
MAE/InfoNCE | P&F | 2020 |
| GraphCL | Attributed | Contrastive/G-G | Attribute Masking Edge Perturbation Random Walk Sampling |
InfoNCE | URL | 2020 |
| IGSD | Attributed | Contrastive/G-G | Edge Perturbation Edge Doffisopm |
InfoNCE | JL/URL | 2020 |
| DACL | Attributed | Contrastive/G-G | Mixup | InfoNCE | URL | 2020 |
| LCC | Attributed | Contrastive/G-G | None | InfoNCE | JL | 2021 |
| CCGL | Attributed | Contrastive/G-G | Information Re-Diffusion | InfoNCE | P&F | 2021 |
| CSSL | Attributed | Contrastive/G-G | NodeInsertion Edge Perturbation Uniform Sampling |
InfoNCE | P&F/JL/URL | 2020 |
| GCC | Unattributed | Contrastive/C-C | Random Walk Sampling | InfoNCE | P&F/URL | 2020 |
| GRACE | Attributed | Contrastive/L-L | Attribute Masking Edge Perturbation |
InfoNCE | URL | 2020 |
| GCA | Attributed | Contrastive/L-L | Attention-based | InfoNCE | URL | 2020 |
| GROC | Attributed | Contrastive/L-L | Gradient-based | InfoNCE | URL | 2021 |
| SEPT | Attributed | Contrastive/L-L | Edge Perturbation | InfoNCE | JL | 2021 |
| STDGI | Spatial-Temporal | Contrastive/L-L | Attribute Shuffling | JS Estimator | URL | 2019 |
| GMI | Attributed | Contrastive/L-L | None | SP Estimator | URL | 2020 |
| KS2L | Attributed | Contrastive/L-L | None | InfoNCE | URL | 2020 |
| CG3 | Attributed | Contrastive/L-L | None | InfoNCE | JL | 2020 |
| BGRL | Attributed | Contrastive/L-L | Attribute Masking Edge Perturbation |
Inner Product | URL | 2021 |
| SelfGNN | Attributed | Contrastive/L-L | Attribute Masking Edge Diffusion |
MSE | URL | 2021 |
| HeCo | Heterogeneous | Contrastive/L-L | None | InfoNCE | URL | 2021 |
| PT-DGNN | Dynamic | Contrastive/L-L | Attribute Masking Edge Perturbation |
InforNCE | P&F | 2021 |
| COAD | Attributed | Contrastive/L-L | None | Triplet Margin Loss | P&F | 2020 |
| Contrst-Reg | Attributed | Contrastive/L-L | Attribute Shuffling | InfoNCE | JL | 2021 |
| DGI | Attributed | Contrastive/L-G | Arbitrary | JS Estimator | URL | 2019 |
| HDMI | Attributed | Contrastive/L-G | Attribute Shuffling | JS Estimator | URL | 2021 |
| DMGI | Heterogeneous | Contrastive/L-G | Attribute Shuffling | JS Estimator/MAE | URL | 2020 |
| MVGRL | Attributed | Contrastive/L-G | Attribute Masking Edge Perturbation Edge Diffusion Random Walk Sampling |
DV Estimator JS Estimator NT-Xent InfoNCE |
URL | 2020 |
| HDGI | Heterogeneous | Contrastive/L-G | Attribute Shuffling | JS Estimator | URL | 2019 |
| Subg-Con | Attributed | Contrastive/L-C | Importance Sampling | Triplet Margin Loss | URL | 2020 |
| Cotext Prediction | Attributed | Contrastive/L-C | Ego-nets Sampling | Cross Entropy | P&F | 2019 |
| GIC | Attributed | Contrastive/L-C | Arbitrary | JS Estimator | URL | 2020 |
| GraphLoG | Attributed | Contrastive/L-C | Attribute Masking | InfoNCE | URL | 2021 |
| MHCN | Heterogeneous | Contrastive/L-C | Attribute Shuffling | InfoNCE | JL | 2021 |
| EGI | Attributed | Contrastive/L-C | Ego-nets Sampling | SP Estimator | P&F | 2020 |
| MICRO-Graph | Attributed | Contrastive/C-G | Knowledge Sampling | InfoNCE | URL | 2020 |
| InfoGraph | Attributed | Contrastive/C-G | None | SP Estimator | URL | 2019 |
| SUGAR | Attributed | Contrastive/C-G | BFS Sampling | JS Estimator | JL | 2021 |
| BiGI | Heterogeneous | Contrastive/C-G | Edge Perturbation Ego-nets Sampling |
JS Estimator | JL | 2021 |
| HTC | Attributed | Contrastive/C-G | Attribute Shuffling | SP Estimator DV Estimator |
URL | 2021 |
| Node Property Prediction | Attributed | Predictive/Node Property | None | MAE | P&F/JL | 2020 |
| S2GRL | Attributed | Predictive/Context-based | None | Cross Entropy | URL | 2020 |
| PairwiseDistance | Attributed | Predictive/Context-based | None | Cross Entropy | P&F/JL | 2020 |
| PairwiseAttrSim | Attributed | Predictive/Context-based | None | MAE | P&F/JL | 2020 |
| Distance2Cluster | Attributed | Predictive/Context-based | None | MAE | P&F/JL | 2020 |
| EdgeMask | Attributed | Predictive/Context-based | None | Cross Entropy | P&F/JL | 2020 |
| TopoTER | Attributed | Predictive/Context-based | Edge Perturbation | Cross Entropy | URL | 2021 |
| Centrality Score Ranking | Attributed | Predictive/Context-based | None | Cross Entropy | P&F | 2019 |
| Meta-path prediction | Heterogeneous | Predictive/Context-based | None | Cross Entropy | JL | 2020 |
| SLiCE | Heterogeneous | Predictive/Context-based | None | Cross Entropy | P&F | 2020 |
| Distance2Labeled | Attributed | Predictive/Context-based | None | MAE | P&F/JL | 2020 |
| ContextLabel | Attributed | Predictive/Context-based | None | MAE | P&F/JL | 2020 |
| HCM | Attributed | Predictive/Context-based | Edge Perturbation | Bayesian Inference | URL | 2021 |
| Contextual Molecular Property Prediction |
Attributed | Predictive/Domain-based | None | Cross Entropy | P&F | 2020 |
| Graph-level Motif Prediction | Attributed | Predictive/Domain-based | None | Cross Entropy | P&F | 2020 |
| Multi-stage Self-training | Attributed | Predictive/Self-training | None | None | JL | 2018 |
| Node Clustering | Attributed | Predictive/Self-training | None | Clustering | P&F/JL | 2020 |
| Graph Partitioning | Attributed | Predictive/Self-training | None | Graph Partitioning | P&F/JL | 2020 |
| CAGAN | Attributed | Predictive/Self-training | None | Clustering | URL | 2020 |
| M3S | Attributed | Predictive/Self-training | None | Clustering | JL | 2020 |
| Cluster Preserving | Attributed | Predictive/Self-training | None | Cross Entropy | P&F | 2019 |
About Task Level, Evaluation Metric, and Evaluation Datasets.
| Methods | Task Level | Evaluation Metric | Dataset |
|---|---|---|---|
| CDNMF | Node | Node Clustering (Acc, NMI) | Cora, Citeseer, Pubmed |
| Graph Completion | Node | Node Classification (Acc) | Cora, Citeseer, Pubmed |
| Node Attribute Masking | Node | Node Classification (Acc) | Cora, Citeseer, Pubmed, Reddit |
| Edge Attribute Masking | Graph | Graph Classification (ROC-AUC) | MUTAG, PTC, PPI, BBBP, Tox21, ToxCast, ClinTox, MUV, HIV, SIDER, BACE |
| Node Attribute and Embedding Denoising |
Node | Node Classification (Acc) | Cora, Citeseer, Pubmed |
| Adjacency Matrix Reconstruction |
Node | Node Classification (Acc) | Cora, Citeseer, Pubmed |
| Graph Bert | Node | Node Classification (Acc) Node Clustering (NMI) |
Cora, Citeseer, Pubmed |
| Pretrain-Recsys | Node/Link | - | ML-1M, MOOCs and Last-FM |
| GPT-GNN | Node/Link | Node Classification (F1-score) Link Prediction (ROC-AUC) |
OAG, Amazon, Reddit |
| GraphCL | Graph | Graph Classification (Acc, ROC-AUC) | NCI1, PROTEINS, D&D, COLLAB, RDT-B, RDT-M5K, GITHUB, MNIST, CIFAR10, MUTAG, IMDB-B, BBBP, Tox21, ToxCast, SIDER, ClinTox, MUV, HIV, BACE, PPI |
| IGSD | Graph | Graph Classification (Acc) | MUTAG, PTC_MR, NCI1, IMDB-B, QM9, COLLAB, IMDB-M |
| DACL | Graph | Graph Classification (Acc) | MUTAG, PTC_MR, IMDB-B, IMDB-M, RDT-B, RDT-M5K |
| LCC | Graph | Graph Classification (Acc) | IMDB-B, IMDB-M, COLLAB, MUTAG, PROTEINS, PTC, NCI1, D&D |
| CCGL | Graph | Cascade Graph Prediction (MSLE) | Weibo, Twitter, ACM, APS, DBLP |
| CSSL | Graph | Graph Classification (Acc) | PROTEINS, D&D, NCI1, NCI109, Mutagenicity |
| GCC | Node/Graph | Node Classification (Acc) Graph Classification (Acc) |
US-Airport, H-index, COLLAB, IMDB-B, IMDB-M, RDT-B, RDT-M5K |
| GRACE | Node | Node Classification (Acc, Micro-F1) | Cora, Citeseer, Pubmed, DBLP, Reddit, PPI |
| GCA | Node | Node Classification (Acc) | Wiki-CS, Amazon-Computers, Amazon-Photo, Coauthor-CS, Coauthor-Physics |
| GROC | Node | Node Classification (Acc) | Cora, Citeseer, Pubmed, Amazon-Photo, Wiki-CS |
| SEPT | Node/Link | - | Last-FM, Douban, Yelp |
| STDGI | Node | Node Regression (MAE, RMSE, MAPE) | METR-LA |
| GMI | Node/Link | Node Classification (Acc, Micro-F1) Link Prediction (ROC-AUC) |
Cora, Citeseer, PubMed, Reddit, PPI, BlogCatalog, Flickr |
| KS2L | Node/Link | Node Classification (Acc) Link Prediction (ROC-AUC) |
Cora, Citeseer, Pubmed, Amazon-Computers, Amazon-Photo, Coauthor-CS |
| CG3 | Node | Node Classification (Acc) | Cora, Citeseer, Pubmed, Amazon-Computers, Amazon-Photo, Coauthor-CS |
| BGRL | Node | Node Classification (Acc, Micro-F1) | Wiki-CS, Amazon-Computers, Amazon-Photo, PPI, Coauthor-CS, Coauthor-Physics, ogbn-arxiv |
| SelfGNN | Node | Node Classification (Acc) | Cora, Citeseer, Pubmed, Amazon-Computers, Amazon-Photo, Coauthor-CS, Coauthor-Physics |
| HeCo | Node | Node Classification (ROC-AUC, Micro-F1, Macro-F1) Node Clustering (NMI, ARI) |
ACM, DBLP, Freebase, AMiner |
| PT-DGNN | Link | Link Prediction (ROC-AUC) | HepPh, Math Overflow, Super User |
| COAD | Node/Link | Node Clustering (Precision, Recall, F1-score) Link Prediction (HitRatio@K, MRR) |
AMiner, News, LinkedIn |
| Contrast-Reg | Node/Link | Node Classification (Acc) Node Clustering (NMI, Acc, Macro-F1) Link Prediction (ROC-AUC) |
Cora, Citeseer, Pubmed, Reddit, ogbn-arxiv, Wikipedia, ogbn-products, Amazo-Computers, Amazo-Photo |
| DGI | Node | Node Classification (Acc, Micro-F1) | Cora, Citeseer, Pubmed, Reddit, PPI |
| HDMI | Node | Node Classification (Micro-F1, Macro-F1) Node Clustering (NMI) |
ACM, IMDB, DBLP, Amazon |
| DMGI | Node | Node Clustering (NMI) Node Classification (Acc) |
ACM, IMDB, DBLP, Amazon |
| MVGRL | Node/Graph | Node Classification (Acc) Node Clustering (NMI, ARI) Graph Classification (Acc) |
Cora, Citeseer, Pubmed, MUTAG, PTC_MR, IMDB-B, IMDB-M, RDT-B |
| HDGI | Node | Node Classification (Micro-F1, Macro-F1) Node Clustering (NMI, ARI) |
ACM, DBLP, IMDB |
| Subg-Con | Node | Node Classification (Acc, Micro-F1) | Cora, Citeseer, Pubmed, PPI, Flickr, Reddit |
| Cotext Prediction | Graph | Graph Classification (ROC-AUC) | MUTAG, PTC, PPI, BBBP, Tox21, ToxCast, ClinTox, MUV, HIV, SIDER, BACE |
| GIC | Node/Link | Node Classification (Acc) Node Clustering (Acc, NMI, ARI) Link Prediction (ROC-AUC, ROC-AP) |
Cora, Citeseer, Pubmed, Amazon-Computers, Amazon-Photo, Coauthor-CS, Coauthor-Physics |
| GraphLoG | Graph | Graph Classification (ROC-AUC) | BBBP, Tox21, ToxCast, ClinTox, MUV, HIV, SIDER, BACE |
| MHCN | Node/Link | - | Last-FM, Douban, Yelp |
| EGI | Node/Link | Node Classification (Acc) Link Prediction (ROC-AUC, MRR) |
YAGO, Airport |
| MICRO-Graph | Graph | Graph Classification (ROC-AUC) | BBBP, Tox21, ToxCast, ClinTox, HIV, SIDER, BACE |
| InfoGraph | Graph | Graph Classification (Acc) | MUTAG, PTC_MR, RDT-B, RDT-M5K, IMDB-B, QM9, IMDB-M |
| SUGAR | Graph | Graph Classification (Acc) | MUTAG, PTC, PROTEINS, D&D, NCI1, NCI109 |
| BiGI | Link | Link Prediction (AUC-ROC, AUC-PR) | DBLP, ML-100K, ML-1M, Wikipedia |
| HTC | Graph | Graph Classification (Acc) | MUTAG, PTC_MR, IMDB-B, IMDB-M, RDT-B, QM9, RDT-M5K |
| Node Property Prediction | Node | Node Classification (Acc) | Cora, Citeseer, Pubmed, Reddit |
| S2GRL | Node/Link | Node Classification (Acc, Micro-F1) Node Clustering (NMI) Link Prediction (ROC-AUC) |
Cora, Citeseer, Pubmed, PPI, Flickr, BlogCatalog, Reddit |
| PairwiseDistance | Node | Node Classification (Acc) | Cora, Citeseer, Pubmed, Reddit |
| PairwiseAttrSim | Node | Node Classification (Acc) | Cora, Citeseer, Pubmed, Reddit |
| Distance2Cluster | Node | Node Classification (Acc) | Cora, Citeseer, Pubmed, Reddit |
| EdgeMask | Node | Node Classification (Acc) | Cora, Citeseer, Pubmed, Reddit |
| TopoTER | Node/Graph | Node Classification (Acc) Graph Classification (Acc) |
Cora, Citeseer, Pubmed, MUTAG, PTC-MR, RDT-B, RDT-M5K, IMDB-B, IMDB-M |
| Centrality Score Ranking | Node/Link/Graph | Node Classification (Micro-F1) Link Prediction (Micro-F1) Graph Classification (Micro-F1) |
Cora, Pubmed, ML-100K, ML-1M, IMDB-M, IMDB-B |
| Meta-path prediction | Node/Link | Node Classification (F1-score) Link Prediction (ROC-AUC) |
ACM, IMDB, Last-FM, Book-Crossing |
| SLiCE | Link | Link Prediction (ROC-AUC, Micro-F1) | Amazon, DBLP, Freebase, Twitter, Healthcare |
| Distance2Labeled | Node | Node Classification (Acc) | Cora, Citeseer, Pubmed, Reddit |
| ContextLabel | Node | Node Classification (Acc) | Cora, Citeseer, Pubmed, Reddit |
| HCM | Node | Node Classification (ROC-AUC) | ACM, Amazon, Enron, BlogCatalog, Flickr |
| Contextual Molecular Property Prediction |
Graph | Graph Classification (Acc) Graph Regression (MAE) |
BBBP, SIDER, ClinTox, BACE, Tox21, ToxCast, ESOL, FreeSolv, Lipo, QM7, QM8 |
| Graph-level Motif Prediction | Graph | Graph Classification (Acc) Graph Regression (MAE) |
BBBP, SIDER, ClinTox, BACE, Tox21, ToxCast, ESOL, FreeSolv, Lipo, QM7, QM8 |
| Multi-stage Self-training | Node | Node Classification (Acc) | Cora, Citeseer, Pubmed |
| Node Clustering | Node | Node Classification (Acc) | Cora, Citeseer, Pubmed |
| Graph Partitioning | Node | Node Classification (Acc) | Cora, Citeseer, Pubmed |
| CAGAN | Node | Node Classfication (Micro-F1, Macro-F1) Node Clustering (Micro-F1, Macro-F1, NMI) |
Cora, Citeseer, Pubmed |
| M3S | Node | Node Classification (Acc) | Cora, Citeseer, Pubmed |
| Cluster Preserving | Node/Link/Graph | Node Classification (Micro-F1) Link Prediction (Micro-F1) Graph Classification (Micro-F1) |
Cora, Pubmed, ML-100K, ML-1M, IMDB-M, IMDB-B |
About category, graph number, node number per graph, edge number per graph, dimensionality of node attributes, class number, and citation papers.
| Dataset | Category | #Graph | #Node (Avg.) | #Edge (Avg.) | #Feature | #Class |
|---|---|---|---|---|---|---|
| Cora | Citation Network | 1 | 2708 | 5429 | 1433 | 7 |
| Citeseer | Citation Network | 1 | 3327 | 4732 | 3703 | 6 |
| Pubmed | Citation Network | 1 | 19717 | 44338 | 500 | 3 |
| Wiki-CS | Citation Network | 1 | 11701 | 216123 | 300 | 10 |
| Coauthor-CS | Citation Network | 1 | 18333 | 81894 | 6805 | 15 |
| Coauthor-Physics | Citation Network | 1 | 34493 | 247962 | 8415 | 5 |
| DBLP (v12) | Citation Network | 1 | 4894081 | 45564149 | - | - |
| ogbn-arxiv | Citation Network | 1 | 169343 | 1166243 | 128 | 40 |
| Social Network | 1 | 232965 | 11606919 | 602 | 41 | |
| BlogCatalog | Social Network | 1 | 5196 | 171743 | 8189 | 6 |
| Flickr | Social Network | 1 | 7575 | 239738 | 12047 | 9 |
| COLLAB | Social Networks | 5000 | 74.49 | 2457.78 | - | 2 |
| RDT-B | Social Networks | 2000 | 429.63 | 497.75 | - | 2 |
| RDT-M5K | Social Networks | 4999 | 508.52 | 594.87 | - | 5 |
| IMDB-B | Social Networks | 1000 | 19.77 | 96.53 | - | 2 |
| IMDB-M | Social Networks | 1500 | 13.00 | 65.94 | - | 3 |
| ML-100K | Social Networks | 1 | 2625 | 100000 | - | 5 |
| ML-1M | Social Networks | 1 | 9940 | 1000209 | - | 5 |
| PPI | Protein Networks | 24 | 56944 | 818716 | 50 | 121 |
| D&D | Protein Networks | 1178 | 284.32 | 715.65 | 82 | 2 |
| PROTEINS | Protein Networks | 1113 | 39.06 | 72.81 | 4 | 2 |
| NCI1 | Molecule Graphs | 4110 | 29.87 | 32.30 | 37 | 2 |
| MUTAG | Molecule Graphs | 188 | 17.93 | 19.79 | 7 | 2 |
| QM9 (QM7, QM8) | Molecule Graphs | 133885 | - | - | - | - |
| BBBP | Molecule Graphs | 2039 | 24.05 | 25.94 | - | 2 |
| Tox21 | Molecule Graphs | 7831 | 18.51 | 25.94 | - | 12 |
| ToxCast | Molecule Graphs | 8575 | 18.78 | 19.26 | - | 167 |
| ClinTox | Molecule Graphs | 1478 | 26.13 | 27.86 | - | 2 |
| MUV | Molecule Graphs | 93087 | 24.23 | 26.28 | - | 17 |
| HIV | Molecule Graphs | 41127 | 25.53 | 27.48 | - | 2 |
| SIDER | Molecule Graphs | 1427 | 33.64 | 35.36 | - | 27 |
| BACE | Molecule Graphs | 1513 | 34.12 | 36.89 | - | 2 |
| PTC | Molecule Graphs | 344 | 14.29 | 14.69 | 19 | 2 |
| NCI109 | Molecule Graphs | 4127 | 29.68 | 32.13 | - | 2 |
| Mutagenicity | Molecule Graphs | 4337 | 30.32 | 30.77 | - | 2 |
| MNIST | Others (Image) | - | 70000 | - | 784 | 10 |
| CIFAR10 | Others (Image) | - | 60000 | - | 1024 | 10 |
| METR-LA | Others (Traffic) | 1 | 207 | 1515 | 2 | - |
| Amazon-Computers | Others (Purchase) | 1 | 13752 | 245861 | 767 | 10 |
| Amazon-Photo | Others (Purchase) | 1 | 7650 | 119081 | 745 | 8 |
| ogbn-products | Others (Purchase) | 1 | 2449029 | 61859140 | 100 | 47 |
If you would like to help contribute this list, please feel free to contact me or add pull request with the following Markdown format:
- Paper Name.
- Author List. *Conference Year*. [[pdf]](link) [[code]](link)This is a Github Summary of our Survey. If you find this file useful in your research, please consider citing:
@article{wu2021self,
title={Self-supervised Learning on Graphs: Contrastive, Generative, or Predictive},
author={Wu, Lirong and Lin, Haitao and Tan, Cheng and Gao, Zhangyang and Li, Stan Z},
journal={IEEE Transactions on Knowledge and Data Engineering},
year={2021},
publisher={IEEE}
}
If you have any issue about this work, please feel free to contact me by email:
- Lirong Wu: [email protected]
- Haitao Lin: [email protected]