本文轉(zhuǎn)載自【AI機器學(xué)習(xí)與知識圖譜】公眾號

在計算機視覺中，模型CNN隨著其層次加深可以學(xué)習(xí)到更深層次的特征信息，疊加64層或128層是十分正常的現(xiàn)象，且能較淺層取得更優(yōu)的效果。

圖卷積神經(jīng)網(wǎng)絡(luò)GCNs是一種針對圖結(jié)構(gòu)數(shù)據(jù)的深度學(xué)習(xí)方法，但是目前大多數(shù)的GCN模型都是淺層的，如GCN，GAT模型都是在2層時取得最優(yōu)效果，隨著加深模型效果就會大幅度下降；GCN隨著模型層次加深會出現(xiàn)Over-Smoothing問題，Over-Smoothing既相鄰的節(jié)點隨著網(wǎng)絡(luò)變深就會越來越相似，最后學(xué)習(xí)到的nodeembedding便無法區(qū)分，模型效果下降。

為什么要將GNN做深，DeeperGNN適用于解決什么問題：少標(biāo)簽半監(jiān)督節(jié)點分類；少特征半監(jiān)督節(jié)點分類。下面再給出幾個概念解釋。

1. Over-fitting：在CNN卷積神經(jīng)網(wǎng)絡(luò)中，若CNN網(wǎng)絡(luò)結(jié)構(gòu)過于復(fù)雜過于Deep，且數(shù)據(jù)量有限的情況下，便會出現(xiàn)Over-fitting問題，Over-fitting就是指模型對于訓(xùn)練數(shù)據(jù)過度學(xué)習(xí)，學(xué)習(xí)到訓(xùn)練數(shù)據(jù)本身而不是訓(xùn)練數(shù)據(jù)的規(guī)律，導(dǎo)致無法在測試集上準(zhǔn)確預(yù)測的情況。

2. Over-Smoothing：在GNN圖神經(jīng)網(wǎng)絡(luò)中，由于圖本身結(jié)構(gòu)上節(jié)點與節(jié)點之間相互連接的特性，并且圖神經(jīng)網(wǎng)絡(luò)一般是通過鄰域匯聚或隨機游走的方式進行表征學(xué)習(xí)，因此當(dāng)圖網(wǎng)絡(luò)一旦變深，便會出現(xiàn)Over-Smoothing問題，Over-Smoothing指的是隨著圖神經(jīng)網(wǎng)絡(luò)加深，學(xué)習(xí)到的節(jié)點表征越來越相似，以至于無法區(qū)分，模型效果也將大幅下降。且在圖網(wǎng)絡(luò)中一般2 Layers時效果最佳。因此如何在DeepGNN中既能學(xué)到更深層次信息又能避免Over-Smoothing顯得至關(guān)重要。

▌必讀系列4篇

【GCNII】Simple and Deep Graph Convolutional Networks [ICML 2020]?

【GRAND】Graph Random Neural Networks for Semi-Supervised Learning on Graphs [NeurIPS 2020]

【DAGNN】Towards Deeper Graph Neural Networks [KDD 2020]

【APPNP】Predict then Propagate: Graph Neural Networks meet Personalized PageRank [ICLR 2019]

▌Guohao Li系列3篇

Guohao Li一直在跟進深度GNN的研究，必讀。

【Guohao Li】個人首頁：https://ghli.org/【Guohao Li】

DeepGCNs: Can GCNs Go as Deep as CNNs? [ICCV 2019]

【Guohao Li】DeeperGCN: All You Need to Train Deeper GCNs [arXiv 2020]

【Guohao Li】Training Graph Neural Networks with 1000 Layers [ICML 2021]

▌2021年最新4篇推薦

Adaptive Universal Generalized PageRank Graph Neural Network [ICLR 2021]

Graph Neural Networks Inspired by Classical Iterative Algorithms [ICML 2021]

AdaGCN: Adaboosting Graph Convolutional Networks into Deep Models [ICLR 2021]

Adaptive Universal Generalized PageRank Graph Neural Network [ICLR 2021]

▌2020年10篇推薦

【DropEdge】Towards Deep Graph Convolutional Networks on Node Classification [ICLR 2020]

【PairNorm】Tackling Oversmoothing in GNNs [ICLR 2020]

Towards Deeper Graph Neural Networks with Differentiable Group Normalization [NeurIPS 2020]

Scattering GCN: Overcoming Oversmoothness in Graph Convolutional Networks [NeurIPS 2020]

Bayesian Graph Neural Networks with Adaptive Connection Sampling [ICML 2020]

Continuous Graph Neural Networks [ICML 2020]

Graph Neural Networks Exponentially Lose Expressive Power for Node Classification [ICLR 2020]

Measuring and Improving the Use of Graph Information in Graph Neural Networks [ICLR 2020]

Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View [AAAI 2020]

【JK-Net】Representation Learning on Graphs with Jumping Knowledge Networks [ICML 2018]

▌其他4篇

Deep Graph Neural Networks with Shallow Subgraph Samplers [arXiv 2020]

Tackling Over-Smoothing for General Graph Convolutional Networks [arXiv 2020]

Effective Training Strategies for Deep Graph Neural Networks [arXiv 2020]

Revisiting Over-smoothing in Deep GCNs [arXiv 2020]

關(guān)注【學(xué)姐帶你玩AI】

論文推薦應(yīng)有盡有