All projects
Graph Attention Networks
Reproducing Graph Attention Networks for node classification on Cora
Research · Solo · February 2026
Overview
An implementation and study of Graph Attention Networks (Velickovic et al., ICLR 2018), reproducing semi-supervised node classification on the Cora citation dataset with PyTorch Geometric.
How it works
Each node applies a shared linear transform, computes attention coefficients with its neighbors, and takes a softmax weighted sum of their features, with several attention heads in parallel for stability. My model has two GATConv layers: 8 heads of 8 features with ELU and 0.6 dropout, then a single head that outputs 7 class logits.
Setup
Cora has 2,708 papers, 5,429 citations, and 1,433 bag of words features per paper, with only 140 labeled training nodes. I trained for 200 epochs with Adam (learning rate 0.005, weight decay 5e-4).
Results
Test accuracy came out around 80%, against about 83% reported in the paper. The notebook also produces training curves and a t-SNE plot of the learned node embeddings.