henhaobio
Sign in

bioRxivBioinformaticsPreprint

ProteinGCN: Protein model quality assessment using GraphConvolutional Networks

Soumya Sanyal, Ivan V. AnishchenkoiD, Anirudh Dagar, David A. BakeriD, Partha Pratim TalukdariD

PDFDOICodeData
In-depth full-text digest available · Jump to digest ↓

Abstract

Blind estimation of local (per-residue) and global (for the whole structure) accuracies in protein structure models is an essential step in many protein modeling applications. With the recent developments in deep-learning, single-model quality assessment methods have been also advanced, primarily through the use of 2D and 3D convolutional deep neural networks. Here we explore an alternative approach and train a graph convolutional network with nodes representing protein atoms and edges connecting spatially adjacent atom pairs on the dataset Rosetta-300k which contains a set of 300k conformations from 2,897 proteins. We show that our proposed architecture, PO_SCPLOWROTEINC_SCPLOWGCN, is capable of predicting both local and global accuracies in protein models at state-of-the-art levels. Further, the number of free parameters in PO_SCPLOWROTEINC_SCPLOWGCN is almost 1-2 orders of magnitude smaller compared to the 3D convolutional networks proposed earlier. We provide the source code of our work to encourage reproducible research.1

In-depth summary

Claude Opus 5.5 · xhigh effort · 8 Nov 2026 · from the full text of version 1 (7 Apr 2020) · CC BY 4.0

Graph convolutions on atom graphs with distance, orientation and bond edge features predict per-residue and global lDDT of protein models, outperforming VoroMQA and retrained Ornate while using about 100k parameters.

Ranking protein structure models and locating their inaccurate regions needs both global and per-residue accuracy estimates; earlier deep learning estimators apply 3D convolutions to voxelized grids, which need rotational data augmentation, or to boxes around each residue (Ornate). ProteinGCN instead represents the model as a graph of atoms, which is rotation-invariant by construction, and adds inter-atomic orientation features that the authors say most molecular graph convolutional networks lack.

Key results

  1. ShownProteinGCN outperforms VoroMQA and Ornate (retrained on Rosetta-300k) in Pearson correlation between predicted and reference lDDT (local Distance Difference Test) on both the Rosetta-300k test split and the relaxed CASP13 server models.

    Table 1 reports Pearson's r against reference lDDT for local and global scores on Rosetta-300k (random 60/20/20 split) and CASP13 (80 targets, 150 models each). The authors call ProteinGCN's gains considerable and consistent. The table's values are not in the available text. · 5.1 Performance Comparisons; 4.1 Protein datasets; 4.2 Baselines; A.1 Training Strategy

  2. SuggestedWith the same features, the graph convolution beats the 3D CNN. ProteinGCN-Base (edge-coordinate features, residue-level loss only) improves consistently over Ornate, which the authors take as evidence that GCNs suit protein structures better than 3D CNNs.

    ProteinGCN-Base is restricted to edge coordinate (EC) features and the local loss so that it is directly comparable to Ornate. The authors report a consistent improvement over Ornate in Table 1 (values not in the available text). Only local correlation is reported for it, because it has no global loss. · 4.2 Baselines; 5.1 Performance Comparisons; Tables

  3. SuggestedIn the ablation, removing the global-score loss term significantly lowers performance on both datasets, and adding edge-coordinate (orientation) features gives some gain in local prediction.

    Table 2 sequentially removes components (edge features among edge distance ED, edge coordinate EC and bond type BT, and the global loss) and reports Pearson's r on Rosetta-300k and CASP13. The values and any significance testing are not in the available text. · 5.2 Ablation Study; Tables

  4. SuggestedOn CASP13 target T1008, predicted global lDDT tracks the reference across models. For three models at different global accuracy levels, the per-residue predictions follow the reference trends, which could point to regions that need refinement.

    Pearson's r = 0.778 between predicted and reference global lDDT for the T1008 models (Figure 3B). Local lDDT profiles of three models are compared visually with the reference (Figure 3C). This is a single target, analysed qualitatively. · 5.3 Qualitative Analysis; Figure legends

  5. ClaimedProteinGCN is compact, with about 100k trainable parameters, which the authors put at 20-fold fewer than Ornate and 3D CNNs. The abstract instead says 1 to 2 orders of magnitude fewer.

    The appendix gives around 100k trainable parameters (depending on the setting). The Introduction says 20-fold fewer learnable parameters than 3D CNNs and the Conclusion says 20x fewer than Ornate. Ornate's own parameter count is not stated in the text. · Abstract; 1 Introduction; 6 Conclusion; A.2 Model Parallelization

Methods, data and reproducibility

Protein models become graphs of heavy atoms (167 one-hot types), each linked to its 50 nearest neighbours, with edge features for Gaussian-expanded distance, coordinates in local atomic frames, and chemical bonding. Edge-gated graph convolutions learn atom embeddings, mean-pooled into residue and protein embeddings that feed heads for local and global lDDT, with equally weighted mean squared error losses.

  • Rosetta-300k: RosettaCM homology models with dual-space relaxation, built for protein chains 50 to 300 residues long with resolution of 2.5 Å or better · n = 2,897 protein chains; 100 models per chain (300k conformations)
  • CASP13 server models selected for stage 2 of the EMA (Evaluation of Model Accuracy) experiment, dual-space relaxed in Rosetta · n = 80 targets; 150 models per target
Methods & tools
ProteinGCN, PyTorch (DataParallel), Rosetta (RosettaCM, dual-space relaxation), VoroMQA, Ornate, lDDT
Assay conditions & compute
Trained with PyTorch DataParallel on a 56-core machine with 6 NVIDIA GTX 1080Ti GPUs, which gave a 30% training speedup. Training ran 50 epochs of SGD (learning rate 0.001, momentum 0.9). No wall-clock time or memory figures are given.
Reproducibility gaps
No download link is given for Rosetta-300k (described in a manuscript under submission) or for the Rosetta-relaxed CASP13 models; no software versions; the hyperparameter grid (Table 3) and local-frame definitions (Table S1) are referenced but not in the available text.
Limitations (3)
  • The authors note that the model's size makes batching hard: larger mini-batches do not fit on a single GPU, which they addressed by spreading training across several GPUs with data parallelism. · stated by the authors
  • Train, validation and test sets come from a random 60/20/20 split. Nothing in the text says proteins or similar sequences were kept apart across sets, so Rosetta-300k test scores may benefit from related chains or models seen in training. · noted in this summary
  • There are only two external baselines (VoroMQA, and an Ornate the authors retrained themselves) and no significance tests. All CASP13 models were Rosetta-relaxed before scoring, so performance on unrelaxed server models is untested. · noted in this summary

Read this if you build quality assessment or scoring functions for protein structure models and want an atom-level graph network with orientation-aware edge features and released code to compare against.

Citations
51

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISHou '19He '23Zamora‐Rese '19Gligorijevi '21Cao '20Chen '22
76% semantic matchBioinformatics
Jie Hou, Renzhi Cao, Jianlin Jack Cheng·bioRxiv·Mar 28, 2019· 20 citations