henhaobio
Sign in

Nature GeneticsGenetic & Genomic MedicinePeer-reviewed journal article

The Polygenic Score Catalog: an open database for reproducibility and systematic evaluation

Samuel A. LambertiD, Laurent GiliD, Simon JuppiD, Scott C. RitchieiD, Yu XuiD, Annalisa BunielloiD, Aoife McMahoniD, Gad AbrahamiD, Michael ChapmaniD, Helen ParkinsoniD, John DaneshiD, Jacqueline A. L. MacArthuriD, Michael InouyeiD

PDFDOIPreprintCodeData
In-depth full-text digest available · Jump to digest ↓

Abstract

Polygenic [risk] scores (PGS) can enhance prediction and understanding of common diseases and traits. However, the reproducibility of PGS and their subsequent applications in biological and clinical research have been hindered by several factors, including: inadequate and incomplete reporting of PGS development, heterogeneity in evaluation techniques, and inconsistent access to, and distribution of, the information necessary to calculate the scores themselves. To address this we present the PGS Catalog (www.PGSCatalog.org), an open resource for polygenic scores. The PGS Catalog currently contains 192 published PGS from 78 publications for 86 diverse traits, including diabetes, cardiovascular diseases, neurological disorders, cancers, as well as traits like BMI and blood lipids. Each PGS is annotated with metadata required for reproducibility as well as accurate application in independent studies. Using the PGS Catalog, we demonstrate that multiple PGS can be systematically evaluated to generate comparable performance metrics. The PGS Catalog has capabilities for user deposition, expert curation and programmatic access, thus providing the community with an open platform for polygenic score research and translation.

In-depth summary

Claude Opus 5.5 · max effort · 8 Nov 2026 · from the full text of version 1 (23 May 2020) · CC BY 4.0

PGS Catalog standardizes 192 curated polygenic scores and metadata (86 traits); reapplied in UK Biobank, eight of nine colorectal cancer scores predicted disease in European-ancestry participants.

Reuse and clinical translation of polygenic scores (PGS) are hampered by incomplete reporting (about 33% of 165 reviewed papers lacked the variant details needed to recompute a score), heterogeneous evaluation designs, and inconsistent score distribution. The Catalog pairs standardized scoring files with metadata following the Polygenic Risk Score Reporting Standards, so published scores can be reapplied to new cohorts and compared on shared samples.

Key results

  1. ShownThe Catalog holds curated, uniformly annotated PGS for diseases (cardiovascular, cancer, diabetes, neurological) and anatomical, cellular and molecular traits, plus external re-evaluations of existing scores; inclusion requires external validation and complete score-calculation data.

    192 PGS mapped to 86 unique ontology terms, curated from 69 publications (the abstract cites 78), the earliest from 2008; over 180 publications screened, 110 eligible. Nine benchmarking publications evaluate nine existing PGS, and nine of the 68 PGS-development papers also compare against prior scores. · Abstract; The PGS Catalog: data content, access, and…

  2. ShownA data model of four linked objects (Scores, Samples, Performance Metrics, Publications), built on GWAS Catalog conventions and Experimental Factor Ontology trait mapping, captures reporting items (PRS-RS) co-developed with the ClinGen Complex Disease Working Group.

    Model derived from 27 highly cited PGS papers (coronary artery disease, type 1 and type 2 diabetes, BMI/obesity, breast and prostate cancer, Alzheimer's disease). Scores get persistent IDs (e.g., PGS000018) and a scoring file with genome build, rsID or position, effect allele and weight; Sample Sets get PSS IDs; metrics include OR/HR, beta, AUROC, C-index. · Development of the PGS Catalog; Supplemental Note 1. PGS Catalog Scoring Files

  3. ShownRe-applying nine catalogued colorectal cancer PGS in UK Biobank, a cohort external to all of them, produced directly comparable metrics; most scores were predictive in European-ancestry participants, and the only non-significant one had few variants and no prior European evaluation.

    Eight of nine PGS predicted colorectal cancer among 409,253 European-ancestry participants (Cox HR/C-index with age as timescale; logistic OR/AUROC; adjusted for sex, array, country, 10 genetic principal components). Two scores had effect sizes similar to prior reports (prior values plotted for PGS000074, PGS000146); PGS000151 (14 variants) was not significant. · Systematic evaluation of PGS yields comparable…; Supplemental Note 4. Colorectal cancer…; Figure legends

  4. SuggestedIn South Asian and African ancestry participants the scores were largely not significantly predictive, consistent with the reduced cross-ancestry transferability the authors highlight, although these groups are much smaller.

    Analysed subsets of 6,086 South Asian and 5,984 African ancestry participants, defined like the White British subset by self-identification plus genetic principal component clustering; scores were standardized and Cox/logistic models fit separately within each ancestry. · The need for an open resource for polygenic…; Systematic evaluation of PGS yields comparable…; Supplemental Note 4. Colorectal cancer…

  5. ShownUnderreporting is common: a sizeable share of PGS papers omit the variant-level details (positions, effect alleles, weights) needed to compute the score in new samples; complete variant information and external validation are both required for Catalog inclusion.

    About 33% of 165 papers reviewed during curation lacked adequate variant information such as chromosomal location, effect allele and weight. New PGS must report variant data, development method and training/evaluation samples, and demonstrate analytic validity in samples not used for training. · The need for an open resource for polygenic…; The PGS Catalog: data content, access, and…; A newly developed PGS

Methods, data and reproducibility

Manual literature curation plus author submissions populate a FAIR-designed (findable, accessible, interoperable, reusable) PostgreSQL database of scores, samples, performance metrics and publications served via web, FTP and REST API; utility shown by scoring nine colorectal cancer PGS in UK Biobank with PLINK and per-ancestry Cox and logistic models.

  • PGS Catalog release: curated published PGS with scoring files and development/evaluation metadata · human · n = 192 PGS; 86 traits; 69 publications (78 per abstract) · FAIRsharing bsg-d001448; Identifiers.org prefix pgs; score IDs e.g. PGS000018
  • Initial review of highly cited PGS development papers (coronary artery disease, diabetes, BMI, breast/prostate cancer, Alzheimer's) used to design the data model · human · n = 27 publications
  • Publications screened for Catalog eligibility and reviewed for completeness of variant-level reporting · human · n = over 180 screened, 110 eligible; 165 reviewed for reporting
  • UK Biobank genotyped participants; colorectal cancer from cancer/death registry linkage (ICD-9/10), censored 2016-03-31; 449 with self-reported history only excluded · human · n = 421,332 (409,253 European; 6,086 South Asian; 5,984 African)
  • Nine colorectal cancer PGS downloaded from the Catalog and recomputed with PLINK in UK Biobank · human · n = nine PGS · EFO_0005842 (trait); includes PGS000074, PGS000146, PGS000151
Methods & tools
NHGRI-EBI GWAS Catalog (ancestry, variant and trait conventions; GCST links), Experimental Factor Ontology (EFO) via EMBL-EBI Ontology Lookup Service (OLS), Europe PMC (publication metadata), Polygenic Risk Score Reporting Standards (PRS-RS), with ClinGen Complex Disease Working Group, PGS Scoring File format (gzipped tab-delimited, PLINK-compatible), PLINK (polygenic score calculation), lifelines (Cox proportional hazards, age as timescale), statsmodels (logistic regression), Django 3.0, Python 3.7, PostgreSQL 11, PGS Catalog REST API and EMBL-EBI FTP server, FAIRsharing.org and Identifiers.org registration, Google Cloud (deployment)
Software & environment
Catalog: Django 3.0, Python 3.7 and PostgreSQL 11 on Google Cloud; codebase under Apache 2.0, data under EMBL-EBI terms of use. Benchmark: PLINK scoring, then Python lifelines (Cox) and statsmodels (logistic regression), versions not stated.
Reproducibility gaps
No code is linked for the UK Biobank benchmark (the GitHub organization hosts the Catalog codebase); the scoring-format URL in Supplemental Note 1 is misspelled (pascataloa.org); per-score estimates appear only in Figure 2 and supplements, not in the text.
Limitations (5)
  • Most PGS derive from European-ancestry cohorts and are known to underperform in non-European individuals; in UK Biobank the nine colorectal cancer scores were largely not significant in South Asian and African ancestry groups. · stated by the authors
  • Published performance estimates depend on each study's design, demographics, case definitions and covariates, and direct external evaluations are scarce, so metrics reported by different papers are not directly comparable. · stated by the authors
  • Scoring files are kept in their original genome builds and no reference-sample calculations or population distributions are yet provided; harmonisation to GRCh37/GRCh38 and such references are planned to reduce variability in user-computed scores. · stated by the authors
  • Non-European benchmark groups are small (6,086 South Asian, 5,984 African participants) and case counts are not given in the main text, so non-significance may reflect low power rather than poor transferability. · noted in this summary
  • The benchmark covers one trait in one cohort and reports relative effect sizes and discrimination (HR/OR, AUROC, C-index) without calibration or absolute risk, so it demonstrates the workflow rather than clinical utility of the scores. · noted in this summary

Read this if you develop, report, or benchmark polygenic scores and want standardized scoring files, reporting items, and programmatic access to published scores with their evaluation metadata.

Citations
713
FWCI
62.00
Field percentile
top 1%

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISLambert '24Wand '21Chen '25Yang '24Moore '25Kolosov '25
83% semantic matchGenetic & Genomic Medicine
Samuel A. Lambert, Benjamin Wingfield, Joel T. Gibson+13 more·medRxiv·May 29, 2024· 37 citations