henhaobio
Sign in

bioRxivBioinformaticsPreprint

GimmeMotifs: an analysis framework for transcription factor motif analysis

Niklas BruseiD, Simon J. van HeeringeniD

PDFDOICodeData

In-depth summary

Claude Opus 5.5 · xhigh effort · 7 Nov 2026 · from the full text of version 1 (20 Nov 2018) · CC BY 4.0

GimmeMotifs bundles motif discovery, scanning, clustering and benchmarking; across 495 ENCODE ChIP-seq peak sets, the best motif from 14 finders out-scored BioProspector, the top single finder, on ROC AUC.

No de novo motif finder performs consistently across data sets, many are tested on only a few, and few tools compare motif activity across two or more data sets. This release grows the earlier ChIP-seq motif discovery pipeline into a framework (command line and Python API) with a clustered non-redundant motif database, large benchmarks of databases and finders, and maelstrom, an ensemble method for differential motif activity across many conditions.

Key results

  1. ShownAcross 14 de novo finders, BioProspector, MEME and Homer consistently give the best primary motifs on ChIP-seq peaks, while Trawler, Improbizer, Posmo and Weeder trail; Weeder, a top scorer in the Tompa et al. benchmark, ranks below average here.

    495 ENCODE peak files (270 proteins); 100 bp around summits of the top 5,000 peaks, half used for prediction and half for evaluation, flanking regions as background. ROC AUC 0.58 to 0.98 (mean 0.75). BioProspector stayed within 0.026 recall at 10% FDR of the best overall motif in 75% of cases; Trawler, Improbizer, Posmo and Weeder lost 0.11 to 0.17 recall on average. · Large-scale benchmark of de novo motif finder…; De novo motif prediction benchmark

  2. ShownPicking the best motif per data set from all finders, as the GimmeMotifs ensemble does, gives higher ROC AUC than the best single tool; no single finder supplied the best motif for every TF.

    Ensemble best-motif ROC AUC distribution significantly above BioProspector, the best single method (p < 1e-21, Wilcoxon signed-rank); Trawler and Posmo significantly below GADEM (p < 0.01). Running MEME at several fixed widths beat MEME with the minw/maxw options. · Large-scale benchmark of de novo motif finder…; Conclusions

  3. ShownThe clustered, non-redundant gimme.vertebrate.v5.0 database separates ReMap ChIP-seq peaks from random genomic background as well as or better than most public motif databases; composite collections do best overall.

    294 TFs from ReMap 2018 (up to 5,000 peaks each); best motif per TF and database. Mean maximum ROC AUC difference between databases 0.05; largest (~0.24) for TFs absent from Factorbook, such as ONECUT1, SIX2 and TP73. On recall at 10% FDR the GimmeMotifs database beat most others; RSAT clustered motifs scored comparably. · Benchmark of transcription factor motif…; Transcription factor motif database benchmark; Conclusions

  4. Suggestedmaelstrom (Lasso, Bayesian ridge, multiclass and boosted-tree regression merged by rank aggregation) recovers known lineage regulators from H3K27ac signal at the 50,000 most dynamic hematopoietic enhancers.

    193 H3K27ac ChIP-seq sets (mainly BLUEPRINT) counted at enhancers from 69 DNase I experiments; their correlation formed six cell-type clusters. High activity: SPI1 (PU.1) in monocytes and macrophages, CEBP in neutrophils and monocytes, RUNX in T and NK cells, GATA1 in erythroid cells; 78 motifs reached absolute activity >= 5. · Differential motif analysis of hematopoietic…; Motif analysis of hematopoietic enhancers; Figure legends

  5. Suggestedmaelstrom also points to factors less studied in blood cells: high motif activity for ZEB1 and Snail-family motifs, and for NANOG in endothelial cells.

    Support comes from cited literature (Snai2 and Snai3 needed for mature T and B cells in mice; thymocyte defect in ZEB1 knockout mice; NANOG in endothelial proliferation and angiogenesis), not from new experiments in this paper. · Differential motif analysis of hematopoietic…

Methods, data and reproducibility

A Python framework (C motif scanner) that runs up to 14 external de novo finders on a prediction split, then filters and clusters motifs on a validation split; it adds scanning, performance metrics (gimme roc) and maelstrom, which merges eight differential-enrichment methods by rank aggregation. Demonstrated via database and de novo finder benchmarks and a hematopoietic enhancer analysis.

  • ReMap 2018 v1.2 non-ENCODE ChIP-seq peaks (hg38), TFs with at least 1,000 peaks, up to 5,000 sampled per TF, against shuffled genomic background · human (hg38 peaks) · n = 294 TFs
  • ENCODE spp ChIP-seq peaks, January 2011 freeze, top 5,000 peaks per file · n = 495 peak files for 270 proteins
  • Hematopoietic DNase I experiments (BLUEPRINT and ROADMAP), merged into a putative enhancer set · human · n = 69 experiments · Supplementary Table S1
  • Hematopoietic H3K27ac ChIP-seq, mainly BLUEPRINT, counted in 2 kb around DNase I summits; 50,000 most dynamic enhancers kept · human (counted at the human DNase I enhancer set) · n = 193 experiments · Supplementary Table S1
  • Public motif databases benchmarked: JASPAR 2018 vertebrate, SwissRegulon, Homer, Factorbook, ENCODE (Kheradpour et al.), HOCOMOCO, RSAT clustered, IMAGE · n = 8 databases, compared with gimme.vertebrate.v5.0
  • gimme.vertebrate.v5.0: motifs from CIS-BP v1.02, ENCODE, Factorbook, HOCOMOCO v11, HOMER v4.10, JASPAR 2018 and SwissRegulon, clustered by agglomerative clustering · vertebrate · n = 1900 clusters before information-content filtering
  • H3K27ac read-count matrix over the merged DNase I enhancer regions · human · 10.5281/zenodo.1488669
  • gimme maelstrom results for the hematopoietic enhancer analysis · human · 10.5281/zenodo.1491482
Methods & tools
GimmeMotifs (gimme motifs, gimme roc, gimme maelstrom), BioProspector, MEME, Homer, scikit-learn, XGBoost, scikit-contrib-lightning, genomepy, MACS2, bowtie2, bedtools, snakemake
Software & environment
Python 3 (C scanning module), Linux/Mac OSX, MIT licence; bioconda install pulls in all 14 de novo finders, also on PyPI, releases archived on Zenodo; depends on genomepy, numpy, scipy, scikit-learn, scikit-contrib-lightning, xgboost, pysam, pandas, seaborn.
Assay conditions & compute
Part of the work ran on the Dutch national e-infrastructure (SURF) and NWO-sponsored supercomputer facilities; no hardware, runtime or memory figures are given. MEME is noted as slower than most other finders.
Reproducibility gaps
Supplementary Tables S1/S2 and File S1 are referenced but not included in the text; S16 spells the analysis repository gimme-analyis while S12 and S13 write gimme-analysis; de novo finders are external dependencies.
Limitations (6)
  • The database benchmark covers one use case (peaks against random genomic background); databases built from ChIP-seq motifs are partly overfit to this task, and ROC AUC is not the most biologically meaningful metric. · stated by the authors
  • The de novo benchmark scores only recovery of the primary motif in top-ranked peaks; it does not test finding low-abundance or co-factor motifs. · stated by the authors
  • Composite databases such as gimme.vertebrate.v5.0 are less well annotated than the curated JASPAR, and ChIP-derived motifs may reflect co-factors rather than the assayed TF, which makes linking motifs to TFs harder. · stated by the authors
  • Only PFM motif models are supported; these cannot capture inter-nucleotide dependencies, and alternative models or deep neural network approaches are left for future work. · stated by the authors
  • The ensemble gain comes from picking the best motif per data set across finders; the text does not say whether that choice used data separate from the evaluation half, so the reported advantage may be optimistic. · noted in this summary
  • maelstrom is checked only qualitatively against known lineage factors in one hematopoietic H3K27ac collection; it is not benchmarked quantitatively against the alternatives the paper names (CentriMo, MARA, IMAGE). · noted in this summary

Read this if you run de novo or known-motif analysis on ChIP-seq or accessibility peaks and want evidence on which motif finders and databases to use, or need to compare motif activity across many conditions.

Citations
81

Abstract

BackgroundTranscription factors (TFs) bind to specific DNA sequences, TF motifs, in cis-regulatory sequences and control the expression of the diverse transcriptional programs encoded in the genome. The concerted action of TFs within the chromatin context enables precise temporal and spatial expression patterns. To understand how TFs control gene expression it is essential to model TF binding. TF motif information can help to interpret the exact role of individual regulatory elements, for instance to predict the functional impact of non-coding variants.\n\nFindingsHere we present GimmeMotifs, a comprehensive computational framework for TF motif analysis. Compared to the previously published version, this release adds a whole range of new functionality and analysis methods. It now includes tools for de novo motif discovery, motif scanning and sequence analysis, motif clustering, calculation of performance metrics and visualization. Included with GimmeMotifs is a non-redundant database of clustered motifs. Compared to other motif databases, this collection of motifs shows competitive performance in discriminating bound from unbound sequences. Using our de novo motif discovery pipeline we find large differences in performance between de novo motif finders on ChIP-seq data. Using an ensemble method such as implemented in GimmeMotifs will generally result in improved motif identification compared to a single motif finder. Finally, we demonstrate maelstrom, a new ensemble method that enables comparative analysis of TF motifs between multiple high-throughput sequencing experiments, such as ChIP-seq or ATAC-seq. Using a collection of ~200 H3K27ac ChIP-seq data sets we identify TFs that play a role in hematopoietic differentiation and lineage commitment.\n\nConclusionGimmeMotifs is a fully-featured and flexible framework for TF motif analysis. It contains both command-line tools as well as a Python API and is freely available at: https://github.com/vanheeringen-lab/gimmemotifs.

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISTsukanov '26Umer '19Khan '22Sharov '25Schultheis '24Ren '16
71% semantic matchBioinformatics
A. V. Tsukanov, V. G. Levitsky·bioRxiv·Sep 20, 2026