henhaobio
Sign in

bioRxivBioinformaticsPreprint

A deep learning approach to pattern recognition for short DNA sequences

Akosua BusiaiD, George E. DahliD, Clara FannjiangiD, David H. Alexander, Elizabeth H. DorfmaniD, Ryan E. Poplin, Cory Y. McLeaniD, Pi-Chuan ChangiD, Mark A. DePristoiD

PDFDOICode

In-depth summary

Claude Opus 5.5 · xhigh effort · 8 Nov 2026 · from the full text of version 3 (10 Aug 2019) · CC BY 4.0

A convolutional network trained on 16S reads from 13,838 species reaches within 2.0% of training-set Bayes-optimal species accuracy and usually beats k-mer, alignment and Kraken2 on held-out-species genera.

Short sequences are usually labelled by k-mer matching (which the paper says loses positional information), alignment (oversimplified affine gap penalties) or HMM-like models (which it argues DNNs beat at modelling dependencies between positions). The authors say earlier neural-network taxonomic classifiers lacked the data and validation to show practical value, and none had classified realistic-scale sequencing data at fine taxonomic resolution or compared in depth against existing tools.

Key results

  1. ShownEach network was trained for one read length and almost memorizes the species labels of its training reads. Summing its species probabilities into higher ranks also gives accurate genus, family and order labels.

    On their own training sets (NCBI25 to NCBI200), every DNN came within 2.0% of Bayes optimal species accuracy, while BWA averaged 4.6% below it. Marginalized predictions were within 1.0%, 0.6% and 0.5% of optimal for genus, family and order. On NCBI250 at species level, longer-read models were within 4.1% of optimal vs 6.1% (DNN25) and 13.6% (DNN50). · Training Performance; Figure legends

  2. ShownFor reads from species held out of training, the networks assign the correct genus more often than an 8-mer naive Bayes classifier. In most settings they also beat BLAST/BWA alignment, and the gap grows as injected noise rises.

    Genus accuracy on held-out-species reads over 24 read-length/noise settings: DNNs beat naive Bayes by 1.9% to 51.9% and the better of BLAST/BWA in 20 of 24 (by 0.1% to 32.5%). In the other 4 (low noise, 200 bp) BLAST led by at most 1.3%. With 5% and 10% noise on 200 bp reads, BWA scored under 3% vs 81.1% and 76.1% for DNN200. · Experiments on Held-out Species; Figure legends

  3. ShownAgainst Kraken2 on held-out-species reads, the networks give up some precision at low noise for much higher sensitivity. Their genus-level F1 is higher in every tested setting.

    DNN F1 exceeded Kraken2's at every read length and noise rate tested. Kraken2's PPV peaked at low noise, but its sensitivity was much lower even on noise-free reads. Once noise passed about 1%, DNN PPV mostly matched Kraken2's and exceeded it on 100 and 200 bp reads. DNN sensitivity, PPV and F1 coincide because it labels every read. The text gives no numeric values. · Kraken Comparison; Figure legends

  4. ShownOn 57 experimental mock-community datasets, a simple iterative mixture estimate over DNN100 read predictions reconstructs composition well at genus level and less well at species level. Six archaeal genera were consistently missed.

    Five of six 20-organism replicates recovered all 17 genera exactly; species calls averaged 21.8 (16.3 true, 5.5 false). 51 runs, 59-organism community: means of 36.0/45 genera, 32.6/56 species. All 57: species PPV 0.657, sensitivity 0.610; genus 0.908, 0.824. Against RDP Classifier figures from D'Amore et al. (which used a larger reference database): 4.1 vs 55.6 spurious genera, 36.0 vs 39.3 correct genera. · Mock Community Evaluation; Figure legends

  5. ClaimedInference is projected to be cheap enough for very high throughput on machine-learning accelerators, but no runtime was measured.

    About 1.2 million floating-point operations per query. The authors project roughly 13 million reads per second on a V100 GPU and 156 million on a Cloud TPU, and note that actual performance will vary. No speed comparison with Kraken2 or the aligners is reported. · 4 Discussion

Methods, data and reproducibility

Three depthwise separable convolution layers feed two to three fully connected layers, tiled along the read and average-pooled into a softmax over 13,838 species, trained in TensorFlow (Adam) on all length-L substrings of NCBI 16S references with 0% to 16% base-flipping noise injected. Higher ranks come from summing species probabilities; community composition from an iterative mixture estimate.

  • 16S rRNA reference sequences from the NCBI RefSeq Targeted Loci Project, cut into all synthetic reads of 25, 50, 100, 150 and 200 bp with NCBI Taxonomy labels · bacteria and Archaea (text notes Archaea are underrepresented) · n = 19,851 reference sequences; 13,838 species from 2,768 genera · BioProjects 33175 and 33317
  • Held-out-species split of the NCBI synthetic reads for genus-level evaluation on novel species · bacteria and Archaea · n = 12,598 training species; 1,230 (S3) or 1,240 (S9) held-out species
  • 20-organism mock community, Illumina MiSeq replicates · n = 6 replicates · PRJEB4688 (ENA); runs ERR348713-5 and ERR619081-3
  • 59-organism mock community, all sequencing runs · mixed community including archaeal genera (e.g. Pyrococcus, Sulfolobus) · n = 51 runs · PRJEB6244 (ENA)
Methods & tools
TensorFlow, Adam optimizer, BLAST, BWA, Kraken2, RDP Classifier (basis of the naive Bayes baseline), NCBI Taxonomy Browser, NCBI RefSeq Targeted Loci Project, European Nucleotide Archive, Illumina MiSeq
Data & accessions
gs://brain-genomics-public/research/seq2species/
Software & environment
TensorFlow with the Adam optimizer (versions not stated). Kraken2 downloaded 2018-09-28 and run with 48 threads; NCBI references downloaded 2017-11-27.
Assay conditions & compute
Training hardware and time not reported. Only projected inference throughput is given: about 1.2 million FLOPs per query, which the authors estimate at about 13 million reads/s (V100 GPU) up to 156 million reads/s (Cloud TPU).
Reproducibility gaps
Hyperparameters, model selection, baseline settings and replicate details are in appendices and supplementary data that are not in this text. The code link points to the general tensorflow/models research directory, and no trained models or software versions are mentioned.
Limitations (6)
  • Paired-end reads were not tested; the authors say concatenating the mates would be a simple extension. · stated by the authors
  • Predicted probabilities may need adjusting (for example, for unequal coverage in the training data) before they are properly calibrated. · stated by the authors
  • Use on full metagenomes would likely need a way to flag queries outside the training domain, the way Kraken2 gains PPV by leaving some reads unclassified. · stated by the authors
  • Base-flipping noise does not capture all aspects of real sequencing errors, and the highest noise rates, where the gains are largest, may be unrealistic for Illumina. Behaviour on more error-prone sequencers still needs testing. · stated by the authors
  • The injected noise is substitution only (no insertions or deletions), so robustness to indel errors is not tested. · noted in this summary
  • Speed is projected from a count of floating-point operations, not measured. No runtime comparison with Kraken2 or the aligners is reported, although the paper calls runtime critical. · noted in this summary

Read this if you classify 16S or other marker-gene reads taxonomically, or want an early benchmark of convolutional networks against BLAST, BWA, naive Bayes and Kraken2 on held-out species under injected sequencing noise.

Citations
63

Abstract

MotivationInferring properties of biological sequences--such as determining the species-of-origin of a DNA sequence or the function of an amino-acid sequence--is a core task in many bioinformatics applications. These tasks are often solved using string-matching to map query sequences to labeled database sequences or via Hidden Markov Model-like pattern matching. In the current work we describe and assess an deep learning approach which trains a deep neural network (DNN) to predict database-derived labels directly from query sequences. ResultsWe demonstrate this DNN performs at state-of-the-art or above levels on a difficult, practically important problem: predicting species-of-origin from short reads of 16S ribosomal DNA. When trained on 16S sequences of over 13,000 distinct species, our DNN achieves read-level species classification accuracy within 2.0% of perfect memorization of training data, and produces more accurate genus-level assignments for reads from held-out species than k-mer, alignment, and taxonomic binning baselines. Moreover, our models exhibit greater robustness than these existing approaches to increasing noise in the query sequences. Finally, we show that these DNNs perform well on experimental 16S mock community dataset. Overall, our results constitute a first step towards our long-term goal of developing a general-purpose deep learning approach to predicting meaningful labels from short biological sequences. AvailabilityTensorFlow training code is available through GitHub (https://github.com/tensorflow/models/tree/master/research). Data in TensorFlow TFRecord format is available on Google Cloud Storage (gs://brain-genomics-public/research/seq2species/). Contactseq2species-interest@google.com Supplementary informationSupplementary data are available in a separate document.

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISBebris '20Zhao '21Wang '23Liang '20Mock '22Kishk '18
68% semantic matchBioinformatics
Kristaps Bebris, Inese Poļaka·Information Technology and Management Science·Dec 15, 2020