henhaobio
Sign in

bioRxivBioinformatics

EcoKMER: One-stop shop for spatio-temporal metagenomic exploration using DataFed

USDOE, USDOE, Conner PhillipsiD, Joshua Brown, Juliette Duncan, Blake NedvediD, Owen P. LeiseriD, Noelani R BoiseiD, Ruonan WuiD, Lindsey AndersoniD, Olga KuchariD, Margaret Cheung, David D. PollockiD, Patrick Widener, Connah JohnsoniD

PDFDOIPreprintCodeData

In-depth summary

Claude Opus 5.5 · xhigh effort · 7 Nov 2026 · from the full text of version 2 (21 Aug 2026) · CC BY 4.0

An R-Shiny app built on DataFed lets researchers filter metagenomic sample metadata by location, date and environmental variables, then download only matching sequence files via Globus.

Repeated environmental sampling yields large, multidimensional metagenomic collections whose size, integration, visualization and provenance tracking slow discovery; downloading whole multi-year, multi-site collections for exploration is impractical. EcoKMER is positioned as a triage layer ahead of heavy analysis on platforms such as KBase and NMDC, adding interactive spatio-temporal maps to the tabular metadata browsing typical of MG-RAST.

Key results

  1. ShownThe app separates metadata exploration from file retrieval and offers two access modes. Online mode authenticates to DataFed, merges collection records into one harmonized table and downloads records through Globus. Offline mode loads local metadata files for field sites with poor connectivity.

    The Home panel is shown in Figure 2a. Online mode pulls current project metadata from live DataFed infrastructure and triggers download of preprocessed sequence data for the filtered selection. Offline mode exports filtered metadata to a file. No performance or usability measurements are reported. · EcoKMER Application Overview; Home Panel; Data retrieval and export; Figure legends

  2. ShownA map tab plots georeferenced samples coloured by date. It clusters markers according to zoom level and averages colours so that clusters keep the median temporal signal, which lets users move from regional trends to seasonal variation at one site without geographic information system (GIS) software.

    Figure 3 shows this on sampling events at a PNNL-Sequim site. Filters (dropdowns for categorical fields, range sliders for numeric ones) and map-based selection drive synchronized table, box plot, line plot and map views. · Data Analysis Panel; Spatio-temporal analysis; Data selection and filtration; Figure legends

  3. ShownMetadata are uploaded to DataFed under JSON schemas that specify column mappings, required fields, provenance relations and Globus paths for linked sequence files. EcoKMER follows DataFed's typed provenance links to find the artifacts behind filtered records and draws them as directed graphs.

    Schema checks run through external tools before upload, and DataFed adds server-side validation as records change. Provenance links are rendered as directed graphs in a figure. No validation statistics or error rates are reported. · Metadata validation and provenance; Figure legends

  4. SuggestedIn an example deployment on a Salish Sea Estuary campaign targeting cyanobacterial populations (with amoeba as a large-celled comparator), EcoKMER plotted metagenomic reads across size fractions and sampling times.

    Samples from a 6-month campaign were split into size fractions, sequenced and aligned. Figure 4 shows metagenomic reads by size fraction (a) and by sampling event time (b). The text reports no biological findings or quantitative results from this campaign. · Findings; Exemplar Analysis Visualization Workflow; Figure legends

  5. ClaimedThe authors say that filtering centrally held metadata before pulling raw files from federated endpoints reduces bandwidth and computational overhead compared with downloading whole collections.

    The authors call the bandwidth saving significant and note that metagenomic collections are commonly terabyte-scale. They give no transfer volumes, timings or other measurements. · Findings; Data retrieval and export; Data storage and access

Methods, data and reproducibility

An R-Shiny dashboard (Home and Data Analysis panels) calls DataFed's Python API via reticulate to retrieve metadata, coordinate Globus downloads and upload records with dependency tracking. Metadata are harmonized (degrees-minutes-seconds coordinates to decimal degrees, multiple date formats parsed, missing or below-detection values to NA) and plotted with leaflet and plotly.

  • Metagenomes from a 6-month Salish Sea Estuary campaign targeting cyanobacterial populations (amoeba comparator), split by size fraction; map example at a PNNL-Sequim site · environmental samples; cyanobacterial populations and amoeba targeted · DataFed Catalog ID 525611288 (NW-BRaVE collections)
Methods & tools
EcoKMER, R Shiny, DataFed, DataFed Python API, Globus, reticulate, leaflet, plotly, renv, JSON schemas
Data & accessions
https://datafed.ornl.gov/
Software & environment
R 4.4.2 or later and Python 3.9.0 or later; renv for R packages, a Python requirements file and a README; platform independent, deployed as a local Shiny server; unit tests for parsing, missing values, rendering, fixture schemas and error handling; Simplified BSD license.
Reproducibility gaps
DataFed data (NW-BRaVE collections, catalog ID 525611288) can be accessed only after registration.
Limitations (3)
  • The authors list in-interface statistical testing for evolutionary analyses, plugins for analysis software such as NW-BRaVE's Hydroplane, and large language model-assisted metadata standardization as planned future developments, not current features. · stated by the authors
  • No quantitative evaluation is given. The claimed savings in bandwidth and computational overhead, as well as responsiveness and usability, are not measured, and there is no comparison with KBase, NMDC or MG-RAST. · noted in this summary
  • The tool is demonstrated on one estuarine sampling campaign curated by the authors' own team, and the example shows read plots without reporting any downstream biological result. · noted in this summary

Read this if you run repeated environmental metagenomic sampling and want a Shiny dashboard to map, filter and triage sample metadata before pulling selected sequence files from DataFed via Globus.

Citations
1

Abstract

Spatially distributed environmental sampling generates highly complex and multidimensional datasets illuminating key insights into microbial diversity, evolutionary-coevolutionary processes, and host-pathogen interactions. While these sampling methods generate high value datasets, dataset size, the dataset integration, visualization, analysis, and provenance tracking present significant bottlenecks to scientific discovery. To address this bottleneck, we developed EcoKMER, an R-Shiny front-end application designed to streamline metagenomic data accessibility and provide geospatial context to data in support of hypothesis-driven investigations into environmental sampling, supported by DataFed as its back-end data management platform. EcoKMER enables interactive visualization and filtering of harmonized metagenomic data and metadata using an interoperable approach, allowing users to extract spatially distributed sample-based metadata on top of environmental parameters such as geolocation, temperature, pH, for investigating ecological changes across time and space enhancing sample processing methods. As an example, we deployed this tool to track analysis of metagenomes from the organisms in the Salish Sea Estuary, consistent with existing community-accepted standards. Built on top of DataFed, a flexible and robust scientific data management system built for data lakehouse architectures, EcoKMER is positioned as a powerful tool to improve sampling strategy decision making, accelerate new insights for collaborative biological and environmental research, and fostering AI-ready analyses designed to enhance discovery and guidance for bioeconomic engineering. The published manuscript is available at https://www.osti.gov/biblio/3412938

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISCrown '24Oh '25Kuhn '25Wagner '18Larkin '21Gautam '23
64% semantic matchBioinformatics
Matthew R. Crown, Matthew Bashton·Bioinformatics Advances·Jan 1, 2024· 2 citations