EMhub: a web platform for data management and on-the-fly processing in scientific facilities

Abstract

Most scientific facilities produce large amounts of heterogeneous data at a rapid pace. Managing users, instruments, reports, and invoices presents additional challenges. To address these challenges, we introduce EMhub, a web platform designed to support the daily operations and record-keeping of a scientific facility. EMhub enables easy management of user information, instruments, bookings, and projects. The application was initially developed to meet the needs of a CryoEM facility, but its functionality and adaptability have proven broad enough to be extended to other data-generating centers. The expansion of EMHub is enabled by the modular nature of its core functionalities. The application allows external processes to be connected via a REST API, automating tasks such as folder creation, user and password generation, and execution of real-time data processing pipelines. EMhub has been used for several years at the Swedish National CryoEM Facility and installed in the CryoEM center at the Structural Biology Department at St. Jude Childrens Research Hospital. A fully automated single-particle pipeline has been implemented for on-the-fly data processing and analysis. At St. Jude, the X-Ray Crystallography Center and the Single-Molecule Imaging Center have already expanded the platform to support their operational and data management workflows.