Open Soil Spectral Library (OSSL): Building reproducible soil calibration models through open development and community engagement

Abstract

Soil spectroscopy is a widely used method for estimating soil properties that are important to environmental and agricultural monitoring. However, a bottleneck to its more widespread adoption is the need for establishing large reference datasets for training machine learning (ML) models, which are called soil spectral libraries (SSLs). Similarly, the prediction capacity of new samples is also subject to the number and diversity of soil types and conditions represented in the SSLs. To help bridge this gap and enable hundreds of stakeholders to collect more affordable soil data by leveraging a centralized open resource, the Soil Spectroscopy for Global Good has created the Open Soil Spectral Library (OSSL). In this paper, we describe the procedures for collecting and harmonizing several SSLs that are incorporated into the OSSL, followed by exploratory analysis and predictive modeling. The results of 10-fold cross-validation with refitting show that, in general, mid-infrared (MIR)-based models are significantly more accurate than visible and near-infrared (VisNIR) or near-infrared (NIR) models. From independent model evaluation, we found that Cubist comes out as the best-performing ML algorithm for the calibration and delivery of reliable outputs (prediction uncertainty and representation flag). Although many soil properties are well predicted, total sulfur, extractable sodium, and electrical conductivity performed poorly in all spectral regions, with some other extractable nutrients and physical soil properties also performing poorly in one or two spectral regions (VisNIR or Neospectra NIR). Hence, the use of predictive models based solely on spectral variations has limitations. This study also presents and discusses several other open resources that were developed from the OSSL, aspects of opening data, current limitations, and future development. With this genuinely open science project, we hope that OSSL becomes the driver of the soil spectroscopy community to accelerate the pace of scientific discovery and innovation.