MillionTrees: A global benchmark for tree detection across geography, sensor, and annotation geometry

Airborne tree monitoring using remotely sensed data, combined with the growth of computer vision models for object detection and segmentation, can facilitate broad scale assessment of tree health, ecology and ecosystem function. However, datasets for airborne tree detection have been largely limited to a single geography, resolution, or annotation approach, making it difficult to develop general models and benchmark their utility for broad applications. We compile existing and newly published annotation datasets to create a comprehensive dataset for training and evaluating airborne tree detection in RGB imagery. The MillionTrees dataset has over a million individual trees from a broad range of forest types, tree species, and geographic locations. In addition, it has six million weakly supervised annotations from a range of sources including airborne LiDAR and photogrammetry, arborist surveys, and simulated landscapes. An evaluation standard is provided to facilitate rapid model development. We benchmark several existing tree detection algorithms and demonstrate that while current open-source models do not generalize across the diverse datasets in the benchmark, training on MillionTrees improves both within-domain and out-of-domain performance. Held-out validation datasets using ground truth derived from terrestrial LiDAR paired with overhead RGB show that overfitting to test data is common and hyperparameter tuning is an underappreciated challenge towards a global zero shot tree detection method. We discuss the limitations of the currently available data for airborne RGB tree detection, how to create more robust evaluation data and the need for better understanding of the uncertainty and utility of tree detection from airborne remote sensing.