Vision Transformers Enable Advanced Plant Phenotyping in Controlled Environments

Abstract

Reliable plant segmentation in high-throughput phenotyping must transfer across species and imaging conditions without repeated model tuning or extensive reannotation. We compare three segmentation strategies using images from Oak Ridge National Laboratory's Advanced Plant Phenotyping Laboratory: (i) fixed color-based thresholding, (ii) supervised U-Nets trained from scratch, and (iii) pretrained vision transformers fine-tuned for binary segmentation. Models were evaluated on a held-out test set and a generalization set that comprised unseen species. On the held-out test set, thresholding, the best U-Net, and the best vision transformer achieved mean Dice scores of 58.3, 96.6, and 97.3, respectively. On the generalization set, the corresponding Dice scores were 56.5, 86.2, and 95.7. Thresholding remained effective on some datasets but failed when plant appearance changed. Supervised U-Net training resolved within-distribution errors but failed to generalize to novel species and backgrounds. Pretrained vision transformers consistently produced high-accuracy segmentations across the evaluated species, views, soil backgrounds, and tray types. These results benchmark the practical progression from fixed rules to task-specific supervision and pretrained visual representations for controlled-environment plant phenotyping.