LabOS: The AI-XR Co-Scientist That Sees and Works With Humans

Abstract

Modern science advances fastest when thought meets action. LabOS represents the first AI co-scientist that unites computational reasoning with physical experimentation through multimodal perception, self-evolving agents, and XR-enabled, embodied human-AI collaboration. To empower agentic AI with embodied intelligence, we built LabSuperVision, the first vision-language-model (VLM) benchmark spanning biomedical and materials science laboratories, designed to evaluate scientific perception and reasoning. This endowed LabOS with visual reasoning capabilities that surpass those of general-purpose models on laboratory-specific tasks. By connecting multi-model AI agents, smart glasses, and human-AI-robot collaboration, LabOS allows AI to see what scientists see, understand experimental context, and participate in real-time execution through immersive XR interaction and robotic autonomation. Across applications-from cancer immunotherapy target discovery to cell engineering-LabOS shows that AI can move beyond computational design to participation, turning the laboratory into an intelligent, collaborative environment where human and machine discovery evolve together.