Model-Based Reinforcement Learning for Ultrasound-Driven Autonomous Microrobots

Abstract

AI has catalyzed transformative advancements across multiple sectors, from medical diagnostics to autonomous vehicles, enhancing precision and efficiency. As it ventures into microrobotics, AI offer innovative solutions to the formidable challenge of controlling and manipulating microrobots, which typically operate within imprecise, remotely actuated systems--a task often too complex for human operators. We implement state-of-the-art model-based reinforcement learning for autonomous control of an ultrasound-driven microrobot learning from recurrent imagined environments. Our non-invasive, AI-controlled microrobot offers precise propulsion, which efficiently learns from images in data-scarce environments. Transitioning from a pre-trained simulation environment, we achieve sample-efficient collision avoidance and channel navigation, reaching a 90% success rate in target navigation across various channels within an hour of fine-tuning. Moreover, our model initially successfully generalized in 50% of tasks in new environments, improving to over 90% with 30 minutes of further training. Furthermore, we have showcased real-time manipulation of microrobots within complex vasculatures and across stationary and physiological flows, underscoring AIs potential to revolutionize microrobotics in biomedical applications, potentially transforming medical procedures.