Department of Mechanical Engineering

Researchers from Santa Clara University, New Jersey Institute of Technology and the University of Hong Kong have been able to successfully teach microrobots how to swim via deep reinforcement learning, marking a substantial leap in the progression of microswimming capability.

There has been tremendous interest in developing artificial microswimmers that can navigate the world similarly to naturally-occuring swimming microorganisms, like bacteria. Such microswimmers provide promise for a vast array of future biomedical applications, such as targeted drug delivery and microsurgery. Yet, most artificial microswimmers to date can only perform relatively simple maneuvers with fixed locomotory gaits.

Caption: The artificial intelligence-powered swimmer switches between different modes of locomotory gaits (color-coded) autonomously in tracing a complex trajectory “SWIM." From: Zou et al., Commun. Phys., 5, 158 (2022).

In the researchers’ study published in Communications Physics, they reasoned microswimmers could learn — and adapt to changing conditions — through AI. Much like humans learning to swim require reinforcement learning and feedback to stay afloat and propel in various directions under changing conditions, so too must microswimmers, though with their unique set of challenges imposed by physics in the microscopic world.

"Being able to swim at the micro-scale by itself is a challenging task," said On Shun Pak, associate professor of mechanical engineering at Santa Clara University. “When you want a microswimmer to perform more sophisticated maneuvers, the design of their locomotory gaits can quickly become intractable.".

By combining artificial neural networks with reinforcement learning, the team successfully taught a simple microswimmer to swim and navigate toward any arbitrary direction. When the swimmer moves in certain ways, it receives feedback on how good the particular action is. The swimmer then progressively learns how to swim based on its experiences interacting with the surrounding environment.

“Similar to a human learning how to swim, the microswimmer learns how to move its ‘body parts’ — in this case three microparticles and extensible links — to self-propel and turn,” said Alan Tsang, assistant professor of mechanical engineering at the University of Hong Kong. “It does so without relying on human knowledge but only on a machine learning algorithm.”

The AI-powered swimmer is able to switch between different locomotory gaits adaptively to navigate toward any target location on its own.

As a demonstration of the powerful ability of the swimmer, the researchers showed that it could follow a complex path without being explicitly programmed. They also demonstrated the robust performance of the swimmer in navigating under the perturbations arising from external fluid flows. 

"This is our first step in tackling the challenge of developing microswimmers that can adapt like biological cells in navigating complex environments autonomously," said Yuan-nan Young, professor of mathematical sciences at New Jersey Institute of Technology.

Such adaptive behaviors are crucial for future biomedical applications of artificial microswimmers in complex media with uncontrolled and unpredictable environmental factors.

"This work is a key example of how the rapid development of artificial intelligence may be exploited to tackle unresolved challenges in locomotion problems in fluid dynamics,” said Arnold Mathijssen, an expert on microrobots and biophysics at the University of Pennsylvania, who was not involved in the research. “The integration between machine learning and microswimmers in this work will spark further connections between these two highly active research areas.”

The team is grateful for the funding support from the National Science Foundation and the computational resources at the WAVE High Performance Computing Center at Santa Clara, which have enabled the pursuit of the research.

See an article highlighting this work by the American Society of Mechanical Engineers.

 The article “Gait switching and targeted navigation of microswimmers via deep reinforcement learning” was published in Communications Physics on June 21, 2022.