A swan’s beak, a blossoming lotus flower, a delicate bracelet: These are some of the highly stylized forms the human hand can make in a classical Indian dance form known as Bharatanatyam. The dancers rely on a visual language of gestures, known as mudras—which involve precise finger, wrist, and palm movements—together with dramatic facial expressions and angular body movements to knit together powerful, ancient stories and express heightened emotion.
Now, a team of scientists is using this language of dancing hands to train robots, which could ultimately lead to better prosthetics design, robot-assisted therapy for stroke survivors, and robots who can perform complex manual tasks such as folding laundry or perhaps even playing a musical instrument.
The mudras, the scientists found, can be broken down into an alphabet of six basic building blocks that together make a good proxy for most hand movements. Instead of teaching robots to mimic natural human gestures, such as pinching or cupping, the scientists are developing methods to teach the machines these artistic forms, first. The researchers published their results in Scientific Reports.
Read more: “Robots Can’t Dance”
“We noticed dancers tend to age super-gracefully,” says Ramana Vinjamuri, an electrical engineer at the University of Maryland, Baltimore County, and a co-author of the new research, in a statement. “That was a huge inspiration for us when we started looking for richer alphabets of movement. With dance, we are looking not just at healthy movement, but super healthy. And so the question became, could we find a ‘superhuman’ alphabet from the dance gestures?”
Vinjamuri began hunting for this so-called alphabet of hand movement more than a decade ago. He was inspired to take cues from Bharatanatyam dancers after a 2023 conference on the brain at the Indian Institute of Technology, where he attended a session that explored how ancient Indian traditions could be recruited to help solve modern problems. While brainstorming, he hit upon the idea of looking to mudras to improve robot hand mobility.
Vinjamuri and his colleagues started out by analyzing 30 natural hand grasps used to pick up objects, ranging from a tiny bead to a water bottle. They found six basic building blocks that when combined could be used to describe 99 percent of the gestures. Then they ran the same test with the mudras, and found a different set of six building blocks that could describe 94 percent of the mudras. But when they used these different sets of alphabets to analyze an unrelated set of gestures—15 letters from the American Sign Language alphabet—the mudras-derived alphabet won the contest, according to the scientists.
“The mudras-derived alphabet is definitely better than the natural grasp alphabet because there is more dexterity and more flexibility,” said Vinjamuri.
The researchers are now training a standalone robotic hand and a humanoid robot to use this mudras alphabet. Ultimately, the goal is to make the robot hand more human-like. What looks like art, the robots are learning as the grammar of biology and movement. 
Enjoying Nautilus? Subscribe to our free newsletter.
Lead image: Monstar Studio / Shutterstock
