Bioinspired Mantis-Cam Enables Underwater Geolocation

Gruev diving with Mantis-Cam. (Credit: Viktor Gruev)

Viktor Gruev is a busy person. He’s a professor twice over, at the University of Illinois in electrical and computer engineering, and at the Carle Illinois College of Medicine, where his research in imaging technologies leads to better diagnostics and treatments for patients. Gruev also spends a lot of time taking cues from nature, and he spoke to EM about recently publishing on two different innovative sensing technologies.

“We had two big papers that came out this week,” explains Gruev. “One was on this underwater geolocalization camera, and the second paper that just came out yesterday focuses on a butterfly-inspired camera for cancer detection.”

The former findings, surrounding Gruev’s Mantis-Cam, demonstrate passive underwater geolocalization using polarized underwater light for the first time. This new technology may enable innovative new undersea navigational techniques and reveal why the migratory behavior of some marine animals has been disturbed in recent years.

Gruev’s team started with a broad range of interests, from remote sensing all the way to biomedical imaging. They took this variety and ran with it as a research strength.

“We kind of find different applications and we go after them,” remarks Gruev. “It’s kind of the same philosophy in the lab; we’re looking for inspiration from nature to build better, smaller, more sensitive imaging sensors. Hopefully we’ll solve problems that were not able to be solved before with state of the art cameras.”

Now we can see through their eyes

Researchers have long known that creatures like the mantis shrimp could see far more than humans can, including polarized light, underwater. We also knew that other undersea animals like octopuses can see polarized light—but the reasons for that ability haven’t been as clear.

“Four years ago, we designed a polarization camera, one of the first published in the literature; we were working closely with marine biologists to look at polarization patterns from animal bodies and to figure out why some of these animals have polarization vision,” details Gruev. “What are they using it for? So we created a technology that can help marine biologists to study this in the animals’ natural habitat.”

Closeup view of Mantis-Cam. (Credit: Viktor Gruev)

While the team was conducting some of these measurements, Gruev noticed that the underwater background was continuously polarized, and was changing continuously depending on where they were taking measurements.

“My marine biologist colleagues were telling me there’s something wrong with your camera, because underwater everything should be horizontally polarized and should not be changing with the time, day, or other factors,” Gruev remarks. “But I was convinced that our camera was working well. We tested it diligently. So I looked at the theory, I revisited the physics of light.”

Gruev and his student dug into how light travels from air to water and wrote some of the equations, and concluded that light shouldn’t be horizontally polarized underwater; instead, it should be changing, as they observed. At that point, they began more systematic measurements, and proposed the idea—disputing the conventional wisdom in the marine biology community—that underwater polarization is changing continuously, rather than stationary and uniform.

“That was interesting, but I said, what we can do is solve an even more interesting, difficult, universal problem: we can geolocate ourselves underwater,” Gruev describes. “Which settles this argument that light under water is heterogeneous. It’s very differently polarized and is changing, and we can use that to geolocate ourselves.”

At first, this was a difficult sell.

“You have to realize, we go on field trips where we record for ten days,” explains Gruev. “You can dive maybe one to two hours per day, depending on the depths. You’re already trying to do many experiments, and then, we ask to put aside 10 minutes to do extra measurements; people are skeptical of your crazy idea. It took us about four years to do the measurements.”

Are marine animals geolocating with polarized light signals?

Mantis-Cam in action. (Credit: Viktor Gruev)

The Mantis-Cam and the research behind it opens up several interesting questions. Near the top of the list: Are marine animals using an ability to perceive and interpret polarized light to geolocate?

“A natural question that comes up from this research is how animals might be using this polarization information for navigational purposes,” states Gruev. “If they’re using it, how they’re using it; this is kind of the question that together with marine biologists we can answer.”

Given the numbers of dolphins and whales stranding in recent years, migratory problems related to inability to geolocate—and what causes that inability—is of high interest. For example, as Gruev points out, more whales are stranding near the shore of California, a new location for them. Some of Gruev’s observations during the research process seem to suggest a relationship between reduced ability to geolocate using polarization signals and pollution in water.

“I was collecting data in southern Europe, and while I was taking measurements I would drop the camera and it would continuously measure for a couple of hours,” Gruev describes. “So I recorded how it looked during the measurement period from about 100 yards above with a drone. As I was recording I noticed a wave of dirt particles in the water where I was recording, which you could only see from above. I noticed, since the drone and camera were synced, that our geolocalization becomes less accurate when the pollution particles were visibly noticeable.”

It’s an interesting point and a real question. As we pollute our water more and more, are the underwater polarization signals changing?

“From a robotics point of view, I can probably fix that,” remarks Gruev. “I can probably account for that and solve that problem for the camera. But from an animal’s point of view, they’re not used to seeing these changes. Maybe our pollution is setting them off course; it’s a possibility. We are seeing a lot of long distance migratory animals ending up in the wrong locations. Is pollution one of the factors? Maybe.”

Using polarization signals for underwater navigation

The follow-up question: Can we humans use this method for underwater navigational purposes?

“We’re hoping to develop a small platform similar to the GoPro camera that will allow us to drift sensors and explore ocean waters,” Gruev describes. “When it finds something interesting, whether it’s a chemical or an object or a species, whatever our task is, we can actually geolocate it. So we could be exploring the vast amount of ocean, the first 100 to 200 meters, using this technology, and finding new species, and understanding what’s happening with pollution. When we find new information, I think it’s critical to geolocate it so we can go back to that site and study it more carefully.”

Viktor Gruev with Mantis-Cam in Mexico. (Credit: Viktor Gruev)

One of the challenges for Gruev and the team moving forward will be improving the accuracy of the method. Ideally, the Mantis-Cam and other versions of the technology will be able to travel down to a kilometer or even sub kilometer and geolocate accurately. Gruev believes that machine learning and big data will certainly help them achieve that goal, but he never forgets what originally inspired the work: nature.

“I think going back to nature matters; if animals are using this information, how are they using it?” Gruev adds. “How’s their visual system optimized to get the accuracy that they need? We can actually learn even more from animals to build better navigational systems. It’s this artificial platform that has to be biocompatible; it has to be environmentally friendly.”

This is an exciting area, reflecting the value of basic scientific research and what it can reveal about our oceans and the environment that we live in.

“Considering 70 percent of the Earth’s surface is water, this kind of technology can help us learn more about animals, pollution, everything around us,” remarks Gruev. “I think it’s important to invest in this kind of research that can lead us to new understandings of the world we live in.”