For Heliconius butterflies, the appearance and scent of a prospective mate are crucial factors in mating behavior. They must concurrently process both sensory signals while having tiny brains. They outperform existing AI algorithms in multisensory decision-making, which necessitate high energy consumption for comparable outcomes. A group of Penn State academics has created a more sophisticated and energy-efficient multi-sensory AI platform to bridge this gap. This discovery might revolutionize the way robotics and intelligent sensors identify possible threats like chemical spills or defective constructions.

"If you think about the AI we have today, we have excellent language processors that use audio or very good image processors based on visuals," stated Saptarshi Das, corresponding author of the paper and associate professor of engineering science and mechanics. However, the majority of animals, including humans, rely on many senses when making decisions. Although AI is fairly good at processing information from a single sense, it is not yet capable of making decisions based on many senses. Heliconius butterflies specially select their mates. To identify if the prospective mate is a Heliconius butterfly, they employ a mix of chemical and visual signals from pheromones emitted by the other butterfly. Even more amazing is the fact that they accomplish this with a little brain that consumes very little energy. Compared to current computers, which use a lot of energy, this is very different. These microscopic animals can carry out intricate computational operations while utilizing several sensory inputs at once.


( Source: Google Images) 

By employing 2D materials, scientists have discovered a means to electrically replicate the behavior of butterflies. The two materials that comprise the hardware platform that the researchers have created are graphene and molybdenum sulfide (MoS2). MoS2, a memtransitor that can handle information and memory functions, was selected because of its capacity to sense light in a way that is similar to the butterfly's vision. Conversely, graphene functions as a chemitransistor, mimicking the brain's pheromone sensing system to identify chemical compounds. To simulate the pheromones emitted by butterflies, the researchers conducted an experiment in which they exposed their dual-material sensor to a variety of colored lights and fluids with diverse chemical compositions. In the same way that a butterfly's ability to successfully mate depends on the wing color and pheromone strength corresponding to each other, they aimed to evaluate how effectively their sensor could integrate data from both the light detector and chemosensors. The output response allowed the researchers to determine that their gadgets could seamlessly combine chemical and optical information. This offers the sensor a tremendous opportunity to analyze and understand many sorts of data at the same time.

“We also introduced adaptability in our sensor’s circuits, such that one cue could play a more significant role than the other,” said Yikai Zheng, a fourth-year doctoral student in engineering science and mechanics and co-author of the study. “This adaptability is akin to how a female butterfly adjusts her mating behavior in response to varying scenarios in the wild.”


( Source: Google Images) 

Researchers claim that dual sensing in a single gadget uses less energy than the way AI systems function now. The novel approach enables the integration of two senses into a single device, as opposed to data being collected from various sensor modules and then sent to a processing module. This lessens wasteful energy use while also cutting down on delays. The next step for the researchers is to enhance their gadget to incorporate three senses, similar to how crayfish detect threats by utilizing chemical, tactile, and visual clues. The ultimate objective is to create hardware AI systems that can manage challenging scenarios involving decision-making in a variety of settings.

Their Vision ~ “We could have sensor systems in places such as a power plant that would detect potential issues such as leaks or failing systems based on multiple sensory cues,” Ghosh said. “Such as a chemical odor, a change in vibration, or detecting weaknesses visually. This would then better help the system and staff determine what they need to do to fix it quickly because it was not just relying on one sense, but multiple ones.”