INESC TEC collaborates with Japanese Honda to create robot with "super-hearing"

Imagine that the TV is on and the kids are playing in a certain room. To add to the noise caused by these two situations, the doorbell rings. Can this sound be detected by a robot? A group of researchers responsible for creating the HEARBO robot, among which is João Oliveira of INESC TEC’s Telecommunications and Multimedia Unit (UTM), ensure that the answer is yes.

The HEARBO has "super ears" that can separate sounds and analyse them individually, which is a significant advance in the robotics field. João Oliveira participated in this work, developing and implementeing a musical beat detector, and creating an algorithm to generate synchronised dance movements to the beat of music in real-time. All the other required robotic hearing components were also integrated to create a scenario of interactive robotic dance.

"The highlight is the robot’s ability to hear and understand the musical beats and continuous live stimuli, while simultaneously managing to verbally interact with humans and overcome the high noise levels generated by the robot’s motors while dancing using real-time noise suppression methods", João Oliveira explains.

When applied to a real scenario, the HEARBO would, for example, be able to locate a call for help in the event of a disaster.

This robot is the end result of two traineeships that the UTM researcher did at the Honda Research Institute, in Japan, as part of his doctoral thesis. João Oliveira also collaborated on three articles regarding thing project, which he presented at international robotics conferences.

The HEARBO was developed by the Honda Research Institute-Japan, in collaboration with INESC TEC and the Artificial Intelligence and Computer Science Laboratory (LIACC).