Aha! How a robot shows that it is learning
A learning robot that keeps making the same mistake may not seem very clever. But behind the scenes, it can still be learning all the time. Researchers at Leiden University have developed signals that allow a robot to show when it is learning something new.
A robot vacuum cleaner bumps into the same chair leg for the third time. To us, it may look as though it is not learning from its mistakes. But its algorithm processes every collision and adjusts its expectations.
‘We can immediately see how the vacuum cleaner should get from A to B, but a robot starts without that knowledge,’ computer scientist Joost Broekens explains. ‘Taking detours, bumping into objects and going over the same spot several times can all be part of the learning process.’
This process consists of calculations that people cannot see. That can easily lead to misunderstandings: we judge the robot by its behaviour, while most of the learning happens internally. So the robot may already have learnt something, even if this is not yet clear from the route it takes.
Robots and emotions
Broekens’ research group studies how robots can make this internal learning process easier for people to understand. His work combines computer science, psychology and research into emotions. ‘I trained as a computer scientist, but by now I can almost call myself an emotion psychologist too,’ he says with a laugh.
PhD candidate Bernard Hilpert studies how people judge robots that are learning. His user research shows that people’s impressions often do not match what is really happening inside the algorithm. If a robot wanders around or repeats mistakes, people quickly assume that it is not learning or is even broken. Hilpert therefore investigated how robots can use subtle signals to show that they are processing new information.
The maths behind the feeling
So how does a robot learn? One way is through reinforcement learning: trying out actions and comparing the results with what it expected based on earlier experiences. The algorithm repeatedly calculates what is known as the ‘temporal difference’: the difference between the expected result and the actual result.
Was the result better than expected? The robot adjusts its expectations in a positive direction. Was it worse? It adjusts them in a negative direction. These signals can be compared with processes in humans that are linked to experiences such as surprise, disappointment and relief.
Subtle changes in posture and movement can already communicate information.
A digital aha moment
To make this invisible learning process visible, the researchers developed the ‘insight signal’. This signal shows that the robot is taking an important step in its learning process. Using a light, sound or movement, the robot can display a kind of digital ‘aha moment’. This helps the human trainer notice that something is changing, even before it becomes visible in the robot’s behaviour.
Does a robot need a human face? No
How can a robot express this kind of emotion in practice? According to Broekens, it does not need a human face or have to beep constantly. Subtle changes in posture and movement can already communicate information. For example, after an unexpected result, a robot could move more slowly or carefully. People often recognise such patterns as hesitation, alarm or caution.
People teach as coaches
People do not train a robot simply by pressing a reward button. They take on different roles: as coaches, they give instructions; as technicians, they check the system; and as designers, they adapt the task or environment. Interfaces that support these natural ways of teaching could improve collaboration between people and robots.
If robots can also show when and how they are learning, people will have a better idea of when they need to help – and when they should let the robot keep trying.
The Human-AI research cluster at Leiden University
In the Human-AI research cluster at LIACS, researchers study the relationship between artificial intelligence, human intelligence and human behaviour. They explore how language and intelligent and social behaviour develop, how AI systems communicate with people, and how technology can support human creativity and understanding. Using games, robots, chatbots, other interactive systems and art, they investigate how people experience and interact with technology. The cluster brings together computer science, artificial intelligence, linguistics and psychology, connecting fundamental research into intelligence with the development of new AI technologies and practical applications.