Humanoid robots are now capable of vocalizing their thought processes, "thinking out loud" in a way that mimics how humans use their inner thoughts to evaluate situations, gain clarity or seek moral guidance, according to a new study from Italy that found robots were better able to resolve conflicts and complete tasks when using self-dialogue.

In a paper published Wednesday in iScience, researchers from the University of Palermo developed a computational model of inner speech based on a cognitive architecture, which is a kind of software system that simulates cognitive functionalities of the brain to model human thought. They created the inner language functionality and then deployed it in Pepper, a humanoid robot first introduced by SoftBank Robotics in 2014. This allowed Pepper to speak out loud to itself while it completed an action.

Pepper is designed to interact with humans, and is most often used in hospitality functions or for research. Arianna Pipitone and Antonio Chella, robotics researchers at the University of Palermo and co-authors of the paper, told The Academic Times that they ran an experiment in which a human user set a dining table with Pepper according to a set of etiquette rules. They gave Pepper some instructions contradicting those rules in order to study how its self-dialogue skills would influence human-robot interactions. 

"The idea is that when we think, we usually talk to ourselves using an inner voice [meaning] our thoughts, and sometimes an outer voice," Pipitone and Chella explained. "Hearing our inner voice as if it were coming from the outside helps us to better understand and clarify our thinking, desires and intentions. We reproduced some characteristics of the human's inner voice in a robot."

Being able to hear the robot's inner voice makes the robot more transparent and accessible to the human user, the authors said, because they can more easily follow the robot's way of thinking. An experienced user can examine the robot's software log, but vocalized speech is ideal for interactive robots like Pepper that are deployed in common settings with the general public.

"People are surprised when they hear a robot's inner voice," Pipitone and Chella said. "They don't expect the robot to 'think,' because usually a robot is considered a machine that simply follows precise pre-programmed instructions."

The authors explained that when the robot sees an apple, for example, it can use the inner speech functionality to recognize the apple and say to itself, "There's an apple." Then the robot begins to think that an apple is a fruit, and it may say to itself, "Apples are fruits." Then it may think about other fruits, like pears and oranges, and the robot could infer "Maybe there's an orange, too," leading it to begin a visual search for an orange in the setting. 

"In this way, the robot starts a form of reasoning based on the inner dialogue," Pipitone and Chella explained. "When the robot detects conflict, it may [get] out of the stalemate by such reasoning [as] 'What should I do in that situation? What do I know about the context? What would happen if I did it?' Inner speech starts further evaluations and concerns, helping the robot to solve conflict in the best way."

The researchers tested Pepper on several tasks with inner speech and without. The robot had a higher task-completion rate when using self-dialogue and was better at resolving dilemmas overall. 

In the study, a human user and Pepper worked together to set a table. Some of the tasks they had to complete were simple with no conflict — for example, a human user asking Pepper, "Give me the napkin." But in some trials, the user asked Pepper to place a napkin in the wrong spot, contradicting the default position from the etiquette rule. This led to a conflict wherein the robot had to evaluate whether to comply with the user's request or follow the rule. Pepper was able to ask itself a series of self-directed questions, conclude that the user might be confused and confirm the user's request. The inner speech function also makes the user aware of the conflict, and they can intervene if necessary. 

"If the user gives the robot a command that does not conform to the rules implemented in the robot, then a tension arises," the authors explained to The Academic Times. "The robot starts to explore all possibilities to escape the threat, and the user becomes aware of the threat." 

In 30 total trials during which Pepper used inner speech while completing the tasks, the robot was successful 87% of the time. And in 30 trials where Pepper did not use inner speech, it was successful 60% of the time. It took Pepper longer to complete the trials using inner speech because it executed more steps and interacted more with the user.

Pepper also outperformed the international standards of functional and moral requirements for collaborative robots using the inner speech function. The guidelines, which were enacted by the International Organization for Standardization and the United Nations Educational, Scientific and Cultural Organization's World Commission on the Ethics of Scientific Knowledge and Technology, define the parameters that robots must meet in collaborative scenarios. A variety of machines adhere to the guidelines, including humanoid artificial intelligence and mechanical arms on a manufacturing line.

As the basis of their model of inner speech, Pipitone and Chella integrated ACT-R, a state-of-the-art cognitive architecture, with the Robot Operating System module, a system for robot programming that enables the robot to perform motor actions. The resulting framework was then deployed on the Pepper robot. 

The authors noted in the paper that while inner speech can be applied in many robotics contexts where humans and robots work together, including learning, regulation and attention, the function is not ideal in every setting. For example, with artificial intelligence assistants such as Siri and Alexa, it would be a waste of time for the user to to hear the systems' inner speech when giving simple commands like "turn on the lights." It also would not be useful in high-risk situations where the robot is expected to act quickly, such as self-driving cars.

"The robot's self-[dialogue] provides many advantages: it makes the robot's underlying decision processes more transparent, and it makes the robot more reliable for the partner," the authors reported in the paper. "Moreover, the interaction becomes more robust because further plans and strategies may emerge by following [the] robot's inner speech."

Though Pipitone and Chella are roboticists and the only authors credited on the paper, they emphasized that the work was an interdisciplinary effort. Their research group consists of software engineers, psychologists, philosophers and neuroscientists.

"We strongly believe that more interdisciplinary research is needed today to advance the state of the art in robotics. This is not easy, of course, because of the diversity of languages and interests," they said. "Perhaps people who are interested in research as a profession should be trained early, since their Ph.D., to do research in groups with different competences."

The study, "What robots want? Hearing the inner voice of a robot," published April 21 in iScience, was authored by Arianna Pipitone and Antonio Chella, University of Palermo.