Robot intuition: predicting your next move!

Home Technologist Online Robot intuition: predicting your next move!

To ensure that robots can work with humans safely and intuitively in the future, control systems engineer Sandra Hirche and her team are researching new control mechanisms based on mathematical models of human behavior.


Their work should enable a robot to accurately predict the movements of its human counterpart and adjust its own reactions accordingly. The aim is to have robots learn from their own experience and continuously improve their own control functionality. Professor Hirche’s project has been funded by the European Research Council (ERC) since 2014.

Changing the world in pioneering spirit

For Sandra Hirche, research means understanding things that no one has understood before. In her own words, she loves the pioneering aspect of her work and the lifelong learning opportunity created by the constant evolution of her research topic. Growing up in Berlin, she started studying physics at the Humboldt University but later switched to engineering. Physics is primarily about observing, creating models and trying to explain the world. “But I also wanted to change the world. In engineering, you actually create something that can be of use to people,” explains the 41-year-old engineer. During her studies, Sandra Hirche completed a nine-month work placement at Boeing in Seattle (USA). At the end of her time in the States, she realized that although it was a “fantastic experience”, her future did not lie in industrial research for military applications. She preferred the academic world. After completing her Ph.D. in 2005, she went to the Tokyo Institute of Technology. The control engineer with a passion for travel relished the cultural challenges of living in a country like Japan and the opportunity to expand her knowledge in the field of robotics. During this time, she was given a lot of freedom to conduct her own research and also had the opportunity to discuss actual technical problems with interesting researchers. On her return, she was appointed Associate Professor at TUM in 2008. She has been Chair of Information-Oriented Control in TUM’s Department of Electrical and Computer Engineering since 2013.

►What does the ERC grant mean for you personally?

Prof. Hirche Photo: © Astrid Eckert / TU Muenchen

Prof. Hirche
Photo: © Astrid Eckert / TU Muenchen

Receiving an ERC grant is like being awarded a highly prestigious prize. It will certainly help my work by enabling me to engage in high-risk research over a timeframe stretching from ten to twenty years.

►What do you mean by high-risk research?

By that, I mean heading into uncharted territory to explore areas that are going to be important for society in the future.

►What are your core areas of research?

We are interested in developing intelligent controls for robots that assist humans. There are many areas of application here, ranging from age-appropriate production environments to smart assistive robots and mobility aids to robot-based rehabilitation training. This also includes robots in the construction and logistics sectors, where heavy loads have to be carried.

►How did you get interested in this area of research?

I came across the subject of haptic human-robot interaction, in particular telerobotics, during my dissertation. I was especially fascinated by the combination of humans and machines and how this interplay can be expressed in mathematical models. This is relevant in every situation where humans come into contact with robots, for example on a production line or through a robotic prosthetic limb.

►What is the main focus of your ERC grant project?

The aim of our project “Control based on Human Models” (CON-HUMO) is to develop intelligent control mechanisms for collaborative robots that respond adaptively to humans and interact with them intuitively. The difficulty here lies in the complex nature of human behavior. We have to create anticipatory controls so that the robots can predict how a human is going to behave and react accordingly.

►How is this different to conventional control engineering?

In conventional control engineering, for example in the case of a factory robot that does not interact directly with humans, we have a physical process: The robot lifts an object from point A to point B. The model of the robot dynamics can be described with mathematical equations, including how the joints and links are connected, how the mass is distributed and what frictional forces apply. This model forms the basis of the control blueprint, defining the motor current required to move the robot arm from A to B. For this to work, we use sensors located for example in the robot’s joints to ensure that the robot can respond to changes in its environment. Mathematical models are key to developing these kinds of controls. Now, we want to do the same but in the context of humans. In other words, we want to map human behavior in mathematical models. That is also what makes this project so challenging.

►Where do you see these challenges?

Computer science provides machine learning methods that can be used to map human behavior in statistical models based on probability theory. This happens automatically based on observations of humans. In control engineering, however, we don’t have any suitable methods that work with these kinds of data driven, statistical models and which we can use to develop corresponding controls offering the necessary guarantees.

►What kind of guarantees do you mean?

We want to guarantee that a robot will not make any unexpected movements, especially when it is in contact with a human. We have to be able to guarantee that a robot can interact safely with people. But we also have to guarantee that it can act intuitively with humans – and this is something that we still don’t know how to formalize. Not only do robots need to act safely, their movements also have to be as natural and understandable for their human counterpart as possible. We have to prove that the robots will act in the correct way. This is the core of our project.

►In other words, you need a design for an intelligent human-machine infrastructure so that both parties can interact without the robot harming the human or unintentionally destroying its environment?

Exactly. To ensure this kind of safety, we need to build suitable control mechanisms. So if you wanted to see the actual physical results of what we are building you would have to look inside the software, where the control technology is embedded. We do have a large variety of robots here at the Chair, but we develop the controls behind them, not necessarily the mechanics. We need to develop predictive models and implement these in the robot’s software. The robot registers the movements a person makes and the forces they exert, for example via cameras and force sensors. Based on this information it can then use these predictive models to compute the requisite motor currents.

►Could one say that you are looking to program a robot with everything that humans have learned to do intuitively based on years of experience?

Yes, you could say so. We have various options for doing this. Psychology offers models that describe simple human movements. However, we also want to predict more complex movement sequences. Because there is a wide spectrum of human behavior, we would ideally like robots to adapt to the actual person they are working with. In other words, the robot would be given a basic structural model. Through its own observations and learning, it would then continually improve the model and with it the predictions on this basic framework. In order to achieve this, we are collaborating with research groups specialized in the field of machine learning. Our part is to develop the controls that can work with these statistical, continuously improving models to compute the appropriate robot behavior.

►Is it at all possible to predict complex human behavior when a robot is in a complex environment with constantly changing parameters?

Ultimately, this is what we want to achieve. But we start in a structured environment, where nothing unexpected happens. There are predefined patterns, for example, for transporting an object from A to B. In such a well-defined factory environment, the variance in movement is comparatively small. But this all changes in unstructured environments, especially when we start planning over longer timelines and also incorporate more complex human actions. Under these conditions, the number of possibilities increases exponentially. In our research we are starting with highly structured interaction. For example, if I tell the robot “give me a tool”, it knows where we are and it also knows that there is a limited number of tools. The robot takes a tool and only has to predict how the human will move and where the handover point will be. Now, however, we are trying to move further away from this highly structured interaction to increasingly unstructured environments. We still have a long way to go but that is exactly what makes our research so fascinating.

Infographics: ediundsepp

Infographics: ediundsepp

►What methods will you be applying?

We are researching analysis and design methods for controls based on data-driven statistical models. For linear, time-invariant dynamical systems, we have an established toolbox of analysis and control synthesis methods. At this point, I should say that in control engineering, what we call a system is actually an abstract description, in other words a mathematical model of a real-life process. This means that every model is an approximation of reality. What we want to do now is to embed human behavior in statistical models. This is why, for instance, we are using a mathematical class of models known as Gaussian processes (GPs) to capture human behavior. What has been missing so far are suitable systematic control design methods that deliver a control mechanism, based on GP models. As an example, the control mechanism will enable the robot to jointly move an object together with the human, with the robot following the movement intention of the human.

►Can you briefly explain what a Gaussian process is?

A Gaussian process is a stochastic process that describes the Gaussian distributions of a continuum of random variables or functions. It can be used to mathematically describe random processes in time series. In machine learning, Gaussian processes (GPs) are used to provide a mechanism for inter- and extrapolation between input and output data. Starting with training data, the process uses each new input to predict the most probable output. GPs have the advantage of being able to approximate non-linear behavior while at the same time making a prediction on the degree of uncertainty. This makes them a very elegant solution. Using GPs, we can not only predict how a system will move in the next few seconds, but also say how certain we are about our prediction. It is this aspect that makes the methods we are developing so special. We want to specifically incorporate a degree of uncertainty about a prediction into the control – after all I can never be 100 percent certain about how a person will behave in the next few seconds. At the end of the day, we want to evaluate the certainty of predictions and give the robot a suitable level of decision-making power.

Interview by Evdoxía Tsakiridou, Faszination Forschung Heft 17 / Herbst 2015


A new miniature robot developed by EPFL researchers can swim with fish, learn how they communicate with each other and…
Robots vs Jobs

The looming confrontation
Technologist Issue 13

A universal basic income would mitigate the negative effects of automation. But it might be more effective if combined with…
Robert West, researcher at EPFLRobert West, researcher at EPFL

A Swiss researcher has created a system that scans Wikipedia for important articles that are missing in other languages. This project…