hy is one person content and another one grumpy? The answer is as elusive as it is intriguing. Even defining happiness is difficult, says Pearl Pu of Switzerland’s École Polytechnique Fédérale de Lausanne (EPFL). “Is it an emotion, an attitude, a mood? What makes us happy today may result in unhappiness tomorrow.”
Traditionally, happiness studies have looked only at correlations with subjective measures of life satisfaction. They have unearthed positive links to sleep, exercise, social interaction and having a sense of purpose. Today researchers like Pu are trying to unpack what affects emotion on a finer scale, moment-to-moment. For this they rely on new technology and citizen science to track happiness on a far larger scale than any laboratory study would allow.
One method is to combine hard data with what people say. Such apps as Mappiness (from the London School of Economics) and Track Your Happiness (from Harvard University) use data picked up from a phone’s GPS to tag results with location and weather data. But Pu is on a quest to understand even more factors that influence well-being.
Digital lifestyle coach
Participants in her experiments wear sensors that record their physical activities and sleep patterns. The researchers then analyse the sensor data and develop algorithms to identify behaviour, looking for differences between groups of users and also between a person’s current and past behaviour. By identifying healthy routines and habits, the hope is to develop a “lifestyle recommender” that can provide accurate, personalised suggestions on daily activities that would make someone healthier and happier.
Many people are willing to wear sensors, but gathering large amounts of data requires cheap devices that have no contact with a person, says Daniël Lakens, a cognitive psychologist at the Eindhoven University of Technology. Smartphone cameras, for example, can be used to study heartbeats. “Although you can’t see it, with every heartbeat all your veins become slightly wider and your skin becomes slightly redder,” he says. A computer filters and amplifies the signal to detect the rhythm.
In such experiments, a camera was able to measure heart-rate differences associated with different emotional states. It is not a perfect science, says Lakens, in part because some emotions – such as anger and extreme happiness – can look similar physiologically. To differentiate further, heart-rate data might be combined with such other information as breathing rates. Lakens says he plans to work with a clinical psychiatrist to develop a system that gets patients to use their own smartphone cameras to tell them when they are stressed – which they often do not realise. This could help them train themselves to avoid stress.
The face reader
Another way for a smartphone to read emotions is by studying faces. In the 1970s Paul Ekman, a psychologist
at the University of California, San Francisco, devised the Facial Action Coding System to classify facial movements. Using this system, researchers today translate tiny movements of facial muscles into emotions. These “microexpressions”, which are often imperceptible to the human eye, involve sequences that occur in quick succession and are almost impossible to fake. “As individuals we probably don’t feel this sequence, but we can analyse it with a video camera,” says Jean-Pierre Thiran, professor of signal processing at EPFL.
Software developed by Thiran and his team can decode emotions in real environments by tracking microexpressions and eye movements, even in low light or with moving faces. Machine-learning techniques allow the software to adjust for differences between individuals. Thiran is now working with Peugeot-Citroen to install software in cars that can detect a driver’s emotional state. This might detect when a driver is getting sleepy, but it could also look for stress and perhaps adapt the ambience of the car accordingly.
Meanwhile nViso, founded by Thiran’s former PhD student Matteo Sorci, is among a number of start-ups that use facial technology to help market researchers test products with a webcam. Unlike questionnaires, microexpressions cannot lie.
“We take thin slices of behaviour and try to figure out how we feel in a certain context,” says Lakens, “but we never measure huge amounts of data because it’s very costly to follow people for days and weeks.” In the long run, smartphones could well be used as constant behavioural labs that provide researchers with even more information. A Google Glass system, which gathers data on what people are seeing, feeling and doing, could gather unprecedented amounts of data on what affects happiness.
Winning is not always best
Sensors and cameras can help reveal emotions that self-reporting might hide, but scientists are also using smartphones to test general theories about happiness on a wider scale. Robb Rutledge, a neuroscientist at University College London, developed an app called The Great Brain Experiment to better understand subjective feelings.
In his experiment, a small sample of volunteers had to play a game in which their choices between guaranteed cash rewards and risky gambles led to wins and losses. Using reports of how happy they felt at each stage, Rutledge came up with a model that described how a person would feel at any point, based on their winnings and experiences in the game. He found that happiness was not just about success or positive events; it reflected moment-to-moment expectations based on the difference between recent experiences and anticipated rewards. “You could be doing just fine. But if your expectation is high, you might not get any happier when you win,” he says. He tested his model on 18,000 app users, showing that it could accurately predict a player’s happiness.
Rutledge hopes this more complex, quantifiable understanding of happiness will help doctors better understand such mood disorders as depression. “Often patients will have reduced pleasure from things that people usually find pleasurable. We’re hoping we can pin down some of what’s different, for example in their reaction to reward and expectations and give an idea of what treatments might
be effective,” he says.
The collection of data on such a large scale carries major responsibility. Who uses the information, and for what purpose, is a real concern. “The trade-off between the protection of end-user privacy versus the benefits these systems can bring to users is really important,” says EPFL’s Pu. She works with experts to protect privacy – for example by scrambling data so that only certain information can be extracted, a process known as obfuscation. The law may need to catch up, too, says Lakens, as there is no real legal difference now between just looking at a video of someone and extracting additional information, such as heart rate.
Engineered content
Daniel Quercia, a computer scientist at the Yahoo Labs based in Cambridge, UK, is harnessing citizen scientists to understand how cities affect emotions. Using a mobile phone app, he asked users which of a random set of two images they considered happier, more beautiful or quieter. Using an algorithm to analyse the pictures, he found which visual cues correlated with preferences. The key to urban happiness seemed to be whatever promoted social interaction. “Green areas, small houses, small streets all made people happy. Negative elements were isolated buildings and moving cars,” he says. Quercia hopes to use the research to build a dictionary of happy elements, accessible to urban designers to help them retrofit cities for happiness. “As a research community, we are increasingly losing the buzz word of ‘smart cities’ in favour or the concept of ‘happy cities’.”
These new insights have already been incorporated in apps that have sprung up to help people live better (see Happy apps, p. 28). Although the changes they inspire may seem small, Paul Dolan, a behavioural scientist at the London School of Economics and author of Happy by Design (2014), says that when it comes to happiness, a nudge in the right direction is powerful. “Listening to French music can make you buy French wine. Citrus smells will inspire you to clean.” Happiness, he says, is no exception: “If we can just become aware of what situations make us happy, we can step back and create environments to make it easier to do what makes us happy.”