The view from an infrared eye-tracking camera.

June 18, 2012

Assistant Professor of Psychology Michael E. Roberts was reading a review article about eye tracking and its application in everything from marketing to computer science, considering ways to use the technology in his own cognitive psychology research. One part stood out. People become habituated to certain types of features in pictures, the article said, and that may cause the pictures to be less memorable over time.

“I thought, that would be pretty easily testable with an eye tracker,” Roberts says.

With the help of Professor of Psychology Terri L. Bonebright, Roberts developed the idea into a research project that’s giving some participants the most rigorous education in art they’ve ever received.  Sitting in front of a computer screen, a participant is asked to learn a series of paintings and the names of the artists who painted them. The original paintings are then mixed into a larger set to test which ones he or she remembers. If nothing else, it’s a pleasant cultural experience.

“It’s not a terrible thing to sit down and look at nice paintings for a while,” says Garth E. Synnestvedt ’14, one of two Science Research Fellows working on the project this summer.

Art appreciation, however, is not the goal of the study. While participants are viewing the paintings, an infrared, eye-tracking camera follows their eye movements as if tracing over the image with an invisible pencil. Where people tend to look isn’t surprising – faces and bright areas attract the most attention – but the eye tracker reveals much more detail than that.

Unseen by the wearer, eye-tracker recordings reveal precise visual navigation through images.

Looking at a picture or a face, or even this little letter e, you might think your eyes are staring straight on, fixed determinedly on a point because you told them to be. They aren’t. Every quarter second, your eyes make a tiny jump called a saccade (suh-KAHD) based on an internal, subconscious urge to focus on something new. Because these tiny movements are unintentional, participants are typically unaware of how they’re really looking at the paintings.

“One of the things that’s interesting to me is the difference between what people are doing consciously and subconsciously,” Haley A. Pratt ’15 says. “When we ask them what strategy they used to remember a painting, what they tell us and what the eye-tracker videos show don’t always match.”

“That’s one of the more interesting aspects of it, this separation of what can you know as the person looking at the painting, and what you can know as the researcher looking at the video, frame-by-frame, watching where your eyes go,” Synnestvedt says.

A fixation model predicts where participants will look in a painting, and in what order.

The saccades will tell the researchers how our brains prioritize features of an image, and whether that affects our ability to remember it. And what if nearly imperceptible changes to those features could make the image more memorable?  Advertisers would love that.

While working on their research, Pratt and Synnestvedt sometimes hear strange noises coming from another part of Harrison Hall’s basement. At first, they weren’t sure what they were hearing. A person? A computer? They weren’t alone.

Around the corner from the eye-tracking lab, Patrick S. Ledwidge ’13 has spent the better part of his summer organizing a library of thousands of recordings, a mix of professors and students, actors and non-actors giving voice to 20 different emotions. Some of the emotions are basic, such as sadness and fear, while others such as pride, gratitude, anxiety and desire are more complex. Collectively known as affect bursts, these grunts, whimpers and sighs can communicate all another person needs to know about your state of mind without speaking a single word.

"Something interesting about vocal affect bursts is that there’s no linguistic processing,” says Ledwidge, who’s working on the research project with Bonebright this summer. “Because of that, there are theories that they’re evolutionarily adapted, so they may be recognized without any language being necessary and, potentially, cross-culturally between people who don’t share a language.”

In the study, participants listen to a series of affect bursts and indicate which emotion they think they’ve just heard after each. Bonebright and Ledwidge hope to use the data they collect to uncover all they can about the encoding and decoding of these sounds. Are actors really better at conveying emotion? Are males or females better at identifying these bursts? The answers could confirm certain stereotypes and inform future studies on the topic.

For now, the only participants for both projects are other students who remain on campus for summer research. Fortunately, that pool is large enough to get things underway. Some, like Pratt and Synnestvedt, are completing their SRF on-campus research requirement, while others received summer funding through programs such as the psychology department’s J. William and Katherine C. Asher Endowed Research Fund, which provided for Ledwidge’s position.

“Originally, I was worried about being on campus for the summer, but Professor Bonebright has given me a lot of independence working on this project,” Ledwidge says. “I can just go with the flow and keep moving at my own pace because there aren’t other things going on during the workday that draw me away from the project. I guess it really confirmed that research is something I want to do in graduate school and in my career.”

“What’s nice about summer research is that faculty and students can collaborate on projects we both have an interest in, and that help students, not only in terms of understanding what happens in their major as a discipline, but also if they’re interested in graduate school,” Bonebright says. “The other great thing about it is that the students learn all sorts of skills that are good no matter what they end up doing.

“It is work, but it’s good work, and it’s really fun,” Bonebright adds. “When you have a student who is really energetic and engaged like these students are, then it’s really a great experience for everybody.”