Do you like the thick brush strokes and soft color palettes of Claude Monet’s impressionist paintings? Or do you prefer Rothko’s bold colors and abstract shapes? Individual art tastes are mysterious, but a new Caltech study demonstrates that a simple computer program can accurately predict which paintings a person will like.
The new study, published in the journal Nature Human Behaviour, enlisted over 1,500 volunteers to rate paintings in the genres of impressionism, cubism, abstract, and color field via Amazon’s crowdsourcing platform Mechanical Turk. The answers of the volunteers were fed into a computer program, and after this training period, the computer could predict the volunteers’ art preferences much better than chance would.
“I used to believe that evaluating art was personal and subjective, so I was surprised by this result,” says lead author Kiyohito Iigaya, a postdoctoral scholar in the laboratory of Caltech psychology professor John O’Doherty.
The findings not only demonstrated that computers can make these predictions, but also provided new insight into how people judge art. “The main point is that we’re getting a better understanding of the mechanism that people use to make aesthetic judgments,” O’Doherty says. “That is, people appear to use basic image features and combine them. That’s the first step in figuring out how the process works.”
A new study showing that computers can predict what paintings people will like offers insight into how our brains make aesthetic judgments.
In the study, the researchers programmed the computer to separate a painting’s visual attributes into low-level features (such as contrast, saturation, and hue) and high-level features (which require human judgment and include characteristics such as whether the painting is dynamic or still).
“The computer program then estimates how much a specific feature is taken into account when deciding how much to like a specific piece of art,” Iigaya explains. “When making these decisions, both low- and high-level features are combined. Once that is estimated, the computer can successfully predict a person’s liking for another previously unseen work of art.”
The volunteers also tended to cluster into three general categories, according to the researchers: those who like paintings with real-life objects, such as an impressionist painting; those who like colorful abstract paintings, such as a Rothko; and those who like complex paintings, such as Picasso’s cubist portraits. The majority of people were classified as “real-life objects” in the first category. “A lot of people liked the impressionism paintings,” Iigaya says.
Furthermore, the researchers discovered that they could train a deep convolutional neural network (DCNN) to predict the volunteer’s art preferences with a similar level of accuracy. A DCNN is a type of machine-learning program in which a computer is fed a series of training images in order for it to learn to classify objects such as cats and dogs. These neural networks are made up of units that are linked together in the same way that neurons in the brain are. The network can “learn” by varying the strength of one unit’s connection to another.
The deep-learning approach in this case did not include any of the low- or high-level visual features used in the first part of the study, so the computer had to “decide” which features to analyze on its own.
“In deep-neural-network models, we don’t actually know how the network is solving a particular task because the models learn on their own, much like real brains,” Iigaya explains. “It can be very mysterious, but when we looked inside the neural network, we could see that it was constructing the same feature categories that we had chosen.”
These findings imply that features used to determine aesthetic preference may emerge naturally in a brain-like architecture. “We are now actively investigating whether this is the case by examining people’s brains while they make these same types of decisions,” O’Doherty says.
In another section of the study, the researchers demonstrated that their simple computer program, which had previously been trained on art preferences, could accurately predict which photos volunteers would prefer. They showed volunteers photographs of swimming pools, food, and other scenes, and the results were similar to those obtained with paintings. The researchers also demonstrated that reversing the order worked: after first training volunteers on photos, they were able to use the program to accurately predict the subjects’ art preferences.
While the computer program was successful in predicting the volunteers’ art preferences, the researchers believe there is still much more to learn about the nuances that go into any one person’s taste.
“There are aspects of preferences that are unique to a given individual that we have not been able to explain using this method,” O’Doherty says. “This more idiosyncratic component may be related to semantic features, such as the meaning of a painting, past experiences, or other individual personal traits that may influence valuation. It may still be possible to identify and learn about those features in a computer model, but doing so will require a more detailed study of each individual’s preferences, which may not generalize across individuals as we discovered here.”
