Teaching a computer to appreciate art

Bryn Nelson Columnist

Is that a van Gogh?

A mathematical program that began as a lark for an Israeli scientist has become a serious effort to match some of the world’s greatest painters with their masterpieces. If the project pans out, it could help point out poor copies and eventually distinguish forgeries from the real deal.

Daniel Keren, a professor in the Department of Computer Science at the University of Haifa, said he’s been contacted by an Italian collector hoping to validate some of his acquired paintings as well as by aficionados embroiled in a controversy over the legitimacy of artworks allegedly by Dutch master Vincent van Gogh.

“I did it for fun, but now people are interested in it, so I will definitely expand,” Keren said.

Research in the rapidly growing field of computer vision, he said, still has plenty of catching up to do if scientists want computers to approximate our own abilities. One stumbling block has been teaching machines how to spot objects that are simple for people to recognize — another human face, for example.

Art as a mathematical formula
For his project, Keren tackled the problem by essentially breaking visually stunning masterpieces into sets of mathematical  formulas. The computer program sought to capture the distinctive styles of different artists by dividing their paintings into discrete blocks and then converting each block into formulas that could be added together and compared.

“Suppose that one painter, he has very many vertical structures,” Keren said. Perhaps the painter favors depicting telephone poles, say, or skyscrapers. Converting blocks from that painting into mathematical symbols similar to the sine and cosine waves familiar to any trigonometry student will yield a distinctive sum of the parts. If another artist paints primarily with horizontal lines — perhaps in the form of logs floating down a river — “in that case, it’s very easy to detect who is painter A and who is painter B.” If a painting includes examples of both styles, the program can color-code each element accordingly to help decide if the whole piece is more A-like or B-like.

So far, Keren and his team have applied the test to five artists, including van Gogh and Rembrandt, surrealists Salvador Dalí and René Magritte, and Russian abstract painter Wassily Kandinsky.

Altogether, Keren’s group used about 30 artworks from each of the five painters, half for the training sessions and half for testing their mathematical model. In all, the model correctly matched 86 percent of paintings it hadn’t previously “seen,” a solid B in most grading schemes. (If the program had been assigning the paintings randomly, it would have received a score of only 20 percent.)

Finding forgeries
The current incarnation might be of use to an art novice, though hardly helpful to an expert, Keren acknowledges.  “I am sure it can be improved,” he said. A key to its continued development will be determining exactly how two paintings differ. If the subject matter is dissimilar but the style is the same, the computer likely will be able to identify the right artist, based on its past learning of core elements such as van Gogh’s characteristic use of swirls or Magritte’s preference for straight lines.

A sudden switch in painting techniques by the same artist, on the other hand, could present a far greater challenge, as would trying to distinguish painters with very similar brushstrokes, like some of the 19th century Impressionists.

Keren said he plans to significantly expand his project to include far more artists, including ones who have adopted similar styles. As for trying to identify potential look-alikes, he said his program could begin by classifying paintings according to a general group — Impressionism versus Surrealism, for instance — and then sort within each group according to increasingly fine-tuned physical traits.

Keren is "cautiously optimistic" that his mathematical program might eventually be useful in detecting fakes. "It will be good to have a database of 20 van Gogh forgeries," he said, allowing the program's formulas to zero in on subtle, but perhaps telling, differences.

MORE FROM SCIENCE

Science Section Front

Thai zoo's 1st baby panda goes on display Nature’s fireworks on display ‘Toy universe’ may solve mystery of life’s origins Mystery solved about dangerous ‘blind’ faults Shrinking sheep mystery solved: Blame warming Study: Tropical rain band shifting north Newsweek: Are the chemicals in fireworks dangerous? Cosmic Log: Nano-wizard takes the prize Ancient mosaic comes out of hiding in Israel Volcano’s eruptions create colorful sunsets Science Section Front   Add Science headlines to your news reader: