Hotmail
Messenger
My MSN
MSN Directory
Software turns photos from bad to good
Program searches Flickr to make imperfect photos into great pictures
 | Interactive map |
If a picture’s worth a thousand words, what can you say with two million snapshots?
With some help from the Flickr photo-sharing Web site, two researchers at Pittsburgh’s Carnegie Mellon University have shown how a new picture-patching program can transform flawed vacation shots into “Wow!”-worthy masterpieces.
Think of it as a bit of picture-perfect revisionist history for the digital age. Or how those snapshot souvenirs from Europe might have looked if that lovely bay hadn’t been blocked by a roof or the charming plaza hadn’t been marred by your two-timing ex.
Unlike existing programs that use bits of the same photo to patch holes, the new program relies on an algorithm that first searches through heaps of digital photos — 2.3 million downloaded from Flickr in this case — for ones that match the gist of the scene. The sophisticated formula tries to match general properties such as shapes, textures and orientations to pick out all photos with, say, a similarly curving bay or a river running through a city.
The program then looks within that subset for good patches by blending candidates with the target photo and finding boundaries that would be least noticeable to viewers. In the finished product, that view-blocking roof might be replaced with sailboats or your ex-boyfriend supplanted with some greenery.
Although attempting to gather all possible images of the world would be pointless, the researchers say their study suggests it might be feasible to collect enough representative scenes so that any input image would yield a “similar enough” photo to produce a realistic composite.
Graduate student James Hays and assistant robotics and computer science professor Alexei Efros are careful to point out that their digital patch is not intended to accurately restore all the information that should have been there but to fill in the missing pixels with images that could have been there. Or at least look like they belong.
Efros said getting a computer to produce a composite scene that is not only seamless but also contextually valid reflects a main challenge of artificial intelligence research.
“If you have a cow standing in a field and you erase the cow, it’s not hard to use the green grass in the field to fill in the hole,” Efros said. But removing a car parked in an alleyway and convincingly filling in the image, he said, is a far bigger hurdle.
“The computer needs to realize, ‘Ah, the car is gone but what’s usually underneath the car is a road, so let’s fill it in,’” Efros said.
Humans are particularly good at using visual cues for such problem-solving, he said, but the field of artificial intelligence is “nowhere near” a solution that allows computers to likewise interpret new scenes as an alleyway or beach or airport check-in counter.
One of the new study’s chief insights is that the problem might be effectively bypassed with a big enough database. With 10,000 images, Hays and Efros found that their program’s closest matches weren’t all that similar to the target photos. With more than two million images, though, they found convincing patches for a significant percentage of their incomplete scenes. For future research, they hope to amass 10 million pictures.
More innovation news |
In a way, Efros said, “we’re cheating the artificial intelligence problem by doing this Google-style lookup.”
Perhaps unsurprisingly, Google has expressed early interest in the approach. The study, presented by Hays earlier this month in San Diego at the Association for Computer Machinery’s International Conference on Computer Graphics and Interactive Techniques, also suggests a method for how the technology and future iterations might be evaluated.
| Rate this story | Low | High |
MORE FROM FRONTIERS |
 Are we closer to a 'Matrix'-style world? |
| Add Frontiers headlines to your news reader: |
Sponsored links
Resource guide