The next ‘American Idol’? Ask your computer

Rising above all the noise
As for the Israeli research trio, no major record labels have yet come courting them for their business acumen in identifying unsigned artists, though Shavitt said they have been in talks with a U.S.-based company that deals with music. In contrast to their success rate of up to 30 percent, the study suggests that the probability of correctly spotting successful new artists without the algorithm would be less than 0.1 percent.

Instead of relying on the actual number of Gnutella requests, the formula tracks how well a band’s queries rank relative to other groups.

“You have to rise to a certain level in the city where you are — in the top 500 or 1,000 queries — or else it’s just noise,” Shavitt said. To attract some attention, at least from the computer, he figures a band would need to garner at least 100 to 150 queries per day.

So far, the database’s gigantic size has prevented more nuanced algorithms that might increase the sensitivity even more. After eliminating queries that originated outside the U.S., that could not be geographically pinpointed from their origin IP (Internet provider) addresses, that were blocked by firewalls or were otherwise ambiguous, the study still processed more than 300 million queries.

Recently, the team began tracking the Direct Connect network, which handles up to 1.5 million queries per day. Although only a fraction of Gnutella’s size, the smaller system may be amenable to more involved algorithms. The research suggests the same analysis could be applicable to other file sharing networks like BitTorrent and eDonkey or to Web sites like YouTube and MySpace.

Discovering a star
No matter what the database, the ability to pick a winner comes back to an idea attributed to China’s Mao Tse-tung: that revolution is like ink stains on paper. The stains begin small but eventually grow and merge into one large stain. Of course, breakthrough artists might chafe at the notion they’ve become a national stain, but the idea that successful bands begin as little more than droplets of local interest before eventually flooding the airwaves has been borne out repeatedly.

The reverse is true for contestants on “American Idol,” who effectively debut in front of a national audience on their way to stardom or infamy. Even so, Shavitt said, a similar algorithm can tap Internet queries for clues to a contender’s prospects.

“You can see if a certain competitor is gaining momentum,” he said. “It’s slightly different from what we are doing, but it can be done.”

Nancy Baym, an associate professor of communication studies at the University of Kansas who studies online music networks, said Shavitt’s study fits well “into a whole of the ways people are trying to mine all the music-related information that’s out there right now.”

And despite the public animosity between record labels and file-sharing networks, she said, many companies have privately recognized the value of such databases and are paying others to mine the information for them.

The real power of popular peer-to-peer networks like Gnutella, as suggested by Shavitt’s study, is that they not only predict trends, but also cause them, Baym said.

“My big-picture thought on all of this is that the labels would do well to release their material to file-sharing networks because it’s a publicity mechanism and ultimately rebounds to them in sales, although many of them vehemently dispute that,” she said.

Making sense of the growing mounds of data will remain a challenge, but Baym said the recent successes in spotting musical trends portend a race to figure out who can be the best predictor — with big money at stake.

For their ongoing research, Shavitt and his collaborators are testing out a new algorithm to see if they can predict how long a song will stay on the charts. And a separate project is examining cultural differences among countries based on music database searches.

“We talk about the world becoming a smaller village,” he said. “When you look at the tastes of people, there are still huge differences between nations.”

Shavitt can personally attest to that. Despite his research, he’s not liable to rush off to a hip-hop concert anytime soon.

“I’m a fan of the data, not the music,” he said, laughing. Well, maybe the Shop Boyz are OK. “I can actually listen to them for two minutes,” he said.

MORE FROM FRONTIERS

Tourist or terrorist? Computer knows

Tourist or terrorist? Computer knows Climbing no obstacle for snakebots The next 'American Idol'? Ask your computer How a deadly avalanche can be triggered Green alternative to plastics: liquid wood Getting lost for better architecture Turning air into drinking water Revving up the race for fuel efficiency How to be a disagreeable (but likable) robot Can concept of clean coal be salvaged?   Add Frontiers headlines to your news reader: