Listening for Ads

Research firm IMMI tracks ad exposure by sampling sound through cell phones.

Nielsen Media Research, well known for measuring TV ratings, has long used special technology in volunteer homes to track TV-viewing habits. But the systems can't take into account the ads that volunteers might encounter outside the home. Now IMMI, a research company based in San Mateo, CA, is using specially designed software on cell phones to study which ads work best.

The technology can also tell advertisers things such as which songs tend to make people change the channel on the radio. Recently, IMMI found that the most effective ads for the Simpsons movie were those that also promoted Burger King.

CEO Tom Zito says that IMMI measures people's exposure to ads from a variety of sources, including radio, TV, movie theaters, and the Internet--something that he says has never been done before.

IMMI CTO Al Alcorn says that using cell phones was an important part of the design. "The question is, 'If you forgot to take [the tracking device] with you when you went to work, would you go back and get it?' If it was your cell phone, you would. If it was just a passive brick that didn't do anything [else], you wouldn't."

Panelists volunteer to be monitored for two years, and they agree to carry a cell phone provided and paid for by IMMI. With information gleaned from the cell phone, IMMI tracks all the audio ads the volunteers encounter. It then tracks how the volunteer responds, listening to determine if the customer goes to an advertised movie or buys an advertised CD, or monitoring the volunteer's position to see if she enters an advertiser's store.

The cell phones work by taking 10-second audio samples of sound around the volunteer every 30 seconds. Acoustic-matching software on the cell phone analyzes the sound waves and creates a "fingerprint" of the sound, Alcorn says. This process reduces a 150-kilobyte sound file to a 1.5-kilobyte fingerprint. Thanks to Moore's Law, he says, modern cell phones are capable of doing the processing unaided. Every 10 or 15 minutes, the phone sends the fingerprints to IMMI's servers in San Jose, where they are compared with known fingerprints of television shows, music, and advertisements. Because the hashed files are so small, Alcorn says, they're easily stored on the phone, so if the volunteer loses cell-phone service temporarily, the files can be sent once service is restored.

Volunteers also plug a Bluetooth beacon into a home outlet. The beacon communicates with the phone to track whether the user is inside or outside the home. Alcorn says the beacons can also be placed in clients' stores to track volunteers who come in. Although he would like to use GPS, Alcorn says that it's currently cost prohibitive to buy this information from the cell-phone companies, which control it.

Although a system like this would seem to raise privacy concerns, Yan Ke, a Carnegie Mellon researcher who worked on developing similar technology for a project at Intel, says that acoustic matching protects privacy because the hashed files can't be reverse-engineered and give no information beyond what is already in the known fingerprint. "It just so happens that the efficient method also preserves privacy," Ke says.

Amanda Welsh, IMMI's COO, agrees: "If you're carrying our phone and you're planning a bank robbery while listening to a radio, all we know is what radio station you're listening to."

Welsh says that IMMI stores personal information about volunteers separately from data collected through the cell phones, and she takes care to ensure that volunteers understand what data is being collected. "There's a small subset of people that will not do this, no matter what we say, but it's a very small subset," she says.

While Google has been researching similar technology, that technology does not seem to be mobile and is not yet in use. (See "Googling Your TV.")

Although IMMI's system works well for ambient audio, it does not yet measure things that volunteers hear through headphones or ads that they see on the Internet. Alcorn says it would be possible to run sound from a device such as an iPod through the phone, but this isn't something currently being done. Welsh says the company's next step is to improve its ability to monitor Internet advertising and media exposure. (Technology Review, 9.05.2007, Erica Naone)

0homefly.gif (8947 bytes)