« August 2007 | Main | October 2007 »
With New York PPM going live in a little more than three months, Arbitron announced that the New York panel is fully recruited. In fact, it stands at 6% over Arbitron's target of 5,170 in the DMA. The numbers appear better than they actually are, since Arbitron is still short--significantly short in key demographics. In the 18-34 demo, for example, Arbitron's target is 931 panelists, but it only has 567.
Many years ago, I was conducting Focus Groups for a radio station, and the discussion turned to music. In passing, one participant mentioned that he was tired of a particular song. Before the discussion had ended, the Program Director had called the control room hot line and instructed the personality on the air to pull the song. Later that evening we were conducting a second discussion with another group of listeners when one participant mentioned that he loved the song that the PD had just pulled. The PD immediately called the hot line and put the song back in.
I was reminded of this knee jerk reaction and misuse of research as I read Arbitron's latest paper attempting to show that PPM can measure listener reactions to music. This is the latest in a series of papers beginning with a presentation by Bob Michaels in 2004, Portable People Meter Programming Insights. Read the PDF here. In the presentation, Mr. Michaels claimed that PPM data from Philadelphia was accurate and detailed enough to prove that Led Zeppelin's Stairway to Heaven was a bigger tune out than CCR's Proud Mary, that Gloria Estefan's version of Have Yourself a Merry Little Christmas was a bigger tune out than the Carpenter's version, and that Howard Stern's listeners didn't like commercials.
Then in 2006, Arbitron produced a study that showed that listeners sit through four, five, even six minutes of commercials. Read the PDF here.The latest study,What Happens When New Music Gets Played: The Impact of New Songs on the Country Radio Audience is described this way on Arbitron's web site:
(The study) reveals how the exposure of new songs on KILT-FM did not undermine the station’s audience levels. The study reveals how, on average, the Country audience grows 1.8% with the exposure of a new song. Among the other key findings are that not all new Country music is created equal when it comes to its impact on audience levels. Established artists perform better than new artists, uptempo songs perform better than slower titles and songs from male artists perform better those from female artists. The study also reveals that it takes about 25 weeks for a new song to have its maximum impact on a Country station’s audience levels. Read the PDF here.
Looking at Arbitron's growing number of papers on what PPM can do, one would think that PPM will tell us what songs to play and how long to play them, the popularity of our personalities, and even what spots turn listeners off. Once PPM is in your market, you will know everything you need to know to create the perfect radio station. The truth, however, is quite the contrary. The documents on Arbitron's web site look impressively authoritative, but are more PR than research. In a perfect world, the PPM could quite possibly prove correct all the assertions about radio tune-outs that Arbitron makes, but this isn't a perfect world.
The PPM that is coming to your market will not be the PPM portrayed in these documents. You can take Arbitron's word for it. In a recent Friday Morning Quarterback column, Gary Marince, Arbitron's Vice President of Programming Services, explains why this way:
I have been asked by a handful of programmers about the merits of marrying the minute-by-minute data in the Analysis tool with a music log to determine what songs are causing tuneout. And the question is . . . should you be doing that? No! More emphatically stated, NO! Don’t go there. It sure is tempting isn’t it? “Let’s see, the song started here and shortly afterwards I lost 75% of my meters. I’ll never play <<insert your top testing record name here>> again.”
PPM was developed to create a quarter hour measurement. It was never, as configured today, intended for use as a music tester...But, even if you wanted to push this a little and say: “Yeah, forget the technicalities, people tuned away – doesn’t that matter?” Sure it matters, but can you confidently identify the reason they tune-out or tuned-away? Was it because of lifestyle – they got to work and were now hearing music from a peer’s radio? Were they listening to a station in a friend’s car – and got out? Until you can substantiate why someone tuned away, there’s no basis for assuming they tuned away because they didn’t like the song.
So Mr. Marince is saying that what Arbitron says PPM can do, it won't really, so you shouldn't use it like Arbitron says it can be used. But there's a more important reason that we shouldn't pay attention to minute by minute PPM data. It has to do with the editing of PPM data. In the editing process, time becomes a somewhat subjective and relative thing. For a full explanation, you should read Arbitron's PPM Description of Methodology, but Mr. Marince does a good job of simplifying a confusing subject. He continues:
PPM has very heavily researched and well thought out business and edit roles. And these rules are used to determine who gets credit and how much listening credit they get. It works, kinda’ like this; if a PPM encoded signal is detected, at 30 seconds past the minute mark, edit rules kick in to determine which minute/minutes will get the credit. Bottom line, if you have a song which starts at 10:30:30 . . . and someone starts listening to your station at 10:30:30 . . . but PPM edit / business rules are such that they’ll round the occasion and report that listening started at 10:30:00 . . . you’d be basing your music testing off of phantom listening and make it look like the previous song had a larger audience than it really did.
You don't really have to understand what he said. You just need to realize that the phrase, "minute to minute data" in the PPM world we are entering isn't really minute by minute. So if you can't wait for PPM to get to your market so you can finally really find out what's going on with your listeners, you've got a big surprise coming your way.
Clear Channel's Houston cluster is making PPM driven programming adjustments to their FM stations. According to client e-mails, the stations are adding a minute to non-morning drive spot-sets, moving from three spot sets to two, and limiting station promos to one minute per hour.
While these changes have been covered extensively in the trades (with the most thorough coverage in Radio Business Report), there has been little discussion of one portion of the e-mail. In its memo, Clear Channel stated that, "research with PPM shows that people are listening through commercial breaks," an apparent reference to research (PDF) that Arbitron published last year. It allegedly showed that in Houston listeners do not tune away when the commercials come on.
In its story, Inside Radio pointed out that, "Cox Radio has promoted its two stop-sets and maximum of ten (commercial) minutes an hour. Cox was the only company that didn't see its ratings fall once PPM came to the market." So we have Clear Channel reminding advertisers of Arbitron's research that supposedly shows that listeners will sit through six minutes of commercials while touting changes designed to reduce clutter and help their PPM numbers. If Arbitron's research is correct, then length of spot-sets shouldn't matter. The number of spot-sets shouldn't matter either.
Clear Channel's words seem to support the research, while their actions seem to suggest they don't. The truth is, Arbitron's study was methodologically flawed, and didn't really resolve the question of what listeners really do when commercials come on. The limited Houston PPM data we have doesn't resolve the question, and we can't yet draw any conclusion about whether two spot-sets has helped Cox. Maybe the ratings have nothing to do with spot-sets.
It would have been far more valuable to Clear Channel if they had taken the opportunity to change the spot-sets on some of the stations, and leave the others alone. Then we could have seen whether the changes helped. By changing all the stations at once, Clear Channel has muddied the water and lost the opportunity to determine whether spot-set length and number really do have an impact on PPM results.
In its latest 8-K filing with the Securities and Exchange Commission, Arbitron announced that it expects earnings per share to be between $1.35 and $1.45 for this year. The company expects revenues to increase between 5.5% and 7.5% compared to last year.
As reported by Inside Radio today, Arbitron continues to miss its PPM panel targets in Philadelphia and Houston. In Philadelphia, July was 179 short while August was slightly better falling 143 panelists short. In Houston, July's panel was 193 short while August's was 176 short.
It is difficult to tell what impact these shortages have on the ratings, since the current PPM software doesn't generate margins of error.
When Media Audit/Ipsos announced the development of their Smart Phone device to measure radio listening, Arbitron dismissed their announcement derisively declaring that, "If all you’ve got is a gizmo, you’ve got a long way to go." The line became the title of an article on the Arbitron website by David Lapovsky who writes, "Its not the electronics of a metering device alone, but the whole system that surrounds the metering device that determines the usefulness of the audience estimates it collects."
Mr Lapovsky’s words certainly resonated with the broadcasters of Houston when it was discovered that some two hundred in-tab panelists disappeared one week. An email from Arbitron declared:
Here’s what happened: a file used to process Unified Weekly Cume Estimates was incompletely loaded into our production system, resulting in the exposure records of approximately 200 In-Tab panelists being excluded. As a result, the Unified Weekly In-Tab appeared to be abnormally low and the error affected the calculation of Weekly Cume estimates.
There has been no further word from Arbitron on how the data from so many panelists could disappear, why the error wasn't caught before the release, or what steps have been taken to see that this doesn’t happen in the future. Next month New York, Arbitron’s largest market, begins PPM measurement. Let’s hope that Mr. Lapovsky’s article becomes required reading for everyone associated with producing that first New York report.
When Media Audit/Ipsos announced the development of their Smart Phone device to measure radio listening, Arbitron dismissed their announcement derisively declaring that, "If all you’ve got is a gizmo, you’ve got a long way to go." The line became the title of an article on the Arbitron website by David Lapovsky who wrote, "Its not the electronics of a metering device alone, but the whole system that surrounds the metering device that determines the usefulness of the audience estimates it collects." Truer words were never spoken.
The success of Arbitron's PPM, the Media Audit/Ipsos Smart Phone, or some yet undiscovered method will rest on not only "the gizmo," but everything else that surrounds the metering device. InSights was created to examine radio's leap into electronic measurement, developments in this rapidly evolving technology, and its impact on radio.
