One of the early assertions about Arbitron’s PPM was that it would provide more accurate ratings than the diary method it replaced. Like most aspects of PPM, the assertion was accepted without debate. But is PPM really more accurate?
The question is far more difficult to answer than you might think. What does accurate mean? How do we measure accuracy? At first it might seem obvious that PPM is more accurate, but it turns out that depending on how we define accuracy, PPM might be less accurate than the diary method.
First, we have to differentiate between accuracy and precision. The words don’t mean the same thing. Something can be accurate and not precise. Something can be precise and not accurate. Take the patterns on two bull’s eyes shown here. The first bullet pattern is very precise with all of the holes in a tight pattern, but all miss the bull’s eye. The shots are not accurate. The second pattern is more scattered, but on average much more accurate. They are closer to the bull’s eye.
We know that PPM can produce more granular ratings, allegedly down to the minute. That would seem more precise than the diary could. But does that make it more accurate? Not necessarily. Neither Arbitron nor any other organization has done the work to really determine which methodology is more accurate.
We also have the very murky area of defining what we are measuring. In the old days, we measured listening. When we asked a person what stations they listened to, we wanted to know the stations they actually listened to. Arbitron doesn’t measure listenership anymore. It measures exposure.
If you walk past the stereos at Walmart on the way to the battery section carrying a PPM, the station playing on the radio will get credit, even though you weren’t even aware of what was playing. If you were a diary keeper, you probably wouldn’t write the station down when you got to filling out your diary.
If you’re a diary keeper who listens to the same station every morning driving to work, you’d probably write the station down all five workdays. If you normally carried a PPM but forget to carry it a couple days a week, the station would lose credit.
In these two very real scenarios, which method provided the more accurate estimate?
If the most important purpose of radio listening estimates is to show advertisers how many people hear their spots, it is quite possible that a diary offers a more accurate estimate than PPM. The more useful method would err by under-estimating listening, not over-estimating listenership. The more accurate method would count only those listeners who were actually aware of what they were listening to, not drive-by exposure.
Given the choice between PPM that measures exposure, not listenership, and diaries that measure listenership weighted by loyalty, advertisers would be better served by using diary estimates rather than PPM estimates.
Radio has not been served by blind acceptance of Arbitron’s PPM. Arbitron’s rush to monetize PPM combined with radio’s unquestioning naivete regarding PPM has stifled the debate on this and many other PPM issues. It is never too late to start asking the important questions.
Comments