When’s the last time somebody interviewed the program director of a station that just tanked in the ratings, asking about the keys to the station’s performance?
Radio isn’t interested in losers, only winners, so the stations that get all the press are the ones that are winning.
With over 2000 radio stations competing in PPM markets, each month hundreds of stations will have sizable gains and a handful will have spectacular gains.
And every month, the winners will get all the attention and speculation about why the stations did so well.
The problem is that nobody can really know why a station has a great month. Maybe it’s something the station did. Maybe it’s something the station didn’t do. Maybe it was something some other station did.
In reality, most of the gains from month to month are just by chance, but human nature and radio being what they are, the idea that most rating gains happen purely by chance is too unsettling.
Despite evidence that rating changes from month to month are purely random, radio pays close attention to the explanations of program directors, general managers and consultants in the belief that somebody must know the secret.
Let’s say a station does something like (say) commercial-free hours, has a good month and it makes the news. The following month other stations try commercial-free hours, and some of those stations also have a good month.
Seeing the number of stations doing commercial-free hours grow, the next month other stations try the same thing, and some of them have a good month.
Does that prove that commercial-free hours are the key to winning in PPM?
Not unless the number of commercial-free winners far out-number the commercial-free stations that were flat or went down.
It is called survivorship bias. We tend to only notice the winners. There may be just as many losers as there are winners doing the same thing, but we ignore the losers and pay attention to the winners.
Think about the Russian roulette scene in Deer Hunter. Robert De Niro’s character spins the gun cylinder, pulls the trigger, and lives. Does that mean De Niro has a special gift?
What if a hundred people decide to do the same thing?
One hundred people spin the cylinder, pull the trigger, and only ten people shoot themselves. That means 90 people are winners. They will be heralded and interviewed about the secret formula for winning.
The 90 survivors spin the cylinder again, 9 shoot themselves, but that still leaves 81 winners prepared to offer their key to staying alive.
You can see where this is going.
Most people win at Russian roulette, but who wins is determined purely by chance. There isn’t a secret formula for winning (except not playing).
Every month when the Arb numbers roll there will be plenty of winners. That means there will be plenty of stations that seem to have the winning formula, even if most won just by chance.
Last December even Arbitron got in on the act by presenting a piece called The Ratings Characteristics of PPM Top Performers (pdf).
The presentation was on pretty shaky methodological ground, but that didn’t dampen the excitement in learning PPM’s secrets.
One trade publication flatly declared:
Arbitron found hardly any difference in the length of listening durations among average stations and top performers, but top performers had significantly more daily listening occasions. It's all about the number of listening occasions.
This is how PPM myths are born.
Let’s look at what Arbitron did. They took the top 3 stations in 33 markets, averaged their share, cume, TSL, and occasions, and then compared them to the average station. You can read their conclusions above.
So the biggest difference between the top stations and the average station was cume. If you want to be a top three station you need nearly four times the audience as the average station.
Not an earth-shattering observation, but a useful reminder.
TSL differences were considerably smaller. Top three stations only have 22% more TSL. Not a huge difference, particularly when the cume difference is 247%!
Daily occasions differ by one single occasion. That’s it. One occasion.
On top of that, the top station in each market has no more occasions than the average of the top three. Despite Arbitron's conclusion, the evidence suggests a rather weak relationship between occasions and winning.
Yet we're told that it's all about occasions.
Arbitron released only 18-34 results for some reason. The presentation alluded to a 25-54 analysis, but those results were not shown.
In our next post we’ll continue discussing the issue of PPM winning secrets and show a Harker Research analysis that raises more questions about Arbitron’s Top Performer claims.
In the meantime, be very wary of people claiming to know the secrets of PPM....and stay away from guns.