Immersed in Billboard stats

For my Statistical Modeling class we have been working -- or, perhaps more accurately, were asked to work -- on a project all quarter in which we collect data and apply models to it. So far, of the three people I've seen present their projects, two examined economic indicators in foreign countries, and the third looked closely at the auto insurance industry. That stuff is, like, way too businessy for this unconventional candidate.

Instead, I will be looking at the Billboard Hot 100 singles chart in the early 1990s. Anyone who has ever said hello to me probably knows that I am obsessed with Billboard. As a wee lad of 12, my mommy would drive me to the library each and every week so that I could read the magazine and photocopy the pages that most interested me. It cost 30 cents; the magazine, at the time, was like $4.50, so a subscription was a little out of the question considering I was really only interested in a few pages, and it's a very thick magazine for industry insiders.

I'm not sure why I found the charts so interesting, but I'm not the only one. There appears to be some sort of dedicated cult of people into this, as is evidenced by the devoted following of Fred Bronson's now defunct Chart Beat Chat columns each week on Billboard.com (and, I assume, in the print edition). He even printed some of my questions! He retired last year. Anyway, I just loved the charts -- I always watched the countdown shows on MTV and VH1 and wrote them down, I had my own personal chart of my 20 favorite songs each week, I listened religiously to Casey's Top 40. Just loved it. My interest waned slightly once I could drive and started interacting with humans to a greater degree, but I still check the Billboard Hot 100 every Thursday to see what's hot. I usually gasp in horror and then complain about how awful music is today.

So my project. Prior to November 1991, Billboard based its Hot 100 Singles chart on sales figures reported by retail stores. There wasn't yet the technology to measure actual sales. Then Nielsen introduced its SoundScan technology, and Billboard started using it to compile its sales data. The Hot 100 Singles chart is sort of the aggregate chart that ranks the popularity of songs across genres, and it combines sales and data information.

Who cares, you ask, if you're even still reading. Well the thing is that in the few years after this change, several important records were broken or tied, most imporatntly the record for longest-running No. 1 song. If you look at this list you will notice that the songs that hold the record for most time at No. 1 were released after this change. The longest running No. 1 song ever is One Sweet Day by Mariah Carey & Boyz II Men, released in 1995. It was No. 1 for 16 weeks. No song has been No. 1 for 15 weeks. Six songs have been No. 1 for 14 weeks and, of those, four were released between 1992 and 1997. The other two were released in 2005 and 2009. Amazingly, as you scroll down, you don't see a single long-running No. 1 song released between 1982 and 1991. It's a little suspicious ... One wouldn't be crazy to hypothesize that the methodology had something to do with it.

Anyway, that's what I'm trying to analyze. Since I went to the trouble of entering in tons of data I'm also looking at some other things, like whether being male or writing your own song has any influence on how long it takes to reach No. 1 or how long you stay there. So far I'm not finding anything too interesting except the fact that the introduction of SoundScan made sounds statitistically significantly more likely to get to No. 1 faster and stay there longer.

"You should work for Billboard," You suggest. I contacted them a couple months ago; they only hire undergrads as interns. :-(