Racist record-store owners, circa 1990-91?

I may have made an amazing discovery and am pretty excited.

I mentioned a couple posts ago that I'm working on a final project for my stats class that looks at a policy change on the Billboard Hot 100 singles chart that was implemented in 1991. The change was the introduction of SoundScan technology, a method invented by Nielsen to count sales of singles. Before that, record stores just reported their sales.

So I created a data set with all sorts of variables to see which ones might have been affected by the change. Most weren't -- things like how many songs debuted each week, how high the highest debut was, the amount of movement into the Top 20, whether men were more likely to be No. 1, etc., were all statistically insignificant. Two things I found that appear to be statistically significant, and which I expected, were that the change made songs go to No. 1 faster and stay there longer.

What's interesting is the third thing I found, something unexpected and even more statistically significant than the first two things: More black artists were in the Top 10 after the change than before. You can actually eyeball this in the graph above (nobody wants to see my Stata output, right?). Time is on the X axis, counted in weeks from Dec. 1, 1990, to Nov. 28, 1992. On the Y axis is how many songs by black artists were in the Top 10. (If you look closely you may wonder why there are fractions of black people ... it's because of duets and groups that shared vocals, like Color Me Badd.) I've drawn in a red line to show where SoundScan was brought in.

I think that unless I've made a mistake in my little model, this is an awfully interesting discovery. It basically implies that record store owners might have been under-reporting sales by black artists. It could certainly be a coincidence too -- maybe there were just more popular songs by black people in 1992 than in 1991. Maybe radio warmed up to black artists in 1992. Both of those seem a little far-fetched though ... There's a pretty stark difference between what you see to the left of the red line and what's on the right. I love being armed with tools to discover things!