Musings on statistical analysis – simple isn’t always right

Those who know me well (and, actually, those who don’t even know me all that well), will know that I am a life-long fan of the UK science fiction TV series Doctor Who. My first memory is of Doctor Who, I ran a Doctor Who library at primary school and a Doctor Who club at secondary school and have written fan fiction. It will therefore come as no surprise that my first brush with quantitative research was all to do with Doctor Who. You might think this is a bit nerdy but brace yourself – it gets nerdier.

The viewing figures for broadcast episodes of the series have been recorded since it started in November 1963. There have been more than 860 episodes, all with viewing figures in millions. For stories of multiple episodes, there is also an average (a mean) viewing figure for the whole story. And, as stories are grouped into series (known as seasons in the original series, broadcast from 1963 to 1989), there is also a mean viewing figure for each series or season. For the stories broadcast in the time of catch-up TV, there are overnight figures, 7-day figures and 28-day figures, to take account of those who don’t watch on transmission. Imagine all those numbers! They are a statistician’s goldmine.

As a child I would carefully transcribe all these numbers from various sources into lists then turn them into graphs and charts for comparison. I had no idea that I was doing a very basic statistical analysis of the data. I still have all those sheets of lists and graphs. All hand drawn, of course, as this was the 1970s and 1980s. They even have my estimate of a trend line, even though I didn’t know what a trend line was.

Fast forward to 1991 and I’m in my last year at university, taking a module called Introduction to statistics. The main assignment was to perform various statistical tests on a dataset of our choice. Naturally, I used the viewing figures from 1963 to 1989 as my dataset. Using mean viewing figures for each story, I correlated them with the day and time of transmission to prove that viewing figures were higher when the programme was broadcast on a Saturday instead of a weekday. There had been a time in the 1980s when the BBC had experimented with broadcasting the show on consecutive weekdays – this was bonkers, in my personal opinion, and now my analysis proved that the viewing figures were damaged by this transmission slot. I felt vindicated.

But the analysis was too simple. There are other complicating factors. The weather can affect viewing – back in the days before catch-up TV, if you missed the original transmission there was no chance of seeing it at a later date. You just missed it. If the weather was nice and you were out somewhere, the chances of persuading the adult in charge that it would be better to go indoors and watch TV were slim. Other things affect the figures, too. Consider how many TV channels there were in the UK in the 1960s. There were only three – BBC1, BBC2 and ITV (with regional variants). This meant that there was very little competition for each programme and viewing figures for popular programmes therefore tended to be high (the mean viewing figure for the first season, broadcast from November 1963 to September 1964 was just over 8 million). Today there are hundreds of channels competing in the same space (and competing with many other sources of entertainment on the internet) so it is not surprising that viewing figures for popular programmes are not as high as they were in the 1960s (the mean viewing figure for the most recent series was just over 5 million).

There was a notable time in the 1970s when a strike at ITV meant that viewing figures for BBC programmes at that time were improved, as there was little other choice of TV to watch. A Christmas special or the first episode of a new series often draws a larger viewership, especially if there is a new actor in the title role, as people are curious about what it will be like. The return of a well-known adversary, such as the Daleks or the Cybermen, in the story title might also encourage more to tune in. There may also be an effect related to what is broadcast immediately before the programme – this is why continuity announcers tell you what’s coming up next, so you will stick with the channel. If there is a family-friendly programme broadcast immediately before the family-friendly Doctor Who, it may be more likely that people will remain tuned in. There is also some discussion in fan circles about the time of year affecting viewing figures as well as the time of day being important. Traditionally, Doctor Who was a Saturday teatime show, broadcast in the early evening (5.20pm to 5.45pm), starting in the autumn and running through to the spring. This avoided broadcasts in the summer months when people were more likely to be outdoors and allowed children to be gently scared in the (literally) darker autumn and winter months.  More recent series have begun at Easter, running into summertime or have been split in two halves or started in the new year. The length of the episodes may also be important – most of the original series episodes were 25 minutes long, with a few in the 1980s that were 45 minutes long. Since 2005 the episodes have been 45 minutes long, with an hour for Christmas specials. Then with the arrival of the first female actor to play the title role, the episodes moved to 50 minutes.

So, for my simple analysis correlating day and time of broadcast with viewing figures, I should really have taken into account the following:

  • What was being broadcast at the same time on other channels
  • What the weather was like
  • Whether there was any industrial action that might have affected viewing figures
  • Whether there was anything special about the episodes, such as a new actor in the title role or a return of a popular adversary, such as the Daleks
  • What was broadcast immediately beforehand
  • The time of day
  • The time of year
  • The length of the episode
  • What the average viewing figures were for each time and day and at that time of year when Doctor Who is not being broadcast.

There would be other contextual factors, too, such as the general popularity of science fiction in the popular media, the way in which the programme was promoted and the ways in which television has changed over the years, but these would not have been quantitative measures.

Finally, there is also another set of data we could take into account – the Audience Appreciation Index (AI), which is a score out of 100 that shows how well liked the broadcast programme was. That could be interesting as shows with low viewing figures could still be well liked and vice versa.

To take all these variables into account would have required multilevel modelling and, alas, my Introduction to Statistics module didn’t cover that. I live in hope that someone somewhere will spend some time doing this analysis for me. (Drop me a line if you do it.) For more information on multilevel modelling, check out the Centre for Multilevel Modelling at the University of Bristol.

I shall continue to muse on this further. I’ve been musing on it for some years already, as I’m sure you can tell. While this post may open up potential research avenues for some, it may convince others that I am obsessed with trivia. And those who know me will say, ‘Yep, a post about Doctor Who. It didn’t take long, did it?’

3 thoughts on “Musings on statistical analysis – simple isn’t always right

  1. Nyder

    Nice article. I can see you’re right about all the other factors that might affect figures… was Wonder Woman on ITV at the same time for example? For me a bigger factor might be who was Who?

    For example, Sylvester McCoy is a top bloke (especially with spoons and ferrets) but when you’ve grown up with Tom and JP he was a bit… well, “slapstick”.

    In other words you need an inkling into viewing figures correlated to relative time.

    Liked by 1 person

    1. Thanks for your thoughts, Nyder! I’m a big McCoy fan personally, but I see the point about different lead actors being more appealing than others to some people. I remember Buck Rogers being on ITV when season 18 was broadcast, so there was actual sci-fi competition and the viewing figures for that show would be interesting to compare. I’m all for adding to the dataset!

      Like

Leave a comment