The voters of America will elect a new President on Tuesday. The job comes with the additional titles ‘Leader of the Free World’, as well as ‘Commander in Chief’ of the planet’s largest military force, so the stakes are high. As is interest in the outcome, understandably.
My work is mainly concerned with sports. But at a big election discussion and interest crosses into territory that I should know something about, which is projecting future events using data and models.
So here is some insight into how I would look at an event like the US Presidential Election…
NOBODY KNOWS SHIT
A good place to start is with the observation that ‘nobody knows shit’.
The election on November the 8th is an event in the future that is subject to random variation, so nobody can ‘predict’ what will happen. Everybody is free to try, and by the law of big numbers plenty of people will guess correctly. But guessing correctly is not the same as predicting something.
To predict the result of the election with great precision it would be necessary to get inside the mind of every possible US voter on the evening of November 7th in order to know what they were going to do. This is the equivalent of looking at an upcoming football match, and attempting to predict exactly what will happen in every second of play.
A genuine scientific prediction of how a game will go would require you to know intimately about the thoughts, abilities and physical capacities of all the players, the referee and assistants, and be able to foresee every bounce of the ball, decision, gust of wind and bobble that was going to happen.
You can’t predict complex future events. Crystal balls aren’t real. All you can do is guess. The best guesses are done making informed forecasts based on available evidence, and that look at future events probablistically.
So the output of my work on a football game would be something like ‘Arsenal have a 67% chance of winning, 20% chance of drawing, and 13% chance of losing’. I don’t say ‘I think Arsenal will win’, even if I estimate that this is the most likely outcome. I say Arsenal = 67%, and our clients make of that information what they will.
To arrive at forecasts of probabilities (aka ‘odds’ or ‘prices’) we weigh hard and soft information (quantitative and qualitative if you want to use long words). Hard information is data on past performances – how many games a team has won/drawn/lost, how many shots etc. they’ve had. Soft information is stuff like team news, and what the state of the weather and pitch is on the day.
Forecasting an election is basically similar. You have hard data in the form of opinion polls and the results of previous elections. And soft evidence is stuff that isn’t yet reflected in the poll, or evades being measured empirically, such as knowledge of each campaign’s ‘ground-game’, which is their on-the-day efforts to get their voters to the polls.
A difference between football and elections is the amount of hard data there is to play with. On a sport like football there is now loads of high quality hard evidence. Teams play lots of matches in which they are trying their hardest to win, so we can make pretty strong conclusions about how good they are. Elections on the other hand are rare. So election forecasters fill in the gaps with data from opinion polls, which is much lower quality hard evidence.
Often it happens that after an election people will complain ‘the polls were wrong’. But this is the wrong way to think about it. The polls can only be wrong if you take them to be a prediction of the actual election. Which you shouldn’t.
Polls are not an election. They are an imperfect snapshot of reality. Taking an average of poll results is a lot better than sticking your finger in the air, or listening to a guy in the pub. And building a good model using poll data is better than that. But it’s still only good for estimating probabilities, not for making a prediction you could stake your life on.
Here’s a way to look at it;
THE FOOT RACE(S) FOR THE WHITE HOUSE
The US Presidential election is a series of 50 individual running races that will happen on November 8th in the 50 states. In every race except Utah, the race for the line is almost certain to be between Hillary Clinton and Donald Trump. First across the line wins each race, and gets all the ‘points’ on offer for that race. More points for races in bigger states. The winner overall will be the one with most points, not necessarily the one with the fastest aggregate time. So winning races in big states is worth more than winning the small ones.
Why won’t the faster runner simply win all 50 races? Well, because the terrain of the course of each race matters a lot. Trump and Clinton have different running styles (as do all runners from the two big parties in America, the Democrats and the Republicans) so they are much more suited to certain kinds of terrain. They are almost certain to each win the races on their favourite types of course, so the overall winner will come down to who wins in the races where the terrain doesn’t give either of them a huge advantage. These are the ones which could go either way, so are known as the ‘swing races’.
The way the campaign works is that in the months before the big day on November 8th, the runners do training runs on each of the 50 courses, and on neutral terrain. The results of these training runs are published as polls. Some people take them as a definite indication of what will happen in the race proper. But that’s a mistake. The smarter people take the hard data and use it to build a model, a picture of how all the 50 races might go and how the final points tally will look.
The hard data is inconclusive because the runners only ever train on a tiny part of the course, covering a fraction of its actual distance. The people who set up and record the results of the training runs often take great care to make the small section of training course used as representative of the overall terrain as possible, but it’s still not the same thing. Although, where the runners have a strong terrain preference the results of even these short runs are generally enough to be sure who is more favoured. But it’s the ‘swing races’ that are trickier.
Although each race is run independently, with the runners starting shoulder to shoulder on the start line, how a runner does in one is correlated with all the others. This is because it can happen that one of the runners simply runs better on the day than they have done in training. It can also happen that on the day, one of the runners will actually cope with specific types of terrain much better than they have on their training runs. In this case their chances of winning most or even all the swing races goes way up.
The possibility that either runner will in fact go better ‘on the day’ than they did in their training performances means the chance of the apparently slower runner winning the overall race is normally a fair bit higher than is estimated by the people who take training runs as gospel. The guy who looks slower in training can easily win the real thing.
But it also means that the chance of one runner winning by a lot of points overall is higher than is generally assumed by people who miss the possibility of correlations between races. There’s a bigger chance of one runner winning most of the points than you would guess using an assumption that all the races are completely independent.
What else affects the races? Well, the runners get to spend money on tweaking the course in each race. They can pay to get a few extra bits of preferential terrain added/removed on the course. There’s a low limit to how much difference they can make – it can’t make a totally unfavourable course winnable or anything – so virtually all of the money the runners spend will be in the swing races where any tiny advantage might make the difference.
On the day of the races the runners can employ people to help keep them on course in the races – these are the Ground Teams. These helpers aren’t allowed to give their runner a full push in the back, and they’re not supposed to get in the way of the other guy, though sometimes, allegedly, they do. In a close race though perhaps they can make a small but significant difference.
As well as the training runs, the two runners face off for three arm-wrestles during the lead-up to the big race day. Arm-wrestles aren’t directly relevant to running obviously, but they can give a rough sense of who is fitter and therefore a more worthy overall winner, and it usually gives a decent psychological boost to whoever wins them (which was Clinton this time round).
Both runners get a new pair of running shoes a couple of months before the big day. These shoes are called ‘Nike Air Conventions’ and normally give the runners a little boost in their training run times right after they first put them on, though their effect will usually have worn off by the time of actual race day.
In theory the runners should spend the weeks and months before the big race day doing their own training, and trying to avoid injuries. But in this year’s race in particular, a lot of the run-up has been dominated by the runners concentrating less on their own fitness than deliberately trying to make the most of injuries to the other one.
In a ‘normal’ year some of the injuries Clinton and Trump have received would have been enough to torpedo their chances against other runners. For instance Trump pulled a hamstring when out on a training run with Billy Bush that seemed likely to cripple him, and possibly even stop him from making the start line. But he’s still limping onwards and now even has a reasonable shot of winning, because Clinton herself still bears nasty scar tissue from several old injuries. Just last weekend, Clinton took a particularly nasty new knock to an old injury to her achilles from a stray kick by James Comey.
In truth, it looks like this year’s Presidential race may well be between two of the worst runners ever to have contested it. It’s not so much about them running really fast, as just staying on their feet gives them a decent chance. Usually all the different terrains in all the courses of the Presidential races will clearly favour one runner or the other by this stage. But such is the poor quality of this year’s contenders that it’s still far from clear who is going to win the overall race.
THE REAL RACE
So when you hear people talking about a candidate having a ‘lead’ in the polls, that’s probably the wrong terminology. ‘Advantage’ would be a better word – as in they appear to have an advantage on the terrain of a particular state race, and/or their overall form appears to give them a basic running speed advantage over their rival.
Polls can certainly be useful in showing who has an advantage on the particular terrain of a state race. Or in the case of a national poll, who is running into better form overall. But they are not a race in themselves, so nobody can be ‘leading’.
Opinion polls can’t be ‘wrong’, either. They aren’t supposed to be a faithful representation of the full election race, no matter what the people who release them claim. They are just little manufactured snapshots. Small scraps of evidence. Dumb numbers. Don’t treat them as predictions of a complex, random future event.
Plenty of things that can have a small-ish effect on the final result just won’t show up in polls. Good polling is really difficult. It’s tough to get a balanced sample of responders to your questions, reflecting who will actually vote on the day. And even then, people can change their minds.
In the 2015 UK General Election, and that country’s 2016 EU referendum, taking poll numbers literally proved wrong as the polls were ‘out’ by several percentage points. The same mistakes in interpretation of data are being repeated in America right now.
Does ‘ground game’ matter much? If it does, and if it’s true that Clinton’s is vastly superior to Trump’s then that’s a ‘soft’ bit of evidence that the polls can’t know.
Are there any other soft factors that could have evaded pollsters and be a real factors in voters’ minds on voting day? Possibly, but I’m a sports guys so I’m not going to get too deep into speculating about them.
If I was modelling the US Election the one thing I can say for sure though is that I would have a fat-tailed distribution of possible outcomes. These are two deeply unpopular candidates, and there are way more undecided voters than normal this far out from election day. And Trump is a most unusual main party candidate. There just isn’t any decent precedent for guessing how late-deciding voters will break this year, and the race is close enough that how the late-deciders vote in the swing states could well decide the thing.
Models that don’t account for the correlations between voting in different states are probably bad models. And a model that ignores the fact that there are a lot of undecided voters is definitely a bad model. A good model will find a smart balance between guessing which way undecideds will break, and acknowledging the fact that it can’t really know for sure by having a wider spread of possible outcomes. For this election that would mean showing Trump as having a better chance of winning than a strict reading of polls (especially ones that simply exclude undecideds) would show – but also a greater chance of a wide Clinton win.
I’d definitely be less certain about who was going to win than the people who extrapolated from a bunch of recent good polls/training runs for Clinton that she had a 95%+ chance to win. No matter what you think of Donald Trump, he has a genuine shot of winning this election. Though also a fair chance of getting absolutely thumped.