What Coincidences Tell Us

Today I happened to walk down South Hill Park in Hampstead. South Hill Park is greatly beloved of fact fans as it's associated with one of the most remarkable coincidences in London's history. South Hill Park is relatively well-known as the street where Ruth Ellis, the last woman to be hanged in the UK, shot her lover outside the Magdala pub. What is less well known is that the second-last woman to be hanged in the UK, Styllou Christofi, also (and completely unrelatedly) committed the murder of which she was convicted on South Hill Park, at house number eleven. The cherry on the aleatory cake is that South Hill Park is, quite unusually, distinctively noose-shaped.

Ruth Ellis, Styllou Christofi, the noose-shaped South Hill Park
Hardened rationalists though we may be, we have to admit that this is an extremely tantalising confluence of events. Great coincidences like these demand our attention. The interesting thing about our reaction to coincidences is what it tells us about our cognitive machinery.
First, though, it is worth noting that, by-and-large, the occurrence of coincidences is not objectively remarkable. Allowing that the South Hill Park coincidence has - let's say - a one-in-a-million probability, and that there are - getting very cavalier with our concepts - several billion facts, it's a certainty that a large number of the facts constitute coincidences. Coincidences involving notorious people are a small proportion of the whole because there aren't that many notorious people, but we shouldn't, objectively, be that surprised.
Why are we, then? To answer this we need to work out what makes something a coincidence in the first place. This is not as easy as it seems it should be. Our first reaction is to say that a coincidence is some kind of low-probability event. But this falls over very quickly on examination, since almost every event is low probability. A bridge hand consisting of all thirteen spades is exactly as probable as a bridge hand consisting of 2-K of spades and the ace of hearts, or indeed any other particular combination of cards. Discovering that all four people in your team share a birthday is exactly as probable as discovering that they all have some other specified but unremarkable combination of birthdays.
What makes something a coincidence is not, then, directly the probability. Coincidences seem instead to require data points with shared values across multiple fields. This is a somewhat abstract way of putting things, but what it means is that:
(Name = Alice, Team = my team, Birthday = 1 June) (Name = Bob, Team = my team, Birthday = 3 August) (Name = Charlie, Team = my team, Birthday = 19 December) (Name = Dave, Team = my team, Birthday = 5 June)
is not a coincidence, because the data have shared values in only one field ('Team'), but that
(Name = Alice, Team = my team, Birthday = 1 June) (Name = Bob, Team = my team, Birthday = 1 June)
(Name = Charlie, Team = my team, Birthday = 1 June)
(Name = Dave, Team = my team, Birthday = 1 June)

is a coincidence, because the data have shared values in two fields ('Team' and 'Birthday'). It's not about probability - the probabilities of both these datasets are equal - but about features of the data. The interesting thing is that, put like this, the connection between coincidence and hypothesis testing becomes very clear. Broadly, shared data values provide evidence for the existence of a hypothesis which posits a lawlike relationship between the two fields; in this case, the data support (to an extent) hypotheses that I only hire people whose birthday is on 1 June, or that babies born in early summer are particularly suited to this kind of work, and so on. It is only the low prior probability for hypotheses of this kind that leads us, ultimately, to dismiss them as possible explanations, and to accept the data as a 'mere' coincidence.
This doesn't of course stop us wondering at them. And this is the interesting thing: it suggests that our sense of wonder at coincidences is how a currently-running unresolved hypothesis-generation algorithm expresses itself. When we get the explanation, the coincidence - and the wonder - go away. If I tell you that last night I was at a party and met not one but two old schoolfriends who I hadn't seen for twenty years, this is a remarkable coincidence until I tell you it was a school reunion, at which point it stops being interesting at least in one regard. The data are the same, but the existence of the explanation rubs away the magic. The feeling of awe we get from coincidences is the feeling of our brain doing science.