My stats guru colleague Dr Andrew Pratley and I are on the move to tackle Quantifornication, the plucking of numbers out of thin air. Here is the first in a series co-written together.

One of the biggest problems with estimates is that it takes a long time to find out if you were right. A year for a budget, two, three or more years for a strategy. Political polling is the best example of being able to assess if the estimate was correct. On election night – and we all know how that has been going lately.

One scenario we have been working on is cyber security to answer the question, how much cyber security is enough? Or, better still, what is the risk?

There are two broad extremes which are worth starting from: do nothing, spend no money and hope for the best; or do everything, have an unlimited budget and know you’re safe. The value of extreme positions is not to advocate for one or the other but to place markers. The current theory is that the more you spend, the more you can do and therefore the safer you are.

Let’s test this out. If you had a house with ten entrances and you could only spend money to secure seven of these, in theory, you’d be safer than someone that could only secure three of the entrances. This statement is only true if you assume that every entrance is equally likely to be used to break-in. This assumption is rarely true, even when it seems entirely reasonable. The reason is explained via the Monty Hall problem:

      Suppose you’re on a game show, and you’re given the choice of three doors:

      behind one door is a car; behind the others, goats. You pick a door, say No. 1,

      and the host, who knows what’s behind the doors, opens another door, say No. 3,

      which has a goat. He then says to you, “Do you want to pick door No. 2?” Is it to your

      advantage to switch your choice?

When presented with this seemingly straightforward problem, the vast majority of people choose to keep their selection. They end up disappointed and go home with a goat.

Our message to you is that not everything is what it seems and that you should be making more accurate assessments. Next week we will discuss how to calculate more accurate estimates, what statisticians call point estimates. Quite different to guesses!

Stay safe and adapt – with better measurement!