I don’t know if you have noticed, but I have been blogging about measurement, data, and statistics for the past 19 weeks. All to defeat quantifornication – the act of pulling numbers out of thin air for decision making. Numbers that are seemingly reliable but are not.

This week, Tuesday 10th November at 15:00hrs AEDT, I am presenting with my colleague Dr Andrew Pratley. Our presentation is Statistics 101 Applied to Controls – How to test and measure control resilience. Our presentation is part of the RMIA and AISA Risk & Cyber Week conference. We only had 25 minutes and so we made the call to prepare case studies for people attending the presentation, to have somewhere to go to dig into these concepts, for a proper understanding of the value of statistics.

Wind forward and Andrew and I have recorded near on an hour of video, explaining three examples of how to use statistics for assessing cyber security controls. You can find them here.

But here is the kick. I am a chemical engineer with and MBA. I did statistics in engineering and in my MBA. Did I understand it? Enough to pass but not enough to do anything practical with it. Andrew has long said that the way statistics are taught, is entirely unconducive to gaining the in-depth understanding needed to apply it.

You will see in each of the three case studies, that Andrew describes the control test and the data created from it. He then, in just a few minutes, runs a statistical test (e.g. a t-test, chi-square) and explains the interpretation of the results. I then asked him clarification questions. And finally, I get stats 101.

My biggest takeaway to share with you was from the last case study, where we discussed a situation where one set of data looks better than another – but is not statistically significant. That is, a non-statistician would have said Option A is better than B. Whereas it isn’t. The next step is to expand the test. As you expand the test, the truth of A being better than B, or not, will become more evident. That is, start small and low cost and expend more effort only as needed.

As you may not have access to the introduction of these case studies if you are not attending Risk and Cyber Week, feel free to get in touch for a little more explanation.

Stay safe and adapt – with better measurement!