When I grow up, I’m gonna be a Statistician!

How many of us said that, I wonder? Rather than children dressing up as sheriffs or doctors or possibly even scientists (?), how many dressed up like Statisticians? Did anyone even know much about Statisticians then? Mathematicians yes, they were sort of nerdy (although that word wasn’t around when I was a kid) but could do important things, like calculate odds of winning at Las Vegas or horse racing, and the chance of thermonuclear war.

 

But when I was young, inspired by Get Smart and The Man from UNCLE and James Bond I  mainly wanted to be a secret agent! I played with the idea of  becoming a private detective, sorry investigator, for a while until I found out that in real life, as opposed to TvLand and BookWorld, they mainly seemed to be involved in divorces. So, when I was in my very early teens, I toyed with the idea of joining the FBI. As an Australian citizen, this would have been rather difficult, I would have had to become a US citizen,  as well as either a lawyer or an accountant first. So I put that idea in the ‘too hard basket’. (Imagine, a lawyer or an accountant!).

Well, I suppose it shows evidence of an inquiring mind. Further steps, trots, canters and gallops along the road to Statistics is a story for another time. But there were a couple of ‘residuals’ from that childhood long ago. Asking questions, even if  no one else was. The desire to do the right thing, and wear the right colour hat (even if in truth the Jack Palance baddie wearing black was far cooler/groovier/jazzier in the Shane movie, although not the book, than the light coloured cloth-wearing goodie, Alan Ladd).

And a 1963 book which I got for Christmas a year or two later, called The How and Why Wonder Book of Robots and Electronic Brains. I still have that book and I cited it in my PhD Thesis, although back then I was more interested in the robots, especially the black and red tin ones that could be wound up with a key!

 

But it was a 1979 Texas Instruments TI-55 (simple) programmable LED calculator I got for my 21st, that came with quite a thick manual, showing how one could do fun things like predicting future sales from advertising expenditure, that gave much more excitement, practicality and crunch to the Psych 101 Stats that I was undertaking.

http://www.datamath.org/Sci/MAJESTIC/TI-55.htm

And then, in the early summer of 1981 when I first used SPSS (submitted to be ran at 2300 hours) on a DEC System 20-60 I was truly hooked.

True, James Bond had his Beretta and Walther PPK and Aston Martin and Bentley and Sea Island shirts and Shaken Not Stirred, but at least in the early days, he never used a programmable calculator, let alone a Computer!

 

When Boogie becomes Woogie, when Dog becomes Wolf

An exciting (and not just for statisticians!) area of application in statistics/analytics/data science relates to change/anomaly/outlier detection, the general notion of outliers (e.g. ‘unlikely’ values) having been covered in a previous post, looking at, amongst other things, very long pregnancies.

But tonight’s fr’instance comes from Fleming’s wonderful James Bond Jamaican adventure novel, Dr No, (also a jazzy 1962 movie) which talks of London Radio Security shutting down radio connections with secret agents, if a change in their message transmitting style is detected. This may have indicated that their radio had fallen into enemy hands.

To use a somewhat less exotic example, imagine someone, probably not James Bond, tenpin bowling and keeping track of their scores, this scenario coming from HJ Harrington et al’s excellent Statistical Analysis Simplified: the Easy-to-Understand Guide to SPC and Data Analysis (McGraw-Hill, 1998).

On the 10th week, the score suddenly drops more than three standard deviations (scatter or variation around the mean or average) below the mean.

Enemy agents? Forgotten bowling shoes? Too many milk shakes?

Once again, an anomaly or change, something often examined in industry (Statistical Process Control (SPC) and related areas) to determine the point at which, in the words of Tom Robbin’s great novel Even Cowgirls Get The Blues, ‘the boogie stopped and the woogie began’.

Sudden changes in operations & processes can happen, and so a usual everyday assembly line (‘dog’) can in milliseconds become the unusual, and possibly even dangerous (‘wolf’), at which point hopefully an alarm goes off and corrective action taken.

The basics of SPC were developed many years ago (and taken to Japan after WW2, a story in itself). Anomaly detection is a fast-growing area. For further experimentation / reading, a recent method based upon calculating the closeness of points to their neighbours is described in John Foreman’s marvellous DataSmart: using Data Science to Transform Information into Insight (Wiley, 2014).

We might want to determine if a credit card has been stolen on the basis of different spending patterns/places, or, to return to the opening example, detect an unauthorised intruder to a computer network (e.g. Clifford Stoll’s trailblazing The Cuckoo’s Egg: Tracking a Spy Through the Maze of Computer Espionage).

Finally, we might just want to figure out just exactly when it was that our bowling performance dropped off!