Design and Measurement
My job description for the past few years has focused around “Training Measurement” which is funny because while this job is, in many ways, the culmination of all my various professional experiences to date, I’ve never had a title that included the word “Measurement”.
Folks tend to believe that I’m a mathemetician, or at least a statistician. I recall an early conversation with a new colleague who wondered when I would share the suite of mathematical equations that I used for training measurement. For some reason the basic addition, subtraction, multiplication, division answer I provided wasn’t really what he was looking for.
No, no, no, no. The key to meausurement is design. Not design of a measurement program, but design of the thing being designed! The simplest answer to “what should I measure” is “you should measure how well something does what it’s supposed to do.”
Simple right? Clearly not. I’m continually surprised at how many organizational interventions are put in place without a clear idea of what they’re supposed to do! Sales training comes to mind. Sales training can help:
- Sell new products to old customers
- Sell old products to new customers
- Sell new products to new customers
- Sell old products to old customers.
In each case, the skillset is somewhat different, but understanding that a company with a cool existing product that wants to break into new markets is targeting #2 above. So the right measure of the training is the degree to which it helps them sell their existing products to a new market segment.
The right success measure for a car could be it’s mileage, carrying capacity, aestetic value or some other measure. In all cases, the design of the product should match the measurement plan. More ranting on this when I can make a clearer case…this seems jumbled as I read it. Ugh.
If you enjoyed this post, please consider to leave a comment or subscribe to the feed and get future articles delivered to your feed reader.
Comments
I, as you, spend an inordinate amount of time talking about causal relationship (or their lack) outside of a laboratory. Future topics include minimum effort for maximum return (and the law of diminishing returns).
I’m glad you like the blog…once I get a critical mass of information I’ll probably look for publication opportunities.
Greg,
Thrilled that you are doing this blog, and I want to point my clients right at it. Lots of my work clients, and lots of my previous jobs in giant tech company-land, are focused on designing and delivering training models, approaches, systems, etc. Ultimately, they all ask “how do I know if it worked?” to which I answer, “what results are you specifically looking for- what do you measure now that would tell you that you got what you had hoped for?” Often a blank stare at me is then inserted.
It really all comes down to the scientific method- hypothesis (e.g. “more sales”), current measure used to represent either “sales” , increase in sales, etc., and then it is about, ideally, controlling for the effect, and test. Given all other things, did sales go up for the population that was trained, versus the population that was not trained? Ultimately, it’s merely a correlation (the stats geek in me knows that proving false has a better r-squared than proving true, but that’s another post), but an entire business function (marketing) is based on such correlation, as there are almost zero causal relationships in business.