Competence Formula
From my pals at Dictionary.com: competency means “sufficiency to satisfy the wants of life.” Isn’t that awesome?
Perhaps a little more specific: competence is “the quality of being competent; adequacy; possession of required skill, knowledge, qualification, or capacity: He hired her because of her competence as an accountant.“ How appropriate that the example given is about someone hired due to competence, given the topic of today’s post.
The most common reason I’m given for developing new-hire or onboarding training is to reduce the time to competence. When I follow with the obvious question (“what is the time to competence now?”) I get all sorts of vague answers. It seems safe to say that very few organizations have operational definitions of competence and then have also gone to the trouble to track the time it takes to get there.
The details of time to competence is for another day. For today, I’m going to say what you already know:
I say you already know this, but sometimes organizations forget it. Specifically they forget that within this formula (if you will), organizations can choose where they want to spend their money:
- Hire experienced employees at a high cost (salary), provide minimal training and they’ll “hit the ground running.”
- Hire less experienced employees at a low cost, provide detailed training and they’ll hit the ground informed, but relatively inexperienced.
- Hire less experienced employees, provide minimal training and expect that they’ll get experience on the job.
I don’t propose that any of these is necessarily the right choice in all instances, but it pays to realize that there are no shortcuts to this rule. One could make the argument that your customers are going to pay one way or another – either through what they pay for your products and services, or what they “pay” for your employees to gain experience through working with them – but it’s unlikely they’re given a choice, so if your organization is operating under #3 above, it’s probably more costly in customer satisfaction (which means repeat business) than you want it to be.
So then…the questions for the day: where does your organization spend its money in this equation? Where should it spend that money? If the answers to these questions are the same, bravo!
~Geek~
Pave the path you’re walking
I’ve said before that design and measurement can’t be separated. The right measurement of anything (training, a coffee cup, this keyboard I’m typing on) is the degree that it does what it’s supposed to do; in other words, what it was designed to do. Assuming that design is about matching the characteristics of a product (training, coffee cup…you get the idea here) to the specific needs of the users of that product, then the needs of those users, as expressed in the design, become the measurement criteria.
Gracious that was mumbo-jumboish. Check out this guy’s photographic description of the difference between the paved paths at UC Berkeley and the dirt paths that people are already walking. His point, excellently illustrated, is that either the campus designers didn’t determine the needs of the population, or the population needs changed. Either way, the paths they paved are in some cases, not the ones the students are walking. The author calls out the university’s solution (a barrier that people keep walking around) as futile and shows ominous pictures of other paths starting to form in the grass.
The point here is obvious. Do you try to force people (users) to the design of your product, or do you try to match your design to their needs. As illustrated by the dirt paths intersecting in different directions from the paved ones, users will find what they want, even if your design attempts to thwart them.
So then you may be thinking “Why should I care, Geek?”
Well I’m glad you asked. In short: beware designing something without spending time analyzing the existing patterns and desires of your target audience otherwise that audience will get its needs met elsewhere.
~Geek~
Mercator Projection
I used to have a professor who ranted (seemingly at random moments in a lecture) that “the map is NOT the territory, people!”
At the time I didn’t care too much what the heck he meant because those inane ramblings never showed up on final exams. Many years later however, I find that this is some of the most useful information I got from my undergraduate time spent. The gist of the message (as I’m interpreting it now) is that the effort of creating a representation of a real-life situation will always result in bias.
This is why there has been so much research into the validity of eye-witness testimony…because while it seem simple that someone on the scene should be able to relay what they saw and what they heard in order to provide an objective account; the truth is that eye-witness testimony is often unreliable.
So today’s lesson is about the subjectivity of reality based on how it’s measured! (yay for subjectivity!)
“The map is not the territory” is nowhere more clear than in how maps are made. Way back in the 1500′s Flemish geographer Gerardus Mercator came up with the most commonly-used method for displaying a sphere (the earth) in two dimensions (on a map). The problem is that most of us (certainly I) grew up believing that the map represented what was true about the earth! Here are some differences:
- Greenland is presented as having roughly as much land area as Africa, when in fact Africa’s area is approximately 14 times greater than Greenland.
- Alaska is presented as having similar or even slightly more land area than Brazil, when Brazil’s area is actually more than 5 times that of Alaska.
- Finland appears with a greater north-south extent than India, although India’s is the greater.
Additionally, the maps I grew up looking at (and still look at) show north at the top. North, of course, is an arbitrary direction associated with one of the two points of the axis on which the earth turns. There is, of course, no “up” in space…so representing north at the top of a map is completely arbitrary. Or, as is more likely, completely self-centered. It is SO not coincidental that most maps that school children see in America have the United States at the top center.
My strong recommendation is that you all go online and order yourself a map with Australia at the top center. Spend some time studying the map. What do you notice that maybe you haven’t noticed before? Personally I don’t think I had ever noticed the Kamchatka Peninsula before but on my revised Australia-centric map, it stands out quite prominently.
The point here is that the choice we make to display or represent a situation will include bias whether we want it to or not. Next time you look at a map or a bar chart or a flow chart or ANY graphic representation…what questions will you ask that help you understand the bias?
~Geek~
Faster Mommy! Faster!
I’m going to go out on a ledge and say that Toyota’s problems with out-of-control acceleration should have been caught sooner. This seems like a case of the emperor wears no clothes…everyone expects that Toyota is the highest quality auto on the road so no one is paying attention to the numbers?
Apparently, while Toyota has some 19% of the market share, they had more than 40% of the complaints about unintentional acceleration. One could make the assumption that if they had 19% of the market…all other things being equal…they shouldn’t have significantly more than 19% of the complaints, right?
Who should have been paying more attention here?
~Geek~
Awash in A Sea of Data
I respectfully submit that the vast majority of corporations and people are awash in a sea of data that they don’t know what to do with.
For individuals, this is about life choices and priorities and yadda yadda yadda.
For corporations, this is about stopping this 4 year-old obsession with more more more (data) and determining what data will drive results. Did I not tell you? Huh?
Now so does The Economist.
~Geek~
Know What’s Important
see more Funny Graphs
Rich Today, Poor Tomorrow
About 7.6 million more people are soon going to be poor who aren’t poor today. I wonder if they’ll get a postcard welcoming them to the club.
The Obama administration is recommending for the first time in 45 years, an update to poverty definitions in America in order to accommodate more modern components like healthcare.
Of course this illustrates what I think was brilliantly (and entertainingly) covered in The West Wing episode “Indians in the Lobby“…namely that we don’t spend enough time thinking about operational definitions for our terms. In other words, most of us probably knew that there is a “poverty line” below which folks are officially poor and above which folks are…well…non-poor (probably not necessarily rich), most of us probably didn’t know what the actual definition of poverty was.
Anyone besides me surprised to learn that healthcare costs weren’t included? WTF?
You may be asking “Gosh, Geek…this is sort of passingly interesting, but why should I give a rat’s ass?” Well I’m glad you asked. Because it turns out that treatment is different, benefits are available differently, perception is different…based on the labels we apply. What are the ramifications of a label of “poor” for someone who was previously “non-poor” (working class?)?
Go forth and define your terms, y’all.
~Geek~
Incenting Weight Loss
In the continuing category of When Good Measurement Goes Bad, today’s question is about Whole Foods. These nice folks, who are (in my humble opinion) the quintessential yuppie expression of enlightenment, have recently decided employees can get an even better employee discount if they are healthier, specifically if they have lower cholesterol, don’t smoke and have a lower BMI.
My hope and expectation is that someone at Whole Foods who is paid to analyze finances, as opposed to being paid to select local organic oranges, ran the numbers on increasing health insurance costs for the coverage provided to employees and determined that these deeper discounts were worth it if they could inspire a healthier workforce and lower some of those costs.
So here’s the rub. It seems sort of uncomfortably oligarchy-ish for Corporate America to incent good behavior this way. Whole Foods is not the boss of their employees’ bodies, darnit!
But didn’t Corporate America kind of already incent us to develop the crappy eating habits that led to all this food-related unhealthiness? Diabetes and obesity and the myriad other maladies that go along with poor eating habits? Didn’t Corporate America, in order to line its already plush pockets, create fast food drive-”thrus”, high fructose corn syrup and Lunchables, and haven’t all of those things already screwed up our quality of life?
I think I’m OK with this. I think I’m OK with Corporate America (in this case, Whole Foods) responding to the only thing that matters to them…how to make more money or spend less money. In this case, if their desire to reduce costs leads them to reward healthy behavior in their employees, I’m all for it.
Go forth and measure…
~Geek~
GraphJam: Music and Pop Culture in Charts and Graphs. Let us explain them.
see more Funny Graphs
I love this one! A recent gamer…(late to the party, I know) I’m getting details personally.
~Geek~
Level 3 Evaluation
For the learning measurement wonks (and you know who you are), it seems important to take a jab at the venerable Dr. Kirkpatrick today.
For those who aren’t immersed in the learning measurement world, here’s the gist: Dr. Donald Kirkpatrick, way back in 1959, wrote his doctoral dissertation on the 4 Levels of learning measurement. His work remains the juggernaut of thinking and execution about measurement since then. If you’re expecting me to flay Dr. Kirkpatrick’s work here…sorry to disappoint, but it’s not going to happen. His work elegantly lays out a progressive measurement approach that makes perfect theoretical sense (yes, there’s a “but” in there). I will say that the way his work is implemented is often different than it was written (specifically Level 4), but I digress.
Level 3 in his model is Evaluating Behavior, particularly behavior on the job. Level 1 measures the learner’s reaction, Level 2 measures whether they learned what they were supposed to learn, Level 3 measures whether they’re using that new knowledge or skill on the job, and Level 4 measures what business results came from all that training. As I said, A great model.
In the training world, Level 4 sometimes gets replaced by Jack Phillips ROI model, which I don’t like very much for a completely different set of reasons. But today, it’s Level 3 upon which I need to geek-out.
For me, Level 3 evaluation (on the job performance) conjures images of supervisors crouching behind file cabinets with clipboards in their hands. Truthfully, I don’t even know whether file cabinets and clipboards are even part of the 21st century office anymore, but that’s the image.
Awright, enough rambling. I believe Level 3, as commonly practiced, is unsustainable. Hard words, I know, but I stand by them.
Up to the point when Level 3 is prescribed, measures of the training (Levels 1 and 2, as well as the others I think are critical earlier in the process) have been of, by, and for the training folks themselves. Once you get to Level 3 however, you’re out in the business…in the productive work flow if you will… and once you introduce a process to the work flow (like asking supervisors to fill out an observation “survey” about their recently trained employees) that doesn’t actually add value to the work flow, you brand it for extinction. Well-meaning training folks will create brilliant Level 3 documents and thrust them like sticks into the rushing flow of the business stream, where they will be snatched from their hands and whirled away. Gracious, that’s a cheesy metaphor. I should write romance novels.
Most companies have some sort of performance evaluation program (if they don’t, shame on them). Some of these are yearly, some are quarterly. Nearly all of them ask that supervisors evaluate employee performance in a number of different categories.
Well heck, if this training is so important to the business (and it is, or we wouldn’t be doing it…right?), then evidence of its successful expression on the job should already be baked into the performance evaluation process.
That said, it’s possible that the training results are a subset of the larger category that actually shows up on the performance evaluation.
(like learning in training how to put a key in the ignition, while the performance evaluation might simply rate how well you drive a car)
In cases like these, a temporary Level 3 survey or observation could be useful in the early stages of the training in order to ensure the link between the training and performance.
(if we find out in the Level 3 that even after training, folks aren’t putting the key in the ignition correctly, then the performance evaluation will fail and that Level 3 observation becomes appropriately predictive)
But as the work flow rushes by, the ideal situation is to have access to the data produced by the (hopefully…please let it be so) electronic performance evaluation process, so you can sort out the one related to the training and get regular data on how well employees are performing in that particular category. This will provide a consistent flow of data that directly reflects what I believe Dr. Kirkpatrick wanted us to learn in Level 3…namely whether people are using what they learned in their regular jobs.
I do believe Dr. Kirkpatrick’s work holds up well after all these years, but I think implementing any learning measurement program requires a bit of real-world scrutiny to see whether we’re creating a measurement process in order to check it off a list, or if we’re creating one that actually allows action to be taken for continuous improvement.
Go forth and measure…
~Geek~