Variability Happens
While not nearly as catchy or bumper-sticker-worthy as “shit happens”, I’m reminded of the fact that “variability happens” pretty consistently.
That’s the intro to why you HAVE to watch this video of Malcolm Gladwell. The Geek loves him some Gladwell, it must be said. But in addition to just getting to watch his gravity defying hairdo move around the stage, you also get to hear a detailed explanation of why data can be misleading if you aren’t careful about your assumptions…in this case that there would be one answer to a research question instead of multiple (variable) answers.
Plus, he explains it all in the context of…wait for it…spaghetti sauce. Thanks to my pal Khary Jackson for posting this on Facebook.
~Geek~
iTunes Conspiracy
OK, maybe I titled this a little dramatically, but it will be fun to see who finds this post in their rant-research!
I work from my home office and listen to my iPod all day…it sort of blends into the background noice. Heck, even when I DO go to the office, I play my iPod on headphones. So I notice patterns that might go unnoticed by a casual listener.
Specifically, I notice that Manheim Steamroller’s Christmas Album seems to be a particular favorite of my iPod’s. Because I have folks ask me about randomization pretty frequently and because the nature of true statistical random selection means that weird patterns can emerge, but over time the weird patterns should fade into the background of random-ness.
So I had a moment between projects today and thought I would do some research on why some artists/songs seem to play in my iPod’s “shuffle” selection more than others.
Interestingly, this guy says that this is actually a helpful feature from our iOverlords! His point is that my iPod pays attention to what songs I skip and eventually exclude those from the shuffle. Bull Pucky! I made (I realize now) the mistake of buying an iTunes special package of Fiona Apple (OMG, I just got the connection…Fiona APPLE: APPLE Computers. Good hell!) that includes not just some songs but Fiona herself droning on about how she wrote songs or how she fought with her studio, or how she goes about being an artist bzzzz bzzzz bzzzz bzzzz bzzzz…I’m sorry, what were you saying?
Well I can tell you that whenever Fiona starts her random rambling, I skip that trash EVERY TIME, and yet I can expect to have it forced on me (despite actually trying to delete it from iTunes at least once) at least twice a day.
These folks say that it IS random, but that I would have to listen to all the songs in the library in one sitting (without skipping or stopping) and I would hear every song once. BULL PUCKY! I have sat at my desk for a 4 – 5 hour stint and heard the same song (out of many thousand possible choices) TWICE!
So I was pleased to come across someone who had conducted an actual experiment and surprise! It IS a conspiracy! (OK, maybe not, but it’s suspicious as hell). Here’s the full article [David Braue for CNET Australia; published on 08 March 2007].
I totally admire the author’s methodology to test the random-ness..the article was worth the read (it’s relatively short) just to see how they setup the experiment. But note that they tested the creation of random playlists, NOT random play on iTunes or iPods. Assuming the randomisation (Australian spelling used in homage to the author) math is the same for creating playlists and for playing songs, these results should be solid.
Their method was to use a combination of songs ripped from CDs and songs bought from iTunes. Artists were equally represented (i.e. 5 songs each in most cases, except where the experiment dictated something different) and results were tracked by what studio the song came from in case (as was the theory they were working from) there was evidence that studios had paid Apple for preferential treatment for their songs. Songs were added to a library, then assigned randomly to playlists (using iTunes feature for that).
If truly random, one would expect songs to show up evenly for all artists, but that’s NOT what happened!
Here are the summary conclusions:
- 20 playlists (10 of 25 songs, and 10 of 40 songs) were created from a pool of 100 iTunes Music Store sourced songs, and 20 additional playlists when the pool was expanded to 200 songs using CD-ripped songs. This provided a total of 1300 slots to be filled at random.
- On average, one would expect each song to appear on 6.5 playlists.
- Popular, top-50 singles were rotated onto our playlists far more frequently than would be expected. Some artists, having just one song in the iTunes Library, were played more often than the entire 5-song collections of other artists.
- Artists and singles purchased through iTunes were played more frequently than those that were not.
- Four songs — Christina Aguilera’s At Last, Creed’s What’s This Life For, Crowded House’s World Where You Live and Led Zeppelin’s Nobody’s Fault But Mine — were in the iTunes Library but were not chosen for any of the 40 playlists generated during this exercise.
- Lionel Richie (Universal) was iTunes’ favourite artist; his songs were chosen 59 times for 40 playlists [iTunes songs only]. Times per possible playlist (TPP) = 1.475.
- Def Leppard (Universal) was iTunes’ favourite artist among songs ripped from CD; their songs were chosen just 24 times for 20 playlists [iTunes songs and ripped MP3s]. TPP = 1.2
- John Mayer (Sony) was iTunes’ least favourite artist; his songs were chosen just 32 times for 40 playlists [iTunes songs only]. TPP = 0.8.
- Oasis (Sony) was iTunes’ least favourite artist; their songs were chosen just 10 times out of 20 playlists. TPP = 0.5.
- Songs from Universal and EMI showed up in more play lists than their share of the iTunes Library would suggest.
Could this be a result of the relative popularity of each label’s artists, or is somebody conspiring to keep Sony’s numbers lower? Or is this just a natural manifestation of the known deficiencies in computers’ random-number algorithms?
It’s obviously difficult to tell whether back-room marketing deals or just dumb luck were responsible for the results we saw, but it appears that we can safely lend credence to the suspicions of myriad iPod users around the world. When it comes to choosing songs, ‘random’ clearly is relative.
Songs from Warner and Sony showed up in fewer play lists than their share of the iTunes Library would suggest. The disparity was striking in Sony’s case, with the company’s 67 songs (the largest single label representation amongst our Library) accounting for 34.18% of our songs, but chosen for just 18.8% of possible playlists.
So there you have it. Mr Brauer doesn’t come to a conclusion of conspiracy, but it can apparently be safely assumed that “shuffle” on your (and my) iPod is NOT actually random.
As much as I would like to think that Apple (or any corporation) was so pure in its intentions that it wouldn’t let something as crass as profit get in the way of purity-of-experience, I’m sure if Sony Music came to Apple with a check and said, “You know…if our songs played a little more often, that would be so cool!” <wink, wink>
~Geek~
Microwave magic
I remember, as some of you also probably do, when microwave ovens arrived on the scene. Magically cooking things without heat, they were all the rage among the early-adopters, but they freaked me out.
I looked into the glass, through the protective screen to try to catch a glimpse of the microwaves being released from (I imagined) their pen in the housing. The little buggers flew pell-mell around the interior making the food heat itself up until it was, well, pasty and dry (ever try to cook a chicken breast in the microwave?).
These days, I still don’t yank the door open while it’s running because of my secret fear that the microwaves can indeed escape, but I do have one in my home and still manage to sleep at night.
Every morning, my coffee ritual involves microwaving about a cup of water until it’s just before boiling (I’ve managed to come up with the right volume of water to be able to hit the 30 second button twice and get what I need), but lately I’ve discovered a new bit of microwave magic!
No matter how many seconds I put the timer on, the turntable always returns my dish (measuring cup) exactly to where I placed it! I never have to reach in the back to get the dish! What amazing microwave technician determined the algorithm that makes this possible?
I may be more impressed with this than with the microwave itself, I don’t know.
~Geek~
It’s all Relative
Handily posted on the back of my M&M’s package, in lovely I’m-a-friend-of-the-earth-green, are the nutrition details for this little bag of magic.
Only 240 calories! 12g of fat! Dude, this is HEALTH FOOD! Must be the peanut. Makes it sort of like a vegetable, sort of like having a bag of green beans.
Wait. 240 calories per serving.
Wait. This is two servings.
Who gets to decide how big a “serving” of M&M’s is?
Bastards. Makes me want to cover my body image issues in an avalanche of peanut M&M’s. That’ll teach ‘em.
~Geek~
BP by the Numbers
Although reports differ, one estimate I read today said that 181 litres of oil had spilled since the BP spill (gush?) happened on April 20.
- Litre-to-gallon conversion is 0.264172051:1
- About 40% of crude oil is converted to gasoline
- So we’ve wasted about 19,126,056 gallons of gasoline*
- It’s 5000 miles from Fairbanks, AK to Miami, FL
- Assuming a conservative average of 20 miles per gallon, a fleet of 38,250 cars could make the round trip from Fairbanks to Miami.
- This is more than the population of Fairbanks. We could have sent them all on a spectacular vacation (in separate cars).
- Sent in Priuses, think of the side trips.
- What else could we have done with this resource?
*Not to ignore or discount the myriad other real costs in lives (11 at the explosion) or natural resources.
Reasons to be forgetful
The graphic description of why I mixed up the name of my friend Judy’s husband with my friend Cheri’s husband. Until this particular error I didn’t even realize that they were filed in the same category.
National Metric Day (What?!)
National Metric Week is the week including October 10th, which is (obviously) National Metric Day! (10/10/10) What?! How have I missed this so long?
How shall we celebrate?! I have only a few months to plan…
~Geek~
Competence Formula
From my pals at Dictionary.com: competency means “sufficiency to satisfy the wants of life.” Isn’t that awesome?
Perhaps a little more specific: competence is “the quality of being competent; adequacy; possession of required skill, knowledge, qualification, or capacity: He hired her because of her competence as an accountant.“ How appropriate that the example given is about someone hired due to competence, given the topic of today’s post.
The most common reason I’m given for developing new-hire or onboarding training is to reduce the time to competence. When I follow with the obvious question (“what is the time to competence now?”) I get all sorts of vague answers. It seems safe to say that very few organizations have operational definitions of competence and then have also gone to the trouble to track the time it takes to get there.
The details of time to competence is for another day. For today, I’m going to say what you already know:
I say you already know this, but sometimes organizations forget it. Specifically they forget that within this formula (if you will), organizations can choose where they want to spend their money:
- Hire experienced employees at a high cost (salary), provide minimal training and they’ll “hit the ground running.”
- Hire less experienced employees at a low cost, provide detailed training and they’ll hit the ground informed, but relatively inexperienced.
- Hire less experienced employees, provide minimal training and expect that they’ll get experience on the job.
I don’t propose that any of these is necessarily the right choice in all instances, but it pays to realize that there are no shortcuts to this rule. One could make the argument that your customers are going to pay one way or another – either through what they pay for your products and services, or what they “pay” for your employees to gain experience through working with them – but it’s unlikely they’re given a choice, so if your organization is operating under #3 above, it’s probably more costly in customer satisfaction (which means repeat business) than you want it to be.
So then…the questions for the day: where does your organization spend its money in this equation? Where should it spend that money? If the answers to these questions are the same, bravo!
~Geek~
Pave the path you’re walking
I’ve said before that design and measurement can’t be separated. The right measurement of anything (training, a coffee cup, this keyboard I’m typing on) is the degree that it does what it’s supposed to do; in other words, what it was designed to do. Assuming that design is about matching the characteristics of a product (training, coffee cup…you get the idea here) to the specific needs of the users of that product, then the needs of those users, as expressed in the design, become the measurement criteria.
Gracious that was mumbo-jumboish. Check out this guy’s photographic description of the difference between the paved paths at UC Berkeley and the dirt paths that people are already walking. His point, excellently illustrated, is that either the campus designers didn’t determine the needs of the population, or the population needs changed. Either way, the paths they paved are in some cases, not the ones the students are walking. The author calls out the university’s solution (a barrier that people keep walking around) as futile and shows ominous pictures of other paths starting to form in the grass.
The point here is obvious. Do you try to force people (users) to the design of your product, or do you try to match your design to their needs. As illustrated by the dirt paths intersecting in different directions from the paved ones, users will find what they want, even if your design attempts to thwart them.
So then you may be thinking “Why should I care, Geek?”
Well I’m glad you asked. In short: beware designing something without spending time analyzing the existing patterns and desires of your target audience otherwise that audience will get its needs met elsewhere.
~Geek~
Mercator Projection
I used to have a professor who ranted (seemingly at random moments in a lecture) that “the map is NOT the territory, people!”
At the time I didn’t care too much what the heck he meant because those inane ramblings never showed up on final exams. Many years later however, I find that this is some of the most useful information I got from my undergraduate time spent. The gist of the message (as I’m interpreting it now) is that the effort of creating a representation of a real-life situation will always result in bias.
This is why there has been so much research into the validity of eye-witness testimony…because while it seem simple that someone on the scene should be able to relay what they saw and what they heard in order to provide an objective account; the truth is that eye-witness testimony is often unreliable.
So today’s lesson is about the subjectivity of reality based on how it’s measured! (yay for subjectivity!)
“The map is not the territory” is nowhere more clear than in how maps are made. Way back in the 1500′s Flemish geographer Gerardus Mercator came up with the most commonly-used method for displaying a sphere (the earth) in two dimensions (on a map). The problem is that most of us (certainly I) grew up believing that the map represented what was true about the earth! Here are some differences:
- Greenland is presented as having roughly as much land area as Africa, when in fact Africa’s area is approximately 14 times greater than Greenland.
- Alaska is presented as having similar or even slightly more land area than Brazil, when Brazil’s area is actually more than 5 times that of Alaska.
- Finland appears with a greater north-south extent than India, although India’s is the greater.
Additionally, the maps I grew up looking at (and still look at) show north at the top. North, of course, is an arbitrary direction associated with one of the two points of the axis on which the earth turns. There is, of course, no “up” in space…so representing north at the top of a map is completely arbitrary. Or, as is more likely, completely self-centered. It is SO not coincidental that most maps that school children see in America have the United States at the top center.
My strong recommendation is that you all go online and order yourself a map with Australia at the top center. Spend some time studying the map. What do you notice that maybe you haven’t noticed before? Personally I don’t think I had ever noticed the Kamchatka Peninsula before but on my revised Australia-centric map, it stands out quite prominently.
The point here is that the choice we make to display or represent a situation will include bias whether we want it to or not. Next time you look at a map or a bar chart or a flow chart or ANY graphic representation…what questions will you ask that help you understand the bias?
~Geek~