If you’re a fan of the HBO show The Wire, “juking the stats” would be a familiar concept. In the show, Baltimore city cops – under pressure from management to improve crime numbers – resort to short term tactics that get better numbers but don’t necessarily reduce crime. Reclassifying crimes to lower categories, increasing the arrest rate by arresting for minor offenses, under reporting crimes are all part of the play book. And as Pryzbylewski – a former cop who becomes a teacher – later finds out, the same story repeats itself in the city schools. Under pressure from the state, to improve standardized test scores teachers focus on teaching for the tests rather than actually educating their students.
Juking the stats is however, not just a great sound bite on a TV show. It is an all too real issue that plagues organizations – public and private sector alike. Performance measurements introduce perverse incentives and it is human nature, when measured, to optimize for the metric against which they are being judged.
The world of software engineering is no stranger to this problem. Software engineering and its management is a complex beast and relative to other engineering disciplines is still in its infancy. We are still figuring out effective ways to track and measure performance. Most methods are far from perfect and suffer from unintended consequences.
In some agile organizations – especially those that are new to agile – measuring team performance by their sprint velocity has become common practice. Far too often, this leads to teams – under pressure to deliver the committed story points in that sprint – unintentionally cutting corners on critical aspects like quality and testing only to pay the price later.
Large engineering programs require teams to report status on a weekly basis, typically as red, yellow or green or some variation thereof. The stigma attached with reporting one’s status as red can lead to teams suppressing problems. Being honest about these issues ahead of time could have fixed those issues, but the pressure to not report red, means these issues remain buried until it’s too late.
In less mature organizations, QA teams are sometimes incentivized by the number and priority of bugs that they open. This invariably leads to bug priority inflation and battles with the development teams. Low team morale is an inevitable side effect.
Then, there is the possibly apocryphal tale of IBM incentivizing programmers by lines of code only to result in programmers intentionally writing verbose code.
In all of these cases, you see teams when pressured by poorly designed incentives and metrics, lose sight of the long term goals and focus on the short term statistics – sometimes overtly, but usually inadvertently. Qualitative attributes like software quality, good design and resilience end up taking a back seat. Measuring and tracking performance is a good thing and is essential for continuous improvement. However, it’s just as important to be aware of the possibility that more often than not, unintended consequences may rear its ugly head. When it does, it is imperative that leaders react and be prepared to either fix the metric or dump them entirely.
“Don’t matter how many times you get burnt, you just keep doin’ the same.” – Bodie
- If you’re not, you should be. Apart from having a great storyline and an excellent cast of characters, it is rich with lessons in economics, management and human behavior. ↩
- Crime Report Manipulation Is Common Among New York Police, Study Finds – NYT ↩
- Criticism for standardized testing as a measure for driving school funding as collected by Wikipedia ↩
- There are a number of examples of the unintended consequences of perverse incentives at play. One of the more interesting examples from Bill Bryson’s A Short History of Everything is the story of paleontologists paying the locals for each fossil fragment they turn in. The paleontologists later find that the locals were smashing larger bone fragments into smaller pieces to maximize gain and in the process rendering the fossils worthless. The authors of Freakonomics collect a few more examples in this NYT article. ↩
- Joshua Kerievsky makes an interesting case for doing away with story points. ↩
- albeit imagined ↩
- Epigraph from “Time after Time”, season 3, episode 1 of The Wire ↩
The Trouble With Harvard: The Ivy League is broken and only standardized tests can fix it
Dear Novak, Love Roger: Novak Djokovic and Roger Federer exchange emails during the US Open Final – another masterpiece by Brian Phillips.
Why You Should Refrigerate Tomatoes and Ignore Anyone Who Says Otherwise: Serious Eats at it again, challenging conventional culinary wisdom with experimentation and data.
I recently read Lawrence in Arabia: War, Deceit, Imperial Folly and the Making of the Modern Middle East – Scott Anderson’s history of the Middle East circa World War I with T. E. Lawrence as the central character. While the book revolves around Lawrence, Anderson also introduces a coterie of peripheral characters who were just as influential in shaping the history of the time.
Unlike what you might expect from a book on history, the book maintains a breezy pace and is a page turner. Anderson also presents a balanced appraisal of the characters involved – Lawrence in particular, which given his legendary status couldn’t have been easy. What makes the book particularly engrossing and relevant is how consequential those events that occurred nearly a hundred years ago were – you could draw a direct correlation between the events described in the book to the state of the Middle East and may be even the world today. A must read for a history buff and highly recommended even otherwise.
Great to see Federer back in top form against Del Potro. Not all of it was pretty, but there were moments where the Federer of old shone through. Next up Nadal. My prediction: they split the first two sets and Nadal runs away with the third, though I’ll be rooting for Federer all the way.
While I’m on a constant quest to cut down on how much TV I watch, I have added a couple more shows to my Netflix Instant queue. I have always loved good crime television. I tried getting my fix with the various Law and Order’s for a while and I hate to admit that I even watched CSI:Miami for a couple of seasons. But those shows never really did it for me.
What I miss the most is the classic whodunit. Agatha Christie’s Poirot – the David Suchet version – was an all time favourite (season 1 is now on Netflix Instant). I recently came across Inspector Lewis imdb netflix on PBS’ Masterpiece Mystery and it is quickly becoming one of my top shows. I’m not quite sure if I think it betters Poirot, but it is close. The setting is the university city of Oxford where Lewis and his partner Hathaway solve a murder or two – usually two – every episode. The dynamic between the down to earth Lewis and the university educated Hathaway, the unique cast of suspects and victims plucked from the world of academia and clergy are what keeps the show going. I also can’t help but feel that Oxford adds a touch of class to what is otherwise the gory world of crime.
The other show is Intelligence imdb netflix based in Vancouver. I like to think of it as the Canadian Wire, though it’s not as nuanced as The Wire. Also unlike the Wire, it has surprisingly little violence. Intelligence is instead about the battle for information between law enforcement and organized crime. Like most modern crime shows, there are no heroes or villains – the good are not all good and the bad are not all bad. Not only are the cops and the criminals fighting each other using a network of informants and moles, they are also fighting amongst each other. This constant struggle more than makes up for the lack of violence.
These are two great crime shows with very different characteristics. While Inspector Lewis enjoys patronage by PBS in the US, it’s a pity that Intelligence hasn’t been broadcast in the US. It’s a show that might have done very well on HBO or Showtime. Well, at least they’re both on Netflix Instant.
According to the calculations of Frey and Stutzer, a person with a one-hour commute has to earn 40 percent more money to be as satisfied with life as someone who walks to the office. Another study, led by Daniel Kahneman and the economist Alan Krueger, surveyed nine hundred working women in Texas and found that commuting was, by far, the least pleasurable part of their day.
That should have made me downright miserable, considering that my new job has increased my commute by about 4000%. I went from walking across my street to driving about 40 minutes each way, in gnarly LA traffic no less.
Luckily, I’ve discovered podcasts – albeit a decade too late – to keep me sane. I’ve mostly stuck to Martin Fowler’s recommendations with the addition of Freakonomics radio. By far the most consistently interesting podcast I’ve come across is RadioLab. The episode Detective Stories with three stories around digging up the past is particularly captivating.
While I believe that podcasts have ensured that my happiness has not dropped or may even have increased, they’ve brought problems of their own. There are times when I realize that I’ll be reaching my destination before the end of an episode, resulting in me quixotically willing the traffic lights to turn red and hoping that my commute was a wee bit longer. File that one under first world problems.
I’ve tentatively settled on Google’s Listen app as my podcast delivery vehicle of choice on Android. It’s buggy and not supported any more by Google, but it syncs with my Google Reader account. I would happily pay for the BeyondPod app if I could figure out how to sync a specific folder within Google Reader.
Sidebar: While searching for the study linking happiness to commute lengths, the first result was Jonah Lehrer’s post (linked above). Turns out that Jonah is also the contributor to RadioLab and contributed to my other favorite episode which randomly enough happens to about the pervasiveness of randomness in our lives.
 Developments in the Measurement of Subjective Well-Being by Kahneman and Krueger
The new version of my RedditTV Boxee app just went live.
RedditTV features the best videos on the web as voted up by the reddit community. The boxee app sources videos from reddit.tv (developed by another reddit enthusiast). This version lets you toggle videos between full screen and a smaller view skip back and forth between videos.
You’ll need Boxee 0.9.21 or higher for the updated features. If you don’t already have boxee download it now and install redditTV for hours of entertainment. If you already have redditTV and boxee 0.9.21, you should get the upgrade automatically.