How long should checklist or a process script be?

The short, and flip,  answer is as long as it needs to be, but no longer and as short as you can make it, but no shorter.  However, I like to keep it to a page so that it is both usable and lean.

I recently read Checklist Manifesto by Atul Gawande. I didn’t expect to learn much because we’ve been doing this for years, but I was surprised at how good  Dr. Gawande’s treatment was.  He told a number of great stories and presented that provided insights I had never considered and did some very good historical research.   I’ll save these for another time. Today, I want to nitpick on a bit of wrongheaded advice about the size of a checklist.

A rule of thumb some use is to keep it to between five and nine items, which is the limit of working memory. Boorman didn’t think one had to be religious on… (page 123)

Really?  I’d read the original source, The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information, Psychological Review 63 (2): 343–355. and it had seemed to me a good reason to make a checklist, not a limit on that checklist.

So I did some research and found out Miller had addressed this issue directly in a correspondence with Mark Halpern. ( Halpern wrote to Miller seeking help with what he considered to be a misreading of Miller’s original paper

The director of the technical writing group which I serve as editor has issued an edict 
that lists and procedures in our printed and screen-displayed documentation should
 not exceed seven, or maybe nine at the most..”

Miller wrote back,

“the point was that 7 was a limit for the discrimination of unidimensional stimuli 
(pitches, loudness, brightness, etc.) and also a limit for immediate recall, neither of 
which has anything to do with a person's capacity to comprehend printed text”

I also found this story with additional commentary on documentation guru Edward Tufte’s web site. . It’s interesting reading.

There are other good reasons to keep the checklist short, but the number 7 research  is relevant neither to document design nor to how people comprehend printed text.


About Bill Nichols

PhD in Physics from Carnegie Mellon University I'm a software team coach and instructor with the TSP Team at the Software Engineering Institute
This entry was posted in Uncategorized. Bookmark the permalink.

One Response to How long should checklist or a process script be?

  1. jdmchale says:

    Another more practical reason for about 7 items on a checklist, for example in a PSP-style personal review checklist, is that you begin to hit diminishing returns. If 80% or 90% of your coding defects are caught by 5 or 6 or 7 checklist items, is it really worth the trouble to get the last 10% to 20%? Probably not. (Pareto analysis = GOOD!)

    The other practical limit is on how long you can actively concentrate using such a checklist. Again from PSP, if you are reviewing at a nominal rate of 200 LOC/hr. and have 5 to 7 review checklist items and 300 LOC to review, that’s going to be a fairly intense 90 mins. of eye-crossing concentration, but it is doable with just a little bit of practice doing real defect-finding reviews with a data-derived checklist. It works out to something like 1500 and 2000 LOC/min/item, a rate which implies for most people that they are not reading but rather scanning and looking for relevant bits of code for that particular checklist item, e.g. off-by-1 errors in a loop-control statement – you can skip all non-loop-control statements for that item – or balancing parentheses – you can skip all statements without parentheses. Adding another half-dozen items makes it take twice as long on average, and 3 hour personal reviews (or any other kind of review) doesn’t work very well at that length.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s