The Illusion of Knowledge, ignorant and unaware

From The Invisible Gorilla, the chapter “Should you be more like a Weather Forecaster or a Hedge Fund Manager?” is about the illusion of knowledge. We often think we know why something works but if we were to test ourselves, it would likely expose our knowledge as shallow and incomplete. We are not only ignorant, but unaware.  Of course, in any project the unknowns can kill you, nonetheless,  yet most people will plunge in with superficial knowledge, not pausing to question or take stock of what they really know, what they don’t know, or what they might not know they don’t know.

The example  (page 123) that caught my attention was that of Tim Robers who won thje $25,000 “TopCoder Open” prize. The competitors are given 6 hours to write a program to satisfy a requirements specification. This is very much like our PSP training exercises. Unlike his competition, Roberts spent the first hour studying the requirements and asking questions, apparently quite a few questions. He then wrote a program that satisfied the requirements, but no more.  It worked, that is it satisfied the requirements, and he finished on time. The book authors described this as an investment in time escaping the illusion of knowledge that paid off handsomely. I would describe this as following a good process.

I’ve observed others follow this process. The first step is to understand the requirements. This involves a lot of questions, “is this needed?”, “what is does that really mean?”,  often it involves writing tests. The tests usually step from the “validation” space into the “verification” space and imply a design, or a beginning of taking the problem from the abstract to the concrete.  When you understand the requirements, you can begin a real design. When you have an adequate design specification, you can code. Like the carpenter who measures twice and cuts once, superior developers think twice, write the code, and only run the tests once. The code just works.  It works  because effort was expended to place the project under intellectual control. The process, with steps, and entry and exit criteria,  helps you to do this.

One anecdote from PSP involved a student (actually a professor)  David . The PSP program assignments are designed to be small enough to complet in an afternoon, typical ranges are 2-6 hours.  Upon  receiving the assignment, most students dove right in to programming a solution. David, however,   started thinking about it. While all other students wrote the working code,  David with laptop shut, thought about the problem, made notes, designed a solution. Only after he had thought through the problem and solution did he open the laptop and  begin to program code.  At this point, David was now about an hour behind everyone else (if you take code as your measure of progress). Guess who finished first.

The Invisible Gorilla,  suggests that diving in results from false confidence  and our illusion of knowledge.  Our process is a systematic coping mechanism to  help overcome that illusion.

Sources Cited:

Chabris, C. F., & Simons, D. J. (2010). The Invisible Gorilla, and Other Ways Our Intuitions Deceive Us. New York: Crown.


About Bill Nichols

PhD in Physics from Carnegie Mellon University I'm a software team coach and instructor with the TSP Team at the Software Engineering Institute
This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s