My latest summary is for The Checklist Manifesto: How to Get Things Right by Atul Gawande. It explains how the humble checklist can increase productivity, improve coordination, and even save lives in fields as diverse as surgery, construction, and aviation.
Buy The Checklist Manifesto at: Amazon | Kobo (affiliate links)
Key Takeaways
- Checklists are incredibly underrated tools that can make a big difference and save lives.
- Our knowledge base has grown increasingly complex.
- In complex situations, it becomes much easier to overlook the basic but important things.
- One way to deal with this increased complexity is through specialisation. This is what medicine has done so far.
- Another way is the checklist. This is what aviation and construction have done.
- Advantages of checklists:
- Ensure people take care of the simple but critical things;
- Make it easier to roll out new best practices quickly;
- Help identify problems and patterns of mistakes; and
- Improve coordination in teams and systems.
- Why are checklists so underrated?
- People think they are too bureaucratic and replace human judgement. (But a good checklist should enable, rather than replace, human judgement and skill, by freeing up the mind so it doesn’t have to worry about small, routine things.)
- Simple arrogance — people don’t believe they need a checklist.
- Checklists are not very exciting or sexy.
- A good checklist should be:
- Precise — it must be clear whether a step has been completed;
- Short — it should not be a “how to guide”; and
- Practical and easy to use — real-world testing is essential.
- Checklists are not a panacea:
- For a checklist to be effective, people have to use it. This may require a major cultural shift.
- Checklists cannot substitute for human judgement in complex situations with high uncertainty.
- Different fields which have successfully deployed checklists include: medicine, aviation, construction, restaurants, and investing.
Detailed Summary of The Checklist Manifesto
Our knowledge base has grown increasingly complex
Gawande draws a distinction between two types of failure:
- Failures of ignorance are those that occur because we had only a partial understanding of how the world works
- Failures of ignorance are those where where we know what to do, but fail to apply it correctly.
For most of human history, we largely failed due to ignorance. But over the last several decades, science has developed so much that failures due to ineptitude have increased significantly.
We judge failures of ineptitude more harshly than failures of ignorance
We’re usually satisfied if someone does their best in a situation where the knowledge on how to fix a problem doesn’t exist. Yet we’ll get infuriated if the knowledge on how to fix the problem did exist, but was not applied correctly. This can be unfair. As our pool of knowledge increases, there is more and more to manage and get right.
Superspecialisation
One answer to this increasing volume and complexity of knowledge has been increasing specialisation.
Superspecialisation in medicine
To practise medicine in the early 20th century, high school graduates just needed a one-year medical degree. By the end of the century, doctors needed:
- a college degree;
- a 4-year medical degree; and
- a further 3 to 7 years of residency training in an individual field of practice (e.g. paediatrics, surgery, neurology).
In recent years, there’s even more. After their residencies, most doctors do fellowships, adding another 1 to 3 years of training. So doctors typically won’t start independent practice until mid-thirties.
Medicine has gone beyond mere specialisation to superspecialisation, especially in surgery. The role of an anaesthesiologist is just to handle pain control and patient stability. Yet even they have divided into paediatric anaesthesiologists, cardiac aanesthesiologists, obstetric anaesthesiologists, and so on.
Increasing levels of specialisation are not unique to medicine. In many fields, it takes longer to train than ever before.
Gawande does not dispute that specialisation has worked and improved the overall level of capability. But basic, avoidable failures persist. There is so much complexity that even the most superspecialised practitioners cannot avoid daily mistakes.
Advantages of checklists
Gawande gives many examples illustrating the various advantages of checklists in different contexts throughout the book. I’ve attempted to summarise those advantages here as follows:
- They ensure people take care of the simple but critical things;
- Makes it easier to roll out new best practices quickly;
- Helps identify problems and patterns of mistakes; and
- Improves coordination in teams and system.
Ensure people take care of the simple but critical things
People — even experts — make mistakes. Our memory and attention are fallible, especially when it comes to mundane matters. It can also be tempting to skip steps that usually don’t matter. For example, most of the time a safety check will not uncover a risk.
Checklists provide protection against such mistakes by making the minimum necessary steps explicit. They can help experts remember the most critical steps and be more disciplined about performing them.
Surprisingly, by making the minimum necessary steps explicit, checklists can even increase efficiency. Gawande gives the example of a value investor whose team uses a checklist to vet possible investment opportunities. Initially, the investor thought the checklist would slow his team down, but it turned out to do the opposite. Not only did the checklist help them make better investment decisions, it enabled them to evaluate many more investment opportunities in less time. [I didn’t find this surprising — it makes perfect sense to me. A (well-designed) checklist avoids time wasted trying to figure out what to do and stops you going around in circles.]
Roll out new best practices quickly
In complex fields like medicine or aviation, it can be hard for experts to keep up with all the changes in “best practices” on top of doing their normal jobs. One study reported it took doctors 17 years on average to adopt major new treatments for at least half of American patients.
Checklists can help incorporate new “best practice” knowledge by translating it into a simple, usable and systematic form.
Example: New aviation procedures to deal with polar ice
In September 2008, the US Federal Aviation Administration (FAA) published new procedures for pilots to follow to deal with ice on polar flights following a crash caused by it.
The new bulletin was incredibly dense and detailed. Professional aviation checklist makers worked to distil the information into a practical, usable form, and then refined it after testing. Pilots had access to the new checklist about a month after the FAA published the new procedures.
Two months later in November, the same disaster almost happened again. But this time, the pilots could follow the checklist and averted a crash.
Identify problems and patterns of mistakes
Checklists help uncover problems that we’d otherwise miss. One way it does this is by bridging the gap between those doing the work and those overseeing it. For example, when hospitals started using Pronovost’s central line infection checklist, the executives discovered that less than a third of the ICUs had the required soap (chlorhexidine soap) and that the sterile drapes needed to cover up patients were often unavailable.
Example: Van Halen’s brown M&Ms
Van Halen was the first band to stage huge productions in small, third-level markets. Their contract rider was incredibly long, with detailed requirements involving a lot of equipment and people.
Buried in the middle of the rider, Van Halen inserted a clause demanding a bowl of M&Ms without any brown ones. That was a test. If they didn’t find brown M&Ms in their bowl, they knew their instructions had been carefully read and followed. But if they did find brown M&Ms, they’d have to check everything and would inevitably find a technical error.
Another way checklists can help uncover problems is by forcing us to confront our repeated mistakes. Few of us bother to examine our mistakes and look for patterns in them. For the most part, we don’t think that we could’ve done much more other than perhaps work a bit harder. But to devise a good checklist, you need to look at past errors and try to come up with a plan for them. The checklist can then help you guard against those mistakes in the future.
Cocaine brain in investing
In investing, an attractive opportunity can get us excited and set our “cocaine brain” off into greed mode. Neuroscientists have found that the prospect of making money stimulates the same primitive reward circuits in the brain that cocaine does.
One investor explains how, when that happened, no matter how objective he tried to be, he’d find himself vulnerable to confirmation bias. His brain would latch onto evidence that confirmed his initial hunch and dismiss any contrary evidence. Checklists can reduce this bias and help investors become more systematic and dispassionate in their analyses.
Improve coordination in teams and systems
Much of modern life involves systems. As complexity increases, we have to rely on a greater division of tasks and specialisation. It’s no longer enough for one person to do a good job; a whole collection of diverse components needs to come together effectively.
Checklists can play a useful coordination role, especially in teams that don’t know each other, or where there are hierarchies that inhibit team members from speaking up.
Checklists set up clear lines of accountability by establishing who is responsible for what, with some critical responsibilities subject to cross-verification. Dispersing responsibility can send the message that everyone is responsible for the overall success of the task and has the power to question the process. The most common obstacle to effective teams is people sticking narrowly to their specialised domains, and thinking “That’s not my problem”. This makes it easy for things to fall through the cracks.
Example: Surgical teams
Many surgical teams in large hospitals have never performed an operation together before. Gawande’s hospital, for example, has 42 operating rooms staffed by more than 1,000 people. Teamwork is critical, but unpredictable.
The surgery checklist that Gawande developed included two key steps:
- Introduce everyone’s name and role at the start. Unsurprisingly, many studies show that people who don’t know one another’s names don’t work together nearly as well as those who do.
- Talk to each other about each case for at least a minute before starting. This seems to increase people’s sense of participation and responsibility, as well as their willingness to speak up later.
Though Gawande admits these steps can feel “hokey”, and often proved controversial, he comes to appreciate how important the two steps are. Researchers found that, not only did these steps increase staff’s average rating of teamwork (from “good” to “outstanding”), they also increased employee satisfaction by 19% and reduced staff turnover.
Why are checklists so underrated?
Too bureaucratic and replaces human judgement
A common objection to checklists is fear of rigidity. After all, checklists are a tool that can centralise decision-making, allowing authorities to dictate to the workers below exactly how to do things. People start imagining mindless automatons blindly following a checklist drawn up by some clueless bureaucrat, instead of responding to real world situations. They may also worry that the checklist will be too long and time-consuming — most people don’t want to fill out more paperwork. [Kahneman, Sibony and Sunstein encountered similar issues when they tried to get people to make decisions more systematically — see Objections to Reducing Noise.]
Gawande argues that a well-made checklist should not replace human expertise and judgement but should instead enable it. It can do this by getting the stupid but critical stuff out of the way, and leaving room for experts to manage complex, unpredictable circumstances. A good checklist should strike a balance between freedom and constraint.
[Checklists] provide a kind of cognitive net. They catch mental flaws inherent in all of us—flaws of memory and attention and thoroughness. And because they do, they raise wide, unexpected possibilities.
The construction industry is a good example of how checklists can enable decentralised, democratic decision-making when unpredictable problems crop up. The communication checklists there don’t dictate how to solve the problem, it just forces the relevant people (the experts) to talk and resolve it together.
Communication checklists in construction
Skyscraper builders run into thousands of difficulties that you can’t foresee when designing the construction schedule. The construction industry deals with unexpected problems using a communication checklist (aka “submittal schedule”).
While it may be impossible to foresee what problems would arise, they could anticipate when and where they could arise. The submittal schedule made sure relevant experts spoke to each other at appropriate points in the process. For example, by the end of the month the contractors, installers, and elevator engineers had to review the condition of the elevator cars. This allowed room for expert judgment, but as part of a group.
Simple arrogance
Some workers feel offended by the suggestion that they need a checklist. Moreover, basic safety checks will feel pointless most of the time.
Despite being a big proponent of the checklist, Gawande confesses that, in his heart of hearts, he didn’t think his safe surgery checklist would make a difference in his surgeries. But when he actually used it, he couldn’t get through even a week without the checklist catching some error he would have otherwise missed, and he’s seen it save a patient’s life.
Not very exciting or sexy
In the 1950s, flying was incredibly dangerous, and pilots were seen as sort of ‘rock stars’ (see e.g. Chuck Yeager). The danger gave pilots status. But as checklists and flight simulators became more prevalent, the risks diminished, as did pilots’ rock star status.
Staff were often resistant to Gawande’s safe surgery checklist in the hospitals where they tested it out. While most staff came around after seeing the checklist in action, some surgeons still remained sceptical. Gawande contrasts the humble checklist with remote-controlled surgical robots. Surgeons and hospitals got far more excited about the latter, even though they only showed modest improvements for a few operations and cost millions of dollars. Unfortunately, checklists are just not very exciting or sexy.
What makes a good checklist
Good checklists are: precise, short, and practical/easy to use.
Precise
Each item in a checklist should be clear and unambiguous. Bad checklists are vague and imprecise, making it hard to tell whether or not a step is completed.
In a team context, it must be clear who is meant to stop things and run through the checklist. For example, in aviation, the “pilot not flying” starts the checklist because the “pilot flying” may be busy with flight tasks. When Gawande’s team set about making a surgical checklist, they made the circulating nurse responsible for the checklist. This elevated the nurse’s role in a surgery.
The language used should also be familiar to those in the profession.
Short
Bad checklists are too long. They treat the people using them as dumb, and try to spell out every single step. Such checklists end up being more distracting than useful.
[Bad checklists] turn people’s brains off rather than turn them on.
A rule of thumb is to keep your checklist between 5 to 9 items (the working memory limit), but you don’t have to be too religious on this. Ideally, the checklist should fit on one page and be easy to read. There shouldn’t be any clutter or unnecessary colours.
Pause points
Pause points are the moments when people should refer to the checklist. At the outset, decide what type of checklist you want:
- DO-CONFIRM. People do their tasks from memory and experience. Then they hit a pause point where they’ll look at the checklist and confirm that they did everything they were supposed to.
- READ-DO. People carry out the tasks as they check them off—it’s more like a recipe.
In making a checklist, you should define clear pause points. For example, in a surgical checklist, pause points might include: before anaesthesia, before incision, and before leaving the operating room.
Around 60-90 seconds after a pause point, a checklist starts to become a distraction and people will start skipping steps. So you want to keep the list short.
Be ruthless — you may need to cut out important steps
A common mistake is to think of a checklist as a “how-to” guide, spelling out all critical steps. But if a critical step is one that people virtually never miss in real life, it doesn’t need to be on the checklist.
Moreover, there are many problems that a checklist cannot anticipate, especially in complex situations. In those cases, the better option may just be to make people stop and talk it through as a team.
A good checklist should just remind people of the most critical steps that even highly skilled professionals can overlook. The checklist is a tool to buttress the expert’s skills, not to replace them. Cutting out important steps can be the most difficult part of the exercise. This may just come down to a judgement call.
Example: Operating room fires vs Operating on wrong patient
When developing their surgical checklist, Gawande’s team had wanted to include steps to deal with operating room fires. Fires were notorious problems, and only a few steps were needed to reduce harm if a fire broke out.
But there were only about 100 surgical fires in the US each year (and far fewer fire-related fatalities), compared to 300,000 surgical site infections. And the fire checks would have made the list substantially longer. So they ended up dropping it.
However, Gawande points out that operating on the wrong patient or the wrong side of the body is also exceedingly rare. But they kept those checks in because they were relatively quick, accepted in many countries, and the mistakes get a lot of attention when they occur.
Practical and easy to use
Bad checklists are impractical and hard to use. They might be drafted by “desk jockeys” who don’t understand the situations in which the checklist is used.
Good checklists should be easy to use even in the most difficult situations. No matter how carefully you design a checklist, you have to test it in the real world, because it’s always more complicated than you’d expect. Before you deploy a checklist, you must test and refine it.
You may need to keep refining it after you’ve deployed it, too. Even simple checklists require frequent updates and ongoing refinement as the world keeps changing and our knowledge base grows. Airline manufacturers for example put a publication date on all their checklists.
Checklists are not a panacea
For a checklist to be effective, people have to use it
A checklist by itself cannot make anyone follow it. Pilots follow checklists because they’re trained to do so, starting in flight school. But when people have not been so trained, incorporating a checklist can be a major cultural change. It can also come with a learning curve — even when people wanted to use the checklist, they might forget occasionally.
In the case of Gawande’s safe surgery checklist, they carefully chose the first groups to use it. They picked people who had the patience to make any changes needed to get it to work, as well as the seniority to get others on board.
It also helped that the checklist worked. Some of the checklist’s toughest critics became its most avid proponents when they saw the checklist prevent an error. An anonymous survey after 3 months in the test hospitals found that 78% of staff had seen the checklist prevent an error. When staff were asked if they wanted the checklist to be used if they were having an operation, 93% said “yes”.
Checklists cannot substitute for human judgement
Checklists can help us with some, but not all, kinds of problems. There’s a distinction between simple, complicated and complex problems:
- Simple problems. For example, baking a cake from a mix. If you follow the recipe correctly, there’s a high chance of success.
- Complicated problems. For example, sending a rocket to the moon. The recipe is not straightforward — it requires coordinating multiple teams and unanticipated difficulties will arise. But you can still repeat the steps and get better at it.
- Complex problems. For example, raising a child. Every child is unique, so the degree of uncertainty is high. Expertise is valuable but not sufficient.
Checklists help protect us against simple problems. In medicine, that might be failing to put on a mask or wash your hands. In law, it may be missing a court deadline.
You want people to make sure to get the stupid stuff right.
By solving the simple problems, checklists leave room for expert judgment to deal with the complex ones.
Checklists in different fields
Medicine
Modern medicine is extremely complex — not just in the breadth of stuff you have to know, but also in how to execute that knowledge.
Gawande describes several checklists successfully used in medicine, before explaining his own efforts to come up with a “safe surgery” checklist:
- Central line infection checklist. Developed in 2001 by Peter Pronovost, this was the first doctor checklist. It included 5 incredibly obvious steps that had been taught for years, yet doctors routinely skipped at least one step in over a third of patients. After it was deployed, infection rates dropped substantially, averting many infections and deaths. When the state of Michigan adopted the checklist in its hospitals, within 18 months, the hospitals saved an estimated $175 million in costs and more than 1500 lives.
- Antibiotic checklist. The most effective way to prevent surgical infections is to give patients an antibiotic in the 60-minute window before the incision. Around 30% of the time, this step is missed because of all the stuff going on as a patient is prepped for surgery. The Columbus Children’s Hospital developed a simple two-step checklist to make sure patients got the antibiotic on time. After 10 months, there was 100% compliance.
Safe surgery checklist
Much of the book describes Gawande’s team’s attempts to come up with an effective “safe surgery” checklist for the World Health Organisation. Their initial checklists were too unwieldy and had to be refined. The final checklist was a DO-CONFIRM list containing 19 checks.
They tested the checklist in 8 hospitals around the world — 4 in high-income countries (the US, Canada, the UK, and New Zealand); and 4 in low- or middle-income countries (the Philippines, Jordan, India and Tanzania). Before using the checklist, hospitals missed at least 1 of 6 basic safety steps in around 67% of patients (regardless of whether it was in a rich or poor country).
Afterwards:
- the rate of major complications fell by 36% in all 8 hospitals;
- infections fell by almost half;
- deaths fell 47%.
Gawande argues the checklist has a lot more untapped potential in medicine. There could be hundreds more checklists for procedures beyond surgery, as well as for specialised procedures and emergency situations, like in aviation.
Aviation
Flying a modern plane is incredibly complicated, yet only 1 in 500,000 flights ever suffers an accident of any kind. The reason is the widespread use of checklists.
Boeing issues over 100 aviation checklists per year (new and revised), put together meticulously by experts. The checklists are simple and short enough to fit on an index card. There are checklists covering both routine aircraft operations (e.g. take-off, landing, taxiing) and emergency situations — pretty much every conceivable emergency a pilot may encounter.
Construction
For most of modern history, one individual — the Master Builder — would design, engineer and oversee construction of a building from start to finish. As construction grew more complex, it became increasingly specialised. When building a skyscraper, there can easily be between 200 to 500 workers on-site daily, from dozens of different subcontractors. Projects today might involve 16 different trades — plumbing, ventilation, wiring, and so on.
But despite all this complexity, building failure rates are incredibly low, especially for skyscrapers, because they follow construction schedules. Construction schedules are like giant checklists. They set out every building task that needs to be accomplished, in what order, and when. This ensures that the various different tradesmen work together in right way, and in the right sequence.
When tasks were completed, the job supervisors would tell the project executive, who would check it off in his computer scheduling program. The project executive would then post a new printout for the next phase of work at least weekly.
Every skyscraper is unique with a vast range of factors to account for, so there’s no textbook formula to follow. Representatives from each of the different trades would contribute to an initial construction schedule, and subcontractors and other independent experts would then double-check it.
Restaurants
Professional kitchens make use of three types of checklists:
- Recipes. This is the most basic checklist. Staff don’t always love following recipes, but professional kitchens require discipline and consistency.
- Customer checklists. When a customer places an order up front, it will be printed on a ticket back in the kitchen, specifying the dishes ordered, table and seat numbers, allergies, preferences or other special notes. Either the chef or sous chef will review every plate before it’s served.
- Communication checklist. Some kitchens also have communication checklists to ensure people dealt with unexpected problems as a team — e.g. staff may have a quick daily “pow wow” 30 mins before opening to run through any unanticipated issues and concerns.
Investments
The book featured several “value investors” who use formal checklists to manage large portfolios. Warren Buffett himself apparently uses a ‘mental checklist’ when looking at potential investments.
Mohnish Pabrai, who managed around $500 million, found that a mental checklist wasn’t enough so he devised a written one. He based it on the mistakes he’d seen and a matching list of checks — around 70 in all. The checklist helped him filter out a lot of investments that initially looked exciting, but less so after a careful, dispassionate analysis. Pabrai estimated that he investigates around 10 or more prospects in-depth for every investment he actually makes.
Geoff Smart, a PhD psychologist studied 51 venture capitalists to see how they made their decisions, and tracked their success over time. Some venture capitalists assess an entrepreneur instinctively based on their gut, while others spend a lot of time gathering information (but still end up going with their gut). There were many different styles.
Smart found the most successful venture capitalists were those that applied a methodical, checklist-driven approach — they had a median return of 80%, while the others were 35% or less. But most investors were intuitive decision-makers, not systematic ones. And the publication of Smart’s findings, more than a decade ago, did nothing to change that.
My Thoughts
The Checklist Manifesto started out as a New Yorker article , “The Checklist“. Even though I enjoyed the read, I’m not convinced there was enough content to justify a whole book. It comes in at 208 pages, which is on the short side, and at times it felt like Gawande was trying to stretch things out. Several of his stories, particularly the medical ones, seemed to me unnecessarily long and complicated.
I did like the aviation and construction examples, though. Often I think about how complex the world is and all the problems that we face — information problems, principal-agent problems, externalities, poor feedback loops, communication difficulties, and so on. It’s actually pretty amazing how well the world works despite all that. Like, we don’t have buildings or planes falling down all the time — which is nice. I also liked the fact that these examples served to illustrate the benefits of range, with Gawande taking best practices from other fields to apply to his own field (surgery).
A few thoughts from me on why checklists are so underrated:
- I think “checklist” is bad name. It suggests a pro forma box-ticking exercise, with no room for discretion. The “list” part in particular conjures up images of annoying paperwork to fill out. I don’t have any great suggestions for a better name, but even “safety check” would be better.
- When a check is “unnecessary” the vast majority of the time, it’s very tempting to skip it. I confess that when I’m rock climbing, I often neglect to check my partner’s tied in correctly because over 99% of the time it’s unnecessary (assuming it’s my regular climbing partner). So even though I understand and accept the benefits of doing such a check, the in-built feedback mechanism is weak — there’s no “reward” for checking, nor any “punishment” for not checking.
- I suspect a huge reason why workers are so resistant to checklists is because they’ve been too scarred in the past by crappy checklists or other poorly-thought-out upper management interventions. So the problem might be that bad checklists are all too common, rather than that checklists in general are underrated.
To that end, I thought the best parts of the book were the parts where Gawande talked about how to develop a good checklist and to get people to use them. I just wish those parts had been longer.
Buy The Checklist Manifesto at: Amazon | Kobo <– These are affiliate links, which means I may earn a small commission if you make a purchase through these links. I’d be grateful if you considered supporting the site in this way! 🙂
If you enjoyed The Checklist Manifesto, you may also like: