This Day On The Street
Continue to site
This account is pending registration confirmation. Please click on the link within the confirmation email previously sent you to complete registration.
Need a new registration confirmation email? Click here

Salmon: Annals of Quantitative Overconfidence, Boeing Edition

NEW YORK ( Reuters Blogs) -- On Jan. 7, the auxiliary power unit (APU) of a Boeing 787 caught fire at Boston's Logan airport. The APU is a lithium-ion battery, roughly one-foot cube, and the consequences of a fire can easily be catastrophic. There was no one on the plane at the time, which is lucky, because the fire was extremely difficult to extinguish, with firefighters encountering "no visibility" thanks to thick smoke. What's more, the "quick-disconnect knob" was melted. In flight, these batteries control critical flight systems: they cannot fail.

And yet, twice in 58,000 hours of usage, the lithium batteries on the new 787 contrived to catch fire; this is obviously not something the FAA -- or even Boeing, for that matter -- can risk happening again. There's really only one thing to be done: All lithium batteries on the 787 must be swapped out for nickel-cadmium or lead-acid batteries, which have the great advantage that they don't catch fire.

The bigger story here, however, is about engineers' hubris and regulatory capture. As the interim report from the National Transportation Safety Board says, the FAA was well aware, when Boeing said it wanted to use lithium batteries, that such batteries are inherently dangerous and have a tendency to catch fire whenever they are used elsewhere.

But Boeing persisted, and came up with some hilariously overprecise probability estimates. The batteries would only emit gas or smoke once every 10 million hours, the company calculated, and would only catch fire once every billion hours. The reasoning is bonkers: Boeing's analysis "determined that overcharging was the only known failure mode that could result in cell venting with fire." They then contrived to conclude that if they put in overcharge protections, the risk of overcharging would be brought down to one in a billion, and that therefore the risk of a fire would also be brought down to one in a billion.

As Steve LeVine notes, Nassim Taleb would take one look at that reasoning and simply laugh. For one thing, how on earth is it possible to determine that the risk of an overcharge is less than or equal to one in a billion? Probabilities that small simply can't be measured. And more importantly, how did Boeing determine that the probability of a fire absent an overcharge was zero? There's good evidence that neither of the battery fires were caused by an overcharge -- but Boeing seems to have decided that fires caused for any non-overcharge reason were, literally, impossible. Once again, it's incredibly hard to conceive of any coherent line of reasoning which could come to that conclusion.

But somehow the FAA accepted Boeing's analysis at face value, and allowed Boeing to install lithium batteries on its planes, just as long as certain safeguards were put in place. This is the same kind of literal quantitative thinking which helped cause the financial crisis. Put engineers in charge of something, and they'll measure what they can measure, they won't measure what they can't measure, and they'll protect against only the things they managed to foresee. And as all of us who spend our lives surrounded by electronic devices know, sometimes they fail. In a sense it doesn't matter what the reason is: failure is just a fact of life, which is a real problem when failure could mean the fiery death of hundreds of people.

Statistically speaking, airplanes are safer today than they've ever been. And electronics are a key part of that trend: they might occasionally fail, but they are also increasingly good at preventing human error, or just at doing the things that fallible humans used to do, only much more reliably. That said, as airplane engineers stop being grease monkeys and start being coders, we're losing a certain amount of holistic and heuristic understanding of how to ensure real-world safety.

If you basically outsource an entire airplane, as Boeing did, you lose your institutional ability to ensure that airplane is safe. And sadly, it seems that Boeing's failures on that front will automatically cascade down to the FAA. The reports and post-mortems surrounding the lithium batteries' safety will be very deep. Let's hope the FAA is just as critical when it comes to its own decision to accept Boeing's analysis at face value.

-- Written by Felix Salmon.

Read more of Felix's blogs at Reuters.

Check Out Our Best Services for Investors

Action Alerts PLUS

Portfolio Manager Jim Cramer and Director of Research Jack Mohr reveal their investment tactics while giving advanced notice before every trade.

Product Features:
  • $2.5+ million portfolio
  • Large-cap and dividend focus
  • Intraday trade alerts from Cramer
Quant Ratings

Access the tool that DOMINATES the Russell 2000 and the S&P 500.

Product Features:
  • Buy, hold, or sell recommendations for over 4,300 stocks
  • Unlimited research reports on your favorite stocks
  • A custom stock screener
Stocks Under $10

David Peltier uncovers low dollar stocks with serious upside potential that are flying under Wall Street's radar.

Product Features:
  • Model portfolio
  • Stocks trading below $10
  • Intraday trade alerts
14-Days Free
Only $9.95
14-Days Free
Dividend Stock Advisor

David Peltier identifies the best of breed dividend stocks that will pay a reliable AND significant income stream.

Product Features:
  • Diversified model portfolio of dividend stocks
  • Updates with exact steps to take - BUY, HOLD, SELL
Trifecta Stocks

Every recommendation goes through 3 layers of intense scrutiny—quantitative, fundamental and technical analysis—to maximize profit potential and minimize risk.

Product Features:
  • Model Portfolio
  • Intra Day Trade alerts
  • Access to Quant Ratings
Real Money

More than 30 investing pros with skin in the game give you actionable insight and investment ideas.

Product Features:
  • Access to Jim Cramer's daily blog
  • Intraday commentary and news
  • Real-time trading forums
Only $49.95
14-Days Free
14-Days Free
AAPL $94.19 0.00%
FB $118.06 0.00%
GOOG $695.70 0.00%
TSLA $222.56 0.00%
YHOO $36.00 0.00%


Chart of I:DJI
DOW 17,651.26 -99.65 -0.56%
S&P 500 2,051.12 -12.25 -0.59%
NASDAQ 4,725.6390 -37.5850 -0.79%

Free Reports

Top Rated Stocks Top Rated Funds Top Rated ETFs