The results are in and automakers have officially submitted their disengagement reports to the state of California. Think of this sort of like a report card for self-driving car companies.
The 2018 disengagement report measures exactly what it sounds like, logging each time a test driver feels the need to disengage the vehicle's autonomous driving software by grabbing the wheel and taking control of the car.
So how did everyone do?
To no surprise, Alphabet's (GOOGL) - Get Report (GOOG) - Get Report Waymo unit was a dominant leader. According to the latest reports (painfully, found here) the platform logged more than 11,150 miles per disengagement. The average American drives just over 14,000 miles per year.
While Waymo's software can't be applied to every dirt road or commute route in the country, to think that someone could use Waymo's self-driving software for an entire year with just one intervention is pretty impressive. It's no wonder Waymo is considered the autonomous driving leader at the moment, particularly with Waymo's commercial service officially underway in Arizona.
Who else is on the list? We've got Cruise -- the autonomous driving subsidiary of General Motors (GM) - Get Report -- which logged just over 5,200 miles per disengagement while a startup called Zoox drove an average 2,000 miles per disengagement.
Tesla (TSLA) - Get Report said it "did not test any vehicles on public roads in California in autonomous mode or operate any autonomous vehicles," but that in the third quarter of 2018 customer-owned "Tesla vehicles experienced 1 crash for every 1.92 million miles driven. In the same timeframe, Tesla vehicles experienced 1 crash for every 3.34 million miles driven with Autopilot engaged."
But, that's part of the problem, too.
The Problem With Disengagement Rates
Disengagement rates have long been a way to measure autonomous driving performances among dozens of different companies. While a company can use its own disengagement rates -- which are self-reported -- to compare its growth over a certain period of time, comparing two different companies to each other is difficult.
For instance, when we were at GTC DC, an artificial intelligence conference hosted by Nvidia (NVDA) - Get Report , it was noted that the company logs a disengagement each time it ends the trip. Nvidia argues that a human has to end the test period and therefore, it counts as a disengagement. Other companies don't agree. While Nvidia's disengagement rate improved tremendously in 2018 vs. 2017 -- logging 20.1 miles per disengagement this year vs. 4.6 miles per disengagement last year -- not every company logs disengagements in the same manner.
So comparing Nvidia to Waymo or Cruise to Daimler (DDAIF) may not be a worthwhile effort. And who knows how Apple logs its disengagements.
Further, another talking at the GTC DC conference was the quality of the disengagement report. If a company is driving on a fairly open freeway under near-perfect driving conditions, then of course, it will have a low disengagement rate. However, that doesn't mean the system is better than a company with a high disengagement rate. Particularly if say, that system has been testing its autonomous fleet in dense urban environments and on busy rush-hour highways with complex on-off ramps.
Does that mean we should abandon the disengagement report altogether? Of course not. But it shouldn't be the sole measuring stick for the industry either, just a component to evaluating its progression.
Shares of Alphabet closed at $1,128.63, up $1.05 per share or 0.09%.
This article is commentary by an independent contributor. At the time of publication, the author had no positions in the stocks mentioned.