The voice assistant from Alphabet's (GOOGL) Google is more clever than Apple's (AAPL) digital voice assistant Siri and competitor Microsoft's (MSFT) Cortana, according to a new study from Loup Ventures released Monday morning.
In the test, the three voice assistants were asked 800 questions and graded on two metrics: Did the assistant understand the question and did the assistant answer the query correctly. The questions fell under five categories:
Local - Where is the nearest McDonald's?
Commerce - Where can I buy more printer paper?
Navigation - How do I get to REI from here?
Information - What is Apple's stock price?
Command - Remind me to call Mom at 2 p.m. today.
Google Assistant gave a stunning performance on the test, understanding 99.9% of the questions and answering 74.8% of them correctly. Coming in second place, Siri understood 94.4% of the questions asked and answered 66.1% of them correctly. In third place, Cortana understood 97.3% of the queries, but only gave the right answer to 48.8% of them.
Looking at performance in specific categories, Apple and Google performed similarly on questions in the local, commerce and command categories, but Google trumped Apple in the navigation and information categories. Meanwhile, Cortana lagged behind the other two in all categories except information, where it placed second.
The study is important because speech-driven user interfaces are key to the next computing paradigm, Loup Ventures founder Gene Munster wrote in the note. In addition, Google's lead on the navigation and information questions is a big takeaway from the test because accuracy in these categories can greatly effect whether people keep using the assistant. For example, users will avoid using an assistant if it fails to give the right directions to a destination.
"In this way, Google's overall accuracy creates a self-reinforcing cycle where users will go to Google for a better experience if Siri or Cortana fail them," Munster wrote.
Cortana came in last place not because it couldn't understand the query, but because it had trouble answering questions. But this function should improve since Microsoft has been investing in its search platform Bing.
"We expect Microsoft to continue to invest in its Cortana platform, as it will be an important part of its AR/VR hardware in the future," Munster wrote.
Siri's came in second, based on its strength at interpreting commands correctly, which is important as Apple pushes its home automation technology known as HomeKit.
"By continuing to outperform Google Assistant, Siri offers the best connection between a user's mobile life and home life, which Amazon will struggle to do without an integrated smartphone platform," Munster explained.
Loup Ventures performed the same 800-question test in a battle between the digital helper Alexa behind Amazon's (AMZN) Echo vs. the helper behind Google Home, earlier this year. The two devices meant to assist you on everyday tasks in your home, such as playing music and reading out the top headlines of the day, both performed well on the test, but Google Assistant came out on top in that competition as well.
The research firm may soon be testing Apple's own Siri-powered version of the Amazon Echo and Google Home, according to a note KGI Securities analyst Ming-Chi Kuo sent Monday to investors. The top analyst said there's more than a 50% chance that the tech giant will announce its first home AI product at Apple Worldwide Developers Conference (WWDC) this June and will start selling it in the second half of this year.
Australian "leaker" Sonny Dickson tweeted last week that Apple's home assistant will be a direct competitor of Amazon's Alexa and will utilize Beats technology and run a version of Apple's iOs. Apple paid $3 billion in 2014 for Beats Electronics, a maker of headphones, speakers and audio software, and Beats Music, a subscription steaming music service. Beats was co-founded by Iovine and Dr. Dre, who both joined Apple as part of the deal.
Apple is currently finalising designs for their Alexa competitor, expected to be marketed as a Siri/AirPlay device.— Sonny Dickson (@SonnyDickson) April 27, 2017
It is believed to carry some form of Beats technology, and is expected to run an variant iOS— Sonny Dickson (@SonnyDickson) April 27, 2017
In addition, Apple is expected to unveil its 10th-anniversary edition of the iPhone sometime between September and November.
Separately, Apple reports 2017 second-quarter results after Tuesday's market close.
Editor's pick: This article was originally published at 1:31 p.m. ET