advertisement
Forums

The Forum is sponsored by 
 

AAPL stock: Click Here

You are currently viewing the Tips and Deals forum
A SIRIs of Unfortunate Events?!.....they tested Siri on the HomePod....
Posted by: NewtonMP2100
Date: February 12, 2018 06:43AM
....answered correctly 52.3% of the time which is at the bottom of AIs...

Google Home answered 81 percent correctly
Amazon's Alexa answered 64 percent correctly
Microsoft's Cortana answered 57 percent correctly.


Siri on HomePod Correctly Answered 52.3% of Queries in New AI Test

.....In a new test shared today by Loup Ventures, Apple's HomePod was put through its paces in categories including Siri, sound quality, and ease of use. For Siri, Loup Ventures' Gene Munster reported that while the AI assistant understood 99.4 percent of queries asked of it, it answered only 52.3 percent of them correctly. Loup Ventures tested three separate HomePods and asked 782 queries total.

Compared to previous tests of rival speakers, HomePod is "at the bottom of the totem pole" in the AI assistant performance category. Google Home answered 81 percent correctly, Amazon's Alexa answered 64 percent correctly, and Microsoft's Cortana answered 57 percent correctly.

Munster broke this information down further, stating that Siri is good for "local" and "commerce" queries, like asking about nearby coffee shops or assisting in buying new shoes. In this area, Siri beats Alexa and Cortana but still falls behind Google Assistant on Google Home.

Despite the low percentage of correctly answering the 782 total queries asked of it, Munster said Siri's overall performance rose above expectations "given the limited scope of HomePod's music focus."

The researchers explained that over time HomePod and Siri should grow to match, or surpass, rival assistants by simply adding query domains like calendar, email, calling, and navigation.

Some domains like navigation, calendar, email, and calling are simply not supported. These questions were met with, “I can’t ___ on HomePod.” Also, in any case that iPhone-based Siri would bring up Google search results, HomePod would reply, “I can’t get the answer to that on HomePod,” which forces you to use your phone or give up on the question altogether. Removing navigation, calling, email, and calendar-related queries from our question set yields a 67% correct response, a jump from overall of 52.3% correct.

This means added support for these domains would bring HomePod performance above that of Alexa (64%) and Cortana (57%), though still shy of Google Home (81%). We know Siri has the ability to correctly answer a whole range of queries that HomePod cannot, evidenced by our note here. Apple’s limiting of HomePod’s domains should change over time, at which point we expect the speaker to be vastly more useful and integrated with your other Apple devices.

As discovered in the research, where HomePod excelled was its "superior" listening skills. The HomePod allows users to speak at a normal volume, even when music from the speaker is particularly loud, and Siri will pick up on the voice and hear the user. "This was HomePod's most stellar feature," according to Munster.

Loup Ventures also favored HomePod's sound quality, saying that "it sounds incredible." Like other reviews and impressions of the device, Munster's enjoyment of the HomePod's audio quality was countered with the speaker's lackluster Siri performance, which he thinks will be changing fairly soon.

"Don’t be fooled by HomePod’s sound quality-focused first step into smart speakers; Apple has a grander vision than delivering a better sounding Echo. While not present in the first version of HomePod (i.e. you can’t even make a phone call with HomePod), we believe Apple’s goal is to make Siri a ubiquitous, ambient presence that connects and controls all your connected devices and services – and to make a leap forward in the transition to voice-first computing.

Visit Loup Ventures to read more of the information gathered from the HomePod "smart speaker gauntlet," including the publication's prediction for demand and market share of HomePod through the rest of 2018.



not very good performance......?!



____________________________________________________

I reject your reality and substitute my own!



Edited 1 time(s). Last edit at 02/12/2018 06:53AM by NewtonMP2100.
Options:  Reply • Quote
Re: A SIRIs of Unfortunate Events?!.....they tested Siri on the HomePod....
Posted by: sekker
Date: February 12, 2018 07:00AM
AI ‘dont’ know...
Options:  Reply • Quote
Re: A SIRIs of Unfortunate Events?!.....they tested Siri on the HomePod....
Posted by: space-time
Date: February 12, 2018 07:00AM
How would a MRF member do in a similar test?
Options:  Reply • Quote
Re: A SIRIs of Unfortunate Events?!.....they tested Siri on the HomePod....
Posted by: tronnei
Date: February 12, 2018 08:04AM
In the two days I've had my HomePod, my only non-musical request was a weather report, which was accurately reported. All my music requests, save one, were 100% spot on. (On the one miss, David Bowie request was misheard as "David Choi". My second request for Bowie was correctly heard and delivered.) I'm also pleased with the sound quality.
Options:  Reply • Quote
Re: A SIRIs of Unfortunate Events?!.....they tested Siri on the HomePod....
Posted by: cbelt3
Date: February 12, 2018 08:20AM
With or without training ?
Options:  Reply • Quote
Re: A SIRIs of Unfortunate Events?!.....they tested Siri on the HomePod....
Posted by: testcase
Date: February 12, 2018 08:51AM
That mirrors my experience quite well. After all these years, Siri is STILL not ready for prime time as far as I'm concerned. The answers I get from "OK Google" are FAR more useful to me. boink smiley
Options:  Reply • Quote
Re: A SIRIs of Unfortunate Events?!.....they tested Siri on the HomePod....
Posted by: N-OS X-tasy!
Date: February 12, 2018 02:13PM
Quote
testcase
That mirrors my experience quite well. After all these years, Siri is STILL not ready for prime time as far as I'm concerned. The answers I get from "OK Google" are FAR more useful to me. boink smiley

Apple should be embarrassed as hell by that.



It is what it is.
Options:  Reply • Quote
Re: A SIRIs of Unfortunate Events?!.....they tested Siri on the HomePod....
Posted by: mikebw
Date: February 12, 2018 02:52PM
Quote
N-OS X-tasy!
Quote
testcase
That mirrors my experience quite well. After all these years, Siri is STILL not ready for prime time as far as I'm concerned. The answers I get from "OK Google" are FAR more useful to me. boink smiley

Apple should be embarrassed as hell by that.

My interpretation is that since Apple now has (arguably) the best sounding device they can only get better AI. Others will get better AI too, but the sound of the device is permanent.

EDIT: apparently the HomePod is an EXCELLENT speaker - [www.reddit.com]



Edited 1 time(s). Last edit at 02/12/2018 04:44PM by mikebw.
Options:  Reply • Quote
Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 269
Record Number of Users: 52 on November 20, 2014
Record Number of Guests: 2330 on October 25, 2018