Previous Blogs

May 7, 2019
Microsoft Bot Frameworks Enable Custom Voice Assistants

May 1, 2019
Dell Technologies Pushes Toward Hybrid Cloud

April 23, 2019
Intel and Nvidia Partner to Drive Mobile PC Gaming

April 16, 2019
Samsung Galaxy Fold Unfolds the Future

April 9, 2019
Google Embraces Multi-Cloud Strategy with Anthos

April 8, 2019
Intel Helps Drive Data Center Advancements

April 2, 2019
Gaming Content Ecosystem Drives More Usage

March 26, 2019
PCs and Smartphones Duke it Out for Gaming Champion

March 19, 2019
PCs and Smartphones Duke it Out for Gaming Champion

March 12, 2019
Proposed Nvidia Purchase and CXL Standard Point to Data Center Evolution

March 5, 2019
Tech Standards Still Making Slow but Steady Progress with USB4 and WebAuthn

February 26, 2019
Second Gen HoloLens Provides Insights into Edge Computing Models

February 19, 2019
IBM’s Watson Anywhere Highlights Reality of a Multi-Cloud World

February 12, 2019
Extending Digital Personas Across Devices

February 5, 2019
Could Embedded 5G/LTE Kill WiFi?

January 29, 2019
Successful IT Projects More Dependent on Culture Than Technology

January 22, 2019
XR Gaming Market Remains Challenging

January 15, 2019
The Voice Assistant War: What If Nobody Wins?

January 8, 2019
Big CES Announcements are TVs and PCs

January 2, 2019
Top Tech Predictions for 2019

2017 Blogs

2016 Blogs

2015 Blogs

2014 Blogs

2013 Blogs


















TECHnalysis Research Blog

May 14, 2019
Next Major Step in AI: On-Device Google Assistant

By Bob O'Donnell

The ability to have a smartphone respond to things you say has captivated people since the first demos of Siri on an iPhone over 7 years ago. Even the thought of an intelligent response to a spoken request was so science fiction-like that people were willing to forgive some pretty high levels of inaccuracy—at least for a little while.

Thankfully, things progressed on the voice-based computing and personal assistant front with the successful launch of Amazon’s Alexa-powered Echo smart speakers, and the Google Assistant found on Android devices, as well as Google (now Nest) Home smart speakers. All of a sudden, devices were accurately responding to our simple commands and providing us with an entirely new way of interacting with both our devices and the vast troves of information available on the web.

The accuracy of those improved digital assistants came with a hidden cost, however, as the recent revelations of recordings made by Amazon Alexa-based devices has laid bare. Our personal information, or even complete conversations from within the privacy of our homes, were being uploaded to the cloud for other systems, or even people, to analyze, interpret, and respond to. Essentially, the computing power and AI intelligence necessary to respond to our requests or properly interpret what we meant required the enormous computing resources of cloud-based data centers, full of powerful servers, running large, complicated neural network models.

Different companies used different resources for different reasons, but regardless, in order to get access to the power of voice-based digital assistants, you had to be willing to give up some degree of privacy, no matter which one you used. It was a classic trade-off of convenience versus confidentiality. Until now.

As Google demonstrated at their recent I/O developer conference, they now have the ability to run the Google Assistant almost entirely on the smartphone itself. The implications of this are enormous, not just from a privacy perspective (although that’s certainly huge), but from a performance and responsiveness angle as well. While connections to LTE networks and the cloud are certainly fast, they can’t compete with local computing resources. As a result, Google reported up to a 10x gain in responsiveness to spoken commands.

In the real-world that not only translates to faster answers, but a significantly more intuitive means of interacting with the assistant that more closely mimics what its like to speak with another human being. Plus, the ability to run natural language recognition models locally on the smartphones opens up the possibility for longer multi-part conversations. Instead of consisting of awkward silences and stilted responses, as they typically do now, these multi-turn conversations can now take on a more natural, real-time flow. While this may sound subtle, the difference in real-world experience literally shifts from something you have to endure to something you enjoy doing, and that can translate to significant increases in usage and improvements in overall engagement.

In addition, as hinted at earlier, the impact on privacy can be profound. Instead of having to upload your verbal input to the cloud, it can be analyzed, interpreted, and reacted to on the device, keeping your personal data private, as it should be. As Google pointed out, they are using a technique called federated learning that takes some of your data and sends it to the cloud in an anonymized form in order to be combined with others’ data and improve the accuracy of its models. Once those models are improved, they can then be sent back down to the local device, so that the overall accuracy and effectiveness of the on-device AI will improve over time.

Given what a huge improvement this is to cloud-based assistants, it’s more than reasonable to wonder why it didn’t get done before. The main reason is that the algorithms and datasets necessary to run this work used to be enormous and could only run with the large amounts of computing infrastructure available in the cloud. In addition, in order to create its models in the first place, Google needed a large body of data to build models that can accurately respond to people’s requests. Recently, however, Google has been able to shrink its models down to a size that can run comfortably even on lower-end Android devices with relatively limited storage.

On the smartphone hardware side, not only have we seen the continued Moore’s law-driven increases in computing power that we’ve enjoyed on computing devices for nearly 50 years, but companies like Qualcomm have brought AI-specific accelerator hardware into a larger body of mainstream smartphones. Inside most of the company’s Snapdragon series of chips is the little-known Hexagon DSP (digital signal processor), a component that is ideally suited to run the kinds of AI-based models necessary to enable on-device voice assistants (as well as computational photography and other cool computer vision-based applications). Qualcomm has worked alongside Google to develop a number of software hooks to neural networks they call the AndroidNN API that allows these to run faster and with more power efficiency on devices that include the necessary hardware. (To be clear, AI algorithms can and do run on other hardware inside smartphones—including both the CPU and GPU—but they can run more efficiently on devices that have the extra hardware capabilities.)

The net-net of all these developments is a decidedly large step forward in consumer-facing AI. In fact, Google is calling this Assistant 2.0 and is expected to make it available this fall with the release of the upcoming Q version of Android. It will incorporate not just the voice-based enhancements, but computer vision AI applications, via the smartphone camera and Google Lens, that can be done on device as well.

Even with these important advances, many people may view this next generation assistant as a more subtle improvement than the technologies might suggest. The reality is that many of the steps necessary to take us from the frustrating, early days of voice-based digital assistants, to the truly useful, contextually intelligent helpers that we’re all still hoping for are going to be difficult to notice on their own, or even in simple combinations. Achieving the science fiction-style interactions that the first voice-based command tools seemed to imply is going to be a long, difficult path. As with the growth of children, the day-to-day changes are easy to miss, but after a few years, the advancements are, and will be, unmistakable.

Here's a link to the column: https://techpinions.com/next-major-step-in-ai-on-device-google-assistant/55638

Bob O’Donnell is the president and chief analyst of TECHnalysis Research, LLC a market research firm that provides strategic consulting and market research services to the technology industry and professional financial community. You can follow him on Twitter @bobodtech.

Podcasts
Leveraging more than 10 years of award-winning, professional radio experience, TECHnalysis Research participates in regular audio podcasts in conjunction with the team at Techpinions.com.
LEARN MORE
  Research Offerings
TECHnalysis Research offers a wide range of research deliverables that you can read about here.
READ MORE

 

b