Qualcomm’s Smart Home Vision Includes TVs That Read Your ‘Emotional State’

LAS VEGAS—Qualcomm has a vision for making your home appliances chattier (but not more connected) via built-in AI chatbots that run locally instead of routing your queries into the cloud. 

“Edge AI enables more processing to occur on the device itself, resulting in faster, more secure, and highly personalized experiences,” Qualcomm says. But some parts of the AI-chatbot demos expected at its CES exhibits may still be a recipe for user unease.

The first demo outlined by Qualcomm is a connected refrigerator with interior cameras (an increasingly common high-end feature) that incorporates Qualcomm’s chatbot. That AI agent, running the QCS8550 chipset, processes camera and microphone inputs during conversations with fridge users to act as a sort of culinary coach.

“With this iteration of the chatbot, users can ask their refrigerator to recommend recipes based on the available ingredients detected through computer vision,” Qualcomm says. “It can also create a shopping list for additional ingredients, order them from a nearby store, and alert a family member’s car to pick them up on the way home.”

A second demo will show how this technology can work when added to a connected TV, which in Qualcomm’s concept includes having the set look at you with its own camera. Think Microsoft’s long-gone Kinect but without games involved. 

The company’s post suggests this will allow natural-language interactions for everyday tasks like switching channels or orchestrating other devices in your connected home but also outlines things that today’s TVs don’t touch.

“These AI capabilities will enable new user experiences such as spur-of-the-moment watch parties with friends, real-time language translation between non-native speaking relatives, personalized movie recommendations based on users’ emotional states, and generating travel plans for your family,” Qualcomm suggests.

Considering how badly AI can misread human emotions, we have questions about how an AI agent, on-device or cloud-based, would pick a flick based on how happy or upset it thinks we are. (Well, unless this thing suggests the 1977 Mel Brooks comedy High Anxiety when it detects anxiety in a viewer, in which case: not creative but still well played, robot.)

The last part of Qualcomm’s CES exhibit will feature its own contribution to the growing CES genre of conversational home robots. This Home Service Robot can chat with you as if it were a more conversationally adept smart speaker on wheels but can also put its on-device AI to work when using its cameras around your house. 

“The built-in cameras enable machine vision and image recognition, allowing the robot to distinguish between a bottle of water and a soda can when prompted verbally,” Qualcomm says. It does not address whether you’d be able to ask this robot or a future version of it to fetch you a beer from a connected fridge running another version of the same AI chatbot.

There may not be an actual market for this robot. Amazon’s Astro hasn’t fared much better than the Fire Phone, but Qualcomm doesn’t seem wrong in emphasizing how on-device AI can avoid privacy risks and other side effects of cloud-based AI.

“There are privacy, IP [intellectual property], safety and security issues,” Qualcomm chief marketing officer Don McGuire told me at Web Summit in November 2023. “There are sustainability issues with regards to power and water consumption at the data center level.”

That last one also means serious costs to the company keeping an AI cloud-based. “I mean, an AI prompt on ChatGPT is seven to 10 times more expensive than a Google search,” he added.