OpenAI powers a robot that can hand people food, pick up trash, put away dishes, and more

Despite reports of enterprises getting cold feet around embracing generative AI due to cost and accuracy issues, it’s clear that in the world of robotics, the AI age is just starting to take off.

Today, Figure, a robotics startup valued at $2.6 billion, founded less than two years ago by former workers at Boston Dynamics, Tesla, Google DeepMind, and Archer Aviation, showed off its first collaboration with new investor and partner OpenAI, maker of ChatGPT, and it is undeniably impressive.

Figure co-founder and CEO Brett Adcock took to his account on the social platform X to post a video demo of a Figure full-sized humanoid robot, the Figure 01 (pronounced “Figure One”), demonstrating its capabilities to interact with a nearby human and its environment, showing the robot following the person’s orders, locating and handing them an object (an apple, in this case), describing what it’s doing and conversing with the person (albeit with slightly delayed reaction time from what we would expect in a typical human-to-human conversation), and identifying, planning and carrying out helpful tasks on its own (in this case, picking up trash and putting dishes into a drying rack).

In a scene straight out of a sci-fi film, the video begins with the human saying “Hey Figure One, what do you see right now?” The robot responds: “I see a red apple on the plate in the center of the table, a drying rack with cups and a plate, and you standing nearby with your hand on the table.”

“Great, can I have something eat?” the human asks.

“Sure thing,” Figure One states, carefully reaching, grasping the apple, and handing it to the human — understanding that the apple is the only edible object in front of it, without the human even specifying.

The video goes on to show Figure picking up trash and putting away the plate and cup in the drying rack.

A new model emerges? OpenAI VLM

Adcock posted in a thread on X that “Figure’s onboard cameras feed into a large vision-language model (VLM) trained by OpenAI,” though it’s unclear if this is a version of GPT-4, OpenAI’s flagship LLM that powers the subscription version of ChatGPT (Plus), such as GPT-4V, if it is a fine-tuned version of such an existing model, or if it is an entirely new model. We’ve reached out to OpenAI for further details on the collaboration and this demo and will update when we hear back.

In an impressive declaration, Adcock also noted that “The video is showing end-to-end neural networks. There is no teleop. Also, this was filmed at 1.0x speed and shot continuously.” In other words: the video was not sped up, as prior demo videos of humanoid robots have often done to showcase more fluidity of movement, and nor was there a human being remotely controlling the robot’s motions in any part behind-the-scenes.

Where Figure goes from here

Figure’s demo video appears to be a significant leap forward in humanoid, general purpose robotics interactions — showing a robot interacting fairly naturally with a person, obeying them, intuiting what they want, and doing so much more smoothly than many previous examples from other companies and researchers.

However, it is of course still just that — a demo, and of a prototype at that. It will likely take significant more work to get such a robot ready for commercial deployment and sell it to businesses and/or individuals. Yet Adcock has openly stated, including his X thread today, that “Our goal is to train a world model to operate humanoid robots at the billion-unit level.”

And on Figure’s website, Adcock’s first-person “master plan” states that “the goal of Figure: to develop general purpose humanoids that make a positive impact on humanity and create a better life for future generations. These robots can eliminate the need for unsafe and undesirable jobs — ultimately allowing us to live happier, more purposeful lives.”

Yet Adcock goes on to say “Our company journey will take decades — and require a championship team dedicated to the mission, billions of dollars invested, and engineering innovation in order to achieve a mass-market impact. We face high risk and extremely low chances of success.”

He also vows: “We will not place humanoids in military or defense applications, nor any roles that require inflicting harm on humans.”

The progress shown by Adock and Figure today, powered by OpenAI, is likely to place much greater pressure on rivals in the humanoid robotics space such as Tesla with its Optimus project and Agility, a humanoid robotics startup working with Amazon on fulfillment roles. It also comes as more companies enter the space, including Hugging Face (which just hired a former Tesla Optimus scientist to lead its newly announced open-source robotics project), and yesterday’s announcement of a startup called Physical Intelligence.

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative enterprise technology and transact. Discover our Briefings.

[ For more curated tech news, check out the main news page here]

The post OpenAI powers a robot that can hand people food, pick up trash, put away dishes, and more first appeared on venturebeat.com