When Amazon began testing Agility Robotics’ 5-foot, 9-inch walking bots in mid-October, the bright blue autonomous workers looked smarter than they were.

“The reality is the robots are not self-aware, they're not even using large language models right now," Agility CEO Damion Shelton said at the time.

Just two months later, that’s changed.

In a two-minute “Embodied AI” demo released Wednesday, Digit responded to a casual spoken prompt from an engineer.

Advertise with us

“Take the box that’s the color of Darth Vader’s lightsaber and move it to the tallest tower,” Pras Velagapudi told Digit in the video.

The bot quickly obliged, picking up a red box from a set of green and blue options, then walking it over to the tallest of four towers. It interpreted the engineer’s natural language command using data in its new LLM, which was trained on pop culture references, the company said.

An “internal monologue” put out by Digit shows the exact decisions the robot made to respond to the prompt.

“Thought: Darth Vader’s lightsaber is red, and so the red box (box2) is the one I need to move,” the monologue said first.

Then, after securing the box and walking over to its destination, the robot had another thought.

Advertise with us

“Now that I’m at tower4, I should place box2 on it.”

The demonstration, which Agility called “a peek into the future,” comes as the company gears up to produce 10,000 Digit units per year at a massive production facility in Oregon.

In addition to the partnership with Amazon, the startup began testing its humanoids this month at a Spanx distribution center in Georgia.

Agility has said its autonomous system was capable of learning and adapting to new warehouses and tasks with minimal training. But AI capabilities will make that process even faster, meaning the bots could be deployed into a broader arena of tasks and eventually make their way outside of warehouses.

“There’s no need to redesign the warehouse or install infrastructure beyond that which is already designed around people,” Agility cofounder Jonathan Hurst said in the announcement.

Advertise with us

Coming at the end of a year when ChatGPT and other new generative AI capabilities were built into thousands of systems, the technology’s inclusion into Digit isn’t exactly a surprise. But Mr. Shelton wasn’t initially convinced his robots needed the new tool.

“In our case, we don't see there being a direct need or desire to have large language models or chatbots controlling the robots,” he told the Post-Gazette shortly after the Amazon announcement. “What customers actually need is not that you're having an interesting conversation with the robot. It’s that it’s doing the same thing repetitively and you're relying on the physics of the robot to do work for you.”

On Thursday, he clarified that LLM capabilities are more on the research and development side and are not yet ready for industrial deployment. But he said the ability to experiment on a functional humanoid was an opportunity he didn’t want to pass up.

Developers are locked in an intense race to bring the most attractive, capable agent to market, said Chris Atkeson, a robotics professor at CMU.

“Whoever comes out first… is probably going to claim a lot of market share,” he said.

Advertise with us

Boston Dynamics started using ChatGPT to give its doglike robot Spot spoken prompts earlier this year. The competitor, one of the first to demonstrate the potential of humanoids, signed a joint agreement with Agility last year not to weaponize the advanced mobile robots.

LLMs improve human robot interaction — an intense field of study and a dedicated institute at Carnegie Mellon University — by allowing robots to build trust with human co-workers, Rishi Malhan, a doctoral candidate at the University of Southern California found in a December report.

They also allow robots to introspect and ask for help, allowing them to gain more autonomy under human supervision, Mr. Malhan found.

GXO Logistics, the company testing Digit in Georgia, told the Robot Report it needs the robots to “learn from our teachings and move product as our business needs.”

It also needs the product to be reasonably priced, he said, though safety benefits help offset some of the investment.

Evan Robinson-Johnson: ejohnson@post-gazette.com and @sightsonwheels