Google has revealed a new approach to using large language models (LLMs) that shows how robots can write their own code on the basis of human instructions.
The latest work built on Google’s PaLM-SayCan model for robots to understand open prompts from humans and respond logically and securely in physical space. It is also built on top of OpenAI’s GPT-3 LLM and related work in automatic code completion, like GitHub’s Copilot feature.
“What if, guided by a human, a robot could automatically write its own code to interact with the world?” Google researchers said. Google says the latest generation of language models, such as PaLM, are capable of complex inference and have also been trained on millions of lines of code. “With instructions in natural language, current language models are very adept at writing not only generic code, but, as we’ve discovered, code that can control robot actions. “
Also: Google is committed to supporting more languages using AI
Google Research calls its new development “Code as policy“and asserts that LLM writes code that can be reused to code robot policy according to natural language commands.
Google The researchers note in a new paper, Code as Policies: Language Model Programs for Embodied Control.
In the examples given, the user would say “lay the blocks on the empty bowl” or “lay the blocks horizontally near the top” of the 2D perimeter of the square. Google’s language modeling programs then write code in Python to precisely instruct the robot to follow said commands. It is based on the constructs of Python programming but also uses libraries like Shapely, in that case for geometric-spatial reasoning.
The improvement that Google claims is that language models may be better for this task than directly learning robot tasks and outputting natural language actions.
“Expanded CaP previous work, PaLM-SayCan, by allowing language models to complete even more complex robotic tasks with full expressions of general-purpose Python code. With CaP, we suggest using language models to write robot code directly through several prompts,” Notes from Google Research.
In addition to generalizing to new instructions, Google says the models can translate precise values, such as velocity, based on ambiguous descriptions like “faster” or “leftward”. “. CaP also supports tutorials in non-English languages and even emojis.
While the model can write code that instructs the robot to push different colored blocks to the top of a 2D square, it cannot translate more complex instructions like “build a house with blocks” because it has no references. 3D, according to Google.
It also warns that, while CaP offers robots more flexibility, it also “increases the potential risk because synthetic programs (unless checked manually during each run) can lead to to unexpected behavior with physical hardware.”
