Google Research and Everyday Robots are bringing together cutting‑edge, natural language research with robots that can learn.
This demo shows a PaLM-enabled helper robot performing a series of complex tasks using chain of thought prompting and the step-by-step solution needed to carry out the requests.
Select a task for the helper robot
See how PaLM-SayCan interprets the task
See how the helper robot executes the task
Learn the step-by-step solution behind PaLM-SayCan
Robots struggle to understand complex instructions that require reasoning. By using an advanced language model, PaLM-SayCan splits prompts into sub-tasks for the helper robot to execute, one-by-one. This also makes it a lot easier for people to get help from robots – just ask using everyday language.
As you can see, these four tasks start the same, but have varying ends. See how slightly changing the request affects the interpretation.
After a task has been requested, PaLM-SayCan uses chain of thought prompting, which interprets the instruction in order to score the likelihood that an individual skill makes progress towards completing the high-level request.
Unlike typical robotic demos that perform a single task in a constrained environment, this research shows you can ask an Everyday Robots helper robot for assistance, and it can carry out the request in a real-world environment using reasoning.
Multiple safety layers exist on the helper robot, including hardware safety layers, e‑stop, and move‑to‑contact behaviors.
Unlike typical robotic demos that perform a single task in a constrained environment, this research shows you can ask an Everyday Robots helper robot for assistance, and it can carry out the request in a real-world environment using reasoning.
Multiple safety layers exist on the helper robot, including hardware safety layers, e‑stop, and move‑to‑contact behaviors.
Unlike typical robotic demos that perform a single task in a constrained environment, this research shows you can ask an Everyday Robots helper robot for assistance, and it can carry out the request in a real-world environment using reasoning.
Multiple safety layers exist on the helper robot, including hardware safety layers, e‑stop, and move‑to‑contact behaviors.
Unlike typical robotic demos that perform a single task in a constrained environment, this research shows you can ask an Everyday Robots helper robot for assistance, and it can carry out the request in a real-world environment using reasoning.
Multiple safety layers exist on the helper robot, including hardware safety layers, e‑stop, and move‑to‑contact behaviors.
Unlike typical robotic demos that perform a single task in a constrained environment, this research shows you can ask an Everyday Robots helper robot for assistance, and it can carry out the request in a real-world environment using reasoning.
Multiple safety layers exist on the helper robot, including hardware safety layers, e‑stop, and move‑to‑contact behaviors.
Unlike typical robotic demos that perform a single task in a constrained environment, this research shows you can ask an Everyday Robots helper robot for assistance, and it can carry out the request in a real-world environment using reasoning.
Multiple safety layers exist on the helper robot, including hardware safety layers, e‑stop, and move‑to‑contact behaviors.
Unlike typical robotic demos that perform a single task in a constrained environment, this research shows you can ask an Everyday Robots helper robot for assistance, and it can carry out the request in a real-world environment using reasoning.
Multiple safety layers exist on the helper robot, including hardware safety layers, e‑stop, and move‑to‑contact behaviors.
Unlike typical robotic demos that perform a single task in a constrained environment, this research shows you can ask an Everyday Robots helper robot for assistance, and it can carry out the request in a real-world environment using reasoning.
Multiple safety layers exist on the helper robot, including hardware safety layers, e‑stop, and move‑to‑contact behaviors.
This is the request the human has given the robot. PaLM-SayCan is constrained to commands that are safe for a robot to perform.
I just worked out. Can you bring me a drink and a snack to recover?
This is how the language model has interpreted the request through chain of thought prompting.
The user has asked for a drink and a snack. I will bring a water bottle and an apple.
find a water
find an apple
find an energy bar
find a lime soda
find a 7up
find a water
find an apple
find an energy bar
find a lime soda
find a 7up
pick up the water
find an apple
bring it to you
pick up the apple
put down the water
bring it to you
find an apple
put down the water
pick up the apple
find an energy bar
put down the water
find an apple
put down the apple
done
put down the energy bar
find an apple
find an energy bar
done
find a water
find a lime soda
pick up the apple
bring it to you
pick up the water
put down the apple
pick up the energy bar
bring it to you
put down the apple
go to the table
pick up the apple
put down the water
put down the apple
put down the water
done
put down the energy bar
put down the orange soda
done
find an apple
find a water
put down the apple
find an energy bar
This is the request the human has given the robot. PaLM-SayCan is constrained to commands that are safe for a robot to perform.
I don’t like caffeinated soda. Can you bring me something else to drink from the table?
This is how the language model has interpreted the request through chain of thought prompting.
The user has asked me to bring a drink that isn't caffeinated. I will bring the water bottle.
find a water
go to the table
find a lime soda
find a redbull
find a 7up
find a water
go to the table
find a lime soda
find a redbull
find a 7up
pick up the water
bring it to you
go to the table
put down the water
pick up the energy bar
bring it to you
put down the water
go to the table
pick up the water
done
put down the water
done
put down the energy bar
put down the apple
put down the orange soda
done
find a water
go to the table
put down the water
find a pepsi
This is the request the human has given the robot. PaLM-SayCan is constrained to commands that are safe for a robot to perform.
I’m tired. Bring me a snack that’ll give me some energy, please.
This is how the language model has interpreted the request through chain of thought prompting.
The user has asked me to bring a snack that has energy. I will bring an energy bar.
find an energy bar
find a redbull
find an apple
find a lime soda
find a water
find an energy bar
find a redbull
find an apple
find a lime soda
find a water
pick up the energy bar
bring it to you
pick up the redbull
pick up the apple
put down the energy bar
bring it to you
put down the energy bar
go to the table
pick up the energy bar
done
put down the energy bar
done
put down the apple
put down the redbull
put down the orange soda
done
find a redbull
go to the table
find a coke
find a pepsi
This is the request the human has given the robot. PaLM-SayCan is constrained to commands that are safe for a robot to perform.
That snack is too small. Bring me one that is more filling.
This is how the language model has interpreted the request through chain of thought prompting.
The user has asked for a snack that is more filling. I will bring multigrain chips.
find the multigrain chips
find a 7up
put down the energy bar
find a redbull
find a water
find the multigrain chips
find a 7up
put down the energy bar
find a redbull
find a water
pick up the multigrain chips
bring it to you
put down the multigrain chips
find an energy bar
find an apple
bring it to you
put down the multigrain chips
go to the table
find an energy bar
done
put down the multigrain chips
done
put down the redbull
put down the energy bar
put down the apple
done
find an apple
find a redbull
find an energy bar
find a grapefruit soda
This is the request the human has given the robot. PaLM-SayCan is constrained to commands that are safe for a robot to perform.
I spilled my orange soda. Can you bring me a replacement, please?
This is how the language model has interpreted the request through chain of thought prompting.
The user has asked me to bring them a replacement for their orange soda. I will bring them a new one.
find an orange soda
find a lime soda
go to the table
find a 7up
bring it to you
find an orange soda
find a lime soda
go to the table
find a 7up
bring it to you
pick up the orange soda
bring it to you
put down the orange soda
find a lime soda
go to the table
bring it to you
put down the orange soda
go to the table
go to the close counter
find a lime soda
put down the orange soda
done
put down the lime soda
put down the water
put down the sponge
done
find a lime soda
put down the lime soda
put down the orange soda
find a sponge
This is the request the human has given the robot. PaLM-SayCan is constrained to commands that are safe for a robot to perform.
I spilled my orange soda. Can you bring me something to clean it up, please?
This is how the language model has interpreted the request through chain of thought prompting.
The user has asked me to bring something to clean up the orange soda. I will bring the sponge.
find a sponge
find an orange soda
find a redbull
find a water
put down the orange soda
find a sponge
find an orange soda
find a redbull
find a water
put down the orange soda
pick up the sponge
bring it to you
pick up the orange soda
put down the sponge
go to the table
bring it to you
put down the sponge
go to the table
done
go to the close counter
put down the sponge
done
put down the orange soda
put down the energy bar
put down the redbull
done
find an orange soda
put down the orange soda
find a sponge
find a water
This is the request the human has given the robot. PaLM-SayCan is constrained to commands that are safe for a robot to perform.
I spilled my orange soda. Can you throw it away, please?
This is how the language model has interpreted the request through chain of thought prompting.
The user has asked me to throw away the orange soda. I will move it to the trash.
find an orange soda
pick up the orange soda
find a sponge
go to the table
find a redbull
find an orange soda
pick up the orange soda
find a sponge
go to the table
find a redbull
pick up the orange soda
put down the orange soda
go to the trash
pick up the redbull
go to the table
go to the trash
put down the orange soda
bring it to you
go to the table
go to the close counter
put down the orange soda
put down the redbull
done
put down the sponge
put down the water
done
find a sponge
put down the orange soda
find a orange soda
go to the table
This is the request the human has given the robot. PaLM-SayCan is constrained to commands that are safe for a robot to perform.
I spilled my orange soda. Can you throw it away, bring me something to help clean it up, and then bring me a replacement?
This is how the language model has interpreted the request through chain of thought prompting.
The user has asked me to do three tasks. I will do one and then the other.
find an orange soda
find a sponge
put down the orange soda
pick up the orange soda
go to the table
find an orange soda
find a sponge
put down the orange soda
pick up the orange soda
go to the table
pick up the orange soda
go to the trash
put down the orange soda
bring it to you
pick up the sponge
go to the trash
put down the orange soda
bring it to you
find an energy bar
go to the close counter
put down the orange soda
find a sponge
put down the redbull
put down the sponge
done
find a sponge
pick up the sponge
find a 7up
find a water
find an energy bar
pick up the sponge
bring it to you
pick up the orange soda
put down the sponge
find a 7up
bring it to you
put down the sponge
go to the table
go to the trash
find a 7up
put down the sponge
find an orange soda
put down the orange soda
find a 7up
find a water
find an orange soda
find a 7up
find a coke
find a water
find a pepsi
pick up the orange soda
bring it to you
put down the orange soda
pick up the sponge
go to the table
bring it to you
go to the table
put down the orange soda
go to the close counter
go to the trash
put down the orange soda
done
put down the sponge
find an orange soda
put down the water
done
find an orange soda
find a sponge
find a water
find an apple
