GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators

Philipp Wu, Yide Shentu, Zhongke Yi, Xingyu Lin, Pieter Abbeel
University of California Berkeley
Paper Code Hardware

Abstract

Imitation learning from human demonstrations is a powerful framework to teach robots new skills. However, the performance of the learned policies is bottlenecked by the quality, scale, and variety of the demonstration data. In this paper, we aim to lower the barrier to collecting large and high-quality human demonstration data by proposing GELLO, a general framework for building low-cost and intuitive teleoperation systems for robotic manipulation. Given a target robot arm, we build a GELLO controller that has the same kinematic structure as the target arm, leveraging 3D-printed parts and off-the-shelf motors. GELLO is easy to build and intuitive to use. Through an extensive user study, we show that GELLO enables more reliable and efficient demonstration collection compared to commonly used teleoperation devices in the imitation learning literature such as VR controllers and 3D spacemouses. We further demonstrate the capabilities of GELLO for performing complex bi-manual and contact-rich manipulation tasks. To make GELLO accessible to everyone, we have designed and built GELLO systems for 3 commonly used robotic arms: Franka, UR5, and xArm. All software and hardware are open-sourced.


GELLO

teaser

GELLO is a low-cost, intuitive teleoperation framework for robot manipulators, designed to be user-friendly and affordable. GELLO is a scaled, kinematically equivalent replica of a given target arm. This enables users to intuitively control the target arm by directly manipulating GELLO to control the joints of the target arm, which reduces the gap in embodiment between the user and the target arm.



While this paradigm offers more intuitive control, it also means that each target arm needs its own specialized device. GELLO utilizes 3D-printed parts and off-the-shelf motors, making it a cost-effective solution to build customized teleoperation devices for high-quality robot learning demonstrations. The parts to build GELLO cost less than $300. In addition, the minimalist design enables easy construction and extension to new robot types. We build GELLOs for 3 different robots. See some example demonstrations below!

This is GELLO built for the UR5. We see that GELLO is a kinimatic replica up to an alpha scale factor (alpha=0.5 in this case). This allows us to control the robot directly in joint space. The kinimatic equivalent structure allows the user to feel kinimatic constratints, enabling more intuative control.

Example Demonstrations

GELLO is effective in bi-manual configurations

Make toast

Extract some floss

Make Ramen

Remove teabag

Make tea

Prepare juicebox

GELLO for the Franka

Stack block

Close and open laptop

Grasp box (with extrinsic dexterity)

Clean whiteboard

Stack cups

GELLO for the xArm

Retrive drinks from fridge

Refill waterbottle

Get hot water

GELLO is effective in bi-manual configurations (same videos shown in real time)

Make toast

Extract some floss

Make Ramen

Remove teabag

Make tea

Prepare juicebox

GELLO for the Franka (same vidoes shown in real time)

Stack block

Close and open laptop

Grasp box (with extrinsic dexterity)

Clean whiteboard

Stack cups

GELLO for the xArm (same videos shown in real time)

Retrive drinks from fridge

Refill waterbottle

Get hot water

Build Your Own GELLO

If you are interested in building your own GELLO, please check out our the assembly instructions here. It just takes ~30 min to assemble!

User Study

We conducted a user study involving 12 participants and 5 tasks. We focus on bi-manual robot teleoperation using two UR5 robots to assess the comparative effectiveness of GELLO vs other commonly used teleoperation devices under controlled conditions. The primary workspace of our experiments is shown below.

teaser

We compare GELLO with VR controllers and 3D mouses, which are shown below. Each user attempts to accomplish the 5 tasks with all 3 teleoperation devices.

teaser
The five tasks we explore are:
  1. Place a hat on a rack,
  2. Open a case and fetch the sleeping mask inside,
  3. Hand over a banana to the kitchen area,
  4. Fold a towel, and
  5. Plug in a USB cable.

These tasks are designed to explore different teleoperation challenges such as articulated object interaction, large workspaces, deformable objects, and precise insertion. See example executions of the 5 tasks with GELLO in the video below.


Task 1: Hang Hat
This task requires the robot to grasp the hat and hang it on the hook.

Task 2: Remove Mask
This task requires the robot to open the case and remove the sleeping eye mask.

Task 3: Handoff Banana
This task requires the robot to hand off the banana and place it into the kitchen.

Task 4: Fold Towel
This task requires the robot to fold the towel in half.

Task 5: Plugin USB
This task requires the robot to plugin the USB cable.

Full User Study Run

Here is an example session from the user study, released with permission from the participant. The user first practices with each device for 5 minutes, then sequentially completes the 5 tasks. The order the user was presented the devices for this session was, GELLO, 3D mouses, and lastly VR.

Results

teaser

Utilizing GELLO results in consistently faster completion times. This not only signifies that GELLO is easier to use with a higher success rate but also, indicates its efficiency; faster completion times would enable users to achieve more successful operations in a given time frame.

BibTeX

@misc{wu2023gello,
    title={GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators},
    author={Philipp Wu and Yide Shentu and Zhongke Yi and Xingyu Lin and Pieter Abbeel},
    year={2023},
}