Dobb·E stands as an open-source framework tailored for instructing robots in household chores via imitation learning, aiming to overcome current limitations in home robotics. This innovative framework offers a cost-effective and user-friendly solution for gathering demonstrations, utilizing a tool known as the Stick, comprised of a $25 Reacher-grabber stick, 3D printed components, and an iPhone.
At the core of Dobb·E lies its ability to harness the Stick to gather data from the Homes of New York (HoNY) dataset, encompassing 13 hours of interactions across 22 residences in New York City. This dataset includes RGB and depth videos, along with action annotations detailing the gripper’s 6D pose and opening angle.
Leveraging this collected data, Dobb·E employs a representation learning model named Home Pretrained Representations (HPR). Built upon the ResNet-34 architecture and trained via self-supervised learning objectives, HPR serves as a foundation for initializing robot policies to execute new tasks in unfamiliar environments.
Impressively, Dobb·E showcases its effectiveness by achieving an average success rate of 81% in solving novel tasks within a mere 15 minutes, based on just five minutes of collected data in a new household. The framework provides convenient access to pre-trained models, code, and documentation through GitHub, while further insights into its methodology and outcomes are available in the open-access paper titled “On Bringing Robots Home”.
