Discover how companies are responsibly integrating AI in production. This invite-only event in SF will explore the intersection of technology and business. Find out how you can attend here.
Earlier this year, in March 2024, the AI developer-focused startup Hugging Face — known for maintaining the largest online repository of open source AI models and code — made waves by poaching Remi Cadene, a former staff scientist at Tesla, Inc., to lead a new open source robotics project.
The news was notable not only because Tesla is such an established name in tech and robotics — it’s recently been pursuing the development of a humanoid robot model called Optimus — but because in general, the entire field of robotics has been supercharged and radically advanced in recent months by the addition of transformer-based large language models (LLMs) and generative AI more broadly, two subjects Hugging Face already specialized in.
Now, today, we are getting our first taste of what Hugging Face’s open source robot vision looks like: the company has announced its new initiative, LeRobot—an open-source toolkit designed to democratize AI robotics and inspire a new generation of roboticists, available now on Github.
“The next step of AI development is its application to our physical world,” posted Cadene in a long thread of messages from his personal account on X (formerly Twitter). “Thus, we are building a community-driven effort around AI for robotics, and it’s open to everyone!”
VB Event
The AI Impact Tour – San Francisco
Join us as we navigate the complexities of responsibly integrating AI in business at the next stop of VB’s AI Impact Tour in San Francisco. Don’t miss out on the chance to gain insights from industry experts, network with like-minded innovators, and explore the future of GenAI with customer experiences and optimize business processes.
Cadene also posted a link to a free Discord chat server for those interested in joining, leveraging the open source code, and discussing the LeRobot initiative.
What is LeRobot?
LeRobot serves as a robust framework akin to the “Transformers” for robotics.
This toolkit is not just a software package but a comprehensive platform that includes a versatile library for sharing, visualizing data, and training state-of-the-art models.
Users can also access a plethora of pretrained models to jumpstart their projects.
Additionally, LeRobot integrates seamlessly with physics simulators, which allows enthusiasts and developers without physical robotics hardware to simulate and test their AI models in a virtual environment.
Capabilities and features
The toolkit’s capabilities extend beyond just software.
It is designed to handle a range of robotic hardware—from simple robotic arms utilized in education and research to the more sophisticated humanoids seen in advanced AI research facilities.
The goal is to provide an AI system that can adapt and control any form of robot, enabling both versatility and scalability in robotics applications.
Cadene posted some examples of the kind of robotic capabilities offered by the code now available in the LeRobot library on Github, including training robots to navigate unmapped spaces and grasp things from video…
And how to pass objects between two robotic gripper hands/arms.
Open-Source philosophy
The decision to make LeRobot open-source is a strategic one, aimed at avoiding the concentration of power and innovation within a handful of corporations.
By offering it freely, Hugging Face invites a global community of developers, researchers, and hobbyists to contribute to and benefit from the collective advancement of AI robotics.
Building a Community and a Data Repository
Underpinning LeRobot’s development is the creation of the largest crowdsourced robotics dataset ever attempted. I
n collaboration with universities, startups, major tech firms, and individual hobbyists, Hugging Face is facilitating the assembly of a vast repository of data.
This includes terabytes of onboard video recordings, which are being formatted using the lightweight LeRobotDataset for quick upload and download via the Hugging Face hub.
“We are building a diverse community from various backgrounds, software and hardware, to develop the next generation of smart robots in the real-world,” Cadene stated on X, reflecting the inclusive vision of the project.
By lowering barriers to entry and fostering an environment of shared knowledge and resources, Hugging Face aims to cultivate a community that could redefine the landscape of AI robotics.
