A group of computer scientists at Carnegie Mellon University is applying generative AI to Legos with “LegoGPT,” which can produce a Lego block design based on the user’s text input.
“Our experiments show that LegoGPT produces stable, diverse, and aesthetically pleasing Lego designs that align closely with the input text prompts,” they wrote in a paper published this week.
(Credit: Carnegie Mellon University)
However, this isn’t your average AI image generator, which spit out any image you request. The difference is that LegoGPT creates “stable” 3D Lego designs that respect the laws of physics.
The paper notes that most AI-powered 3D object generation doesn’t translate into the real world because the “objects may be difficult to assemble or fabricate using standard components” or “the resulting structure may be physically unstable even if assembly is possible.” Translation: AI generates some truly strange stuff that you couldn’t easily turn into a Lego design.
(Credit: Carnegie Mellon University)
Researchers approached the problem by training the AI model on “physically stable Lego designs paired with captions.” This led the team to create a virtual collection, dubbed the StableText2Lego dataset, with 47,000+ Lego structures. Each structure was also run through a program to calculate and assign a “stability score.”
(Credit: Carnegie Mellon University)
In addition, LegoGPT can construct physically stable designs by checking for errors during the generation process. “If the resulting design is unstable…we roll back the design to the state before the first unstable brick was generated,” the researchers wrote. “We repeat this process iteratively until we reach a stable structure…and continue generation from the partial structure.”
According to the paper, LegoGPT generated a physically stable design over 98% of the time, outperforming other AI approaches. The program was also smart enough to create valid Lego designs free of errors that adhered to the text prompt 100% of the time.
Recommended by Our Editors
(Credit: Carnegie Mellon University)
In addition, the researchers say the designs from LegoGPT “can be assembled manually by humans and automatically by robotic arms” in the real world.
The researchers uploaded LegoGPT to GitHub, so anyone can download the program and try it out. However, it’s currently limited to “20 × 20 × 20 grid” 3D Lego creations. LegoGPT also appears to be focused on simpler designs, rather than the pricey sets the company sells styled after hit properties like Star Wars and Harry Potter.
Still, the paper adds: “Our method currently supports a fixed set of commonly used Lego bricks. In future work, we plan to expand the brick library to include a broader range of dimensions and brick types, such as slopes and tiles, allowing for more diverse and intricate Lego designs.”
Get Our Best Stories!
Your Daily Dose of Our Top Tech News
By clicking Sign Me Up, you confirm you are 16+ and agree to our Terms of Use and Privacy Policy.
Thanks for signing up!
Your subscription has been confirmed. Keep an eye on your inbox!
About Michael Kan
Senior Reporter
