BrickGPT (LegoGPT): Free AI for Stable, Buildable LEGO Designs From Text

BrickGPT (formerly LegoGPT) is an open-source AI project from researchers at Carnegie Mellon University that turns text descriptions into physically stable, buildable LEGO-style models.

Describe a chair, guitar, or small vehicle, and it generates the individual brick placements for a model you can assemble.

BrickGPT uses a language model fine-tuned from Llama-3.2-1B-Instruct to predict the brick placements one at a time. It checks proposed bricks and regenerates parts of a design that fail physical stability analysis.

Features

• Generate brick-by-brick models from natural-language descriptions.
• Reject invalid or overlapping bricks and regenerate unstable sections through physics-aware rollback.
• Preview results in an online demo with adjustable generation settings.
• Export local results as PNG previews, brick-coordinate TXT files, and LDraw models.
• Apply text-guided UV textures or individual brick colors to existing designs.
• Convert OBJ, GLB, and other 3D meshes into brick layouts.

Use Cases

• Personal LEGO-style builds: Generate a model of a familiar object, inspect the brick placements, and assemble it using physical bricks.
• Design exploration: Change the prompt or random seed to produce different brick arrangements for an object before building a prototype.
• 3D model conversion: Turn an existing OBJ or GLB mesh into a brick layout for assembly planning or further editing in LDraw software.
• Engineering education: Compare generated structures and examine how brick connections and support affect physical stability.
• Robotics research: Study brick-level assembly sequences using the team’s demonstrated examples of models built by people and robotic arms.

How BrickGPT Creates Stable Brick Models

BrickGPT was fine-tuned using StableText2Brick, a dataset of more than 47,000 stable brick layouts representing over 28,000 distinct 3D objects. Each layout is encoded as an ordered sequence of brick dimensions and coordinates. Given a description and the bricks already placed, the model predicts the next brick in the sequence.

As it generates the structure, BrickGPT rejects malformed bricks, unsupported dimensions, and placements that collide with existing bricks. It checks physical stability after completing the design. If the structure is unstable, it removes the affected bricks and subsequent placements, returns to a stable point, and generates replacement bricks.

How to Try BrickGPT

Use the Online Demo

Open the BrickGPT online demo, enter a description, and generate a rendered preview with a copyable list of brick sizes and coordinates. You can use an example prompt or adjust the seed, temperature, maximum brick count, and regeneration settings.

Run BrickGPT Locally

Clone the repository and initialize the ImportLDraw submodule used to render brick models:

git clone https://github.com/AvaLovelace1/BrickGPT.git
cd BrickGPT
git submodule update --init

For PNG rendering, put the background EXR file in ImportLDraw/loadldraw. Download and extract the LDraw parts library into your home directory. If you store the parts in another location, set LDRAW_LIBRARY_PATH to the extracted ldraw directory.

The default inference command uses Gurobi for physics-based stability checks. Configure a Gurobi license before running it. Install the Python dependencies, set your Hugging Face token, and launch BrickGPT:

uv sync
export HF_TOKEN=YOUR_HUGGING_FACE_TOKEN
uv run infer

To run the generator without Gurobi, replace the last command with uv run infer --use_gurobi False. This selects the simpler connectivity-based stability check.

Output Files and Additional Tools

Local inference saves a rendered .png preview, a .txt file listing brick sizes and coordinates, and an .ldr file for LDraw-compatible design software. In the text output, 1x2 (16,18,0) represents a 1×2 brick at the specified position.

For coloring and texturing, the tools in src/texture take an existing brick layout and a text prompt. They can produce UV textures and rendered meshes or save a colored .ldr model. The mesh-to-brick converter takes an existing 3D mesh, such as an OBJ or GLB file, and exports a brick structure as JSON, TXT, or LDraw.

To convert an existing mesh using the included utility, run this command from the src/mesh2brick directory:

uv run mesh2brick input.obj output.ldr

Related Resources

• BrickGPT Model Weights
• StableText2Brick Dataset
• Generating Physically Stable and Buildable Brick Structures from Text
• CADAM AI: Free AI Text-to-3D & Image-to-3D Model Generator

FAQs

Is BrickGPT free and open-source?

Yes. The BrickGPT model, StableText2Brick dataset, and most of the source code are available under the MIT License. The bundled ImportLDraw and FlashTex submodules have their own licenses.

Which bricks and objects does BrickGPT support?

BrickGPT uses eight brick footprints: 1×1, 1×2, 1×4, 1×6, 1×8, 2×2, 2×4, and 2×6. All bricks are one unit tall, and models are generated on a 20×20×20 grid. The training set includes 21 object categories, such as furniture, vehicles, and musical instruments. Results for objects outside these categories can be less reliable.

Does BrickGPT generate step-by-step building instructions?

BrickGPT outputs an ordered list of brick placements and an LDraw model that you can inspect in compatible software. It does not produce a conventional illustrated LEGO instruction booklet.

How long does it take to generate a model?

A 59-brick table in the project’s example run took 63.53 seconds to generate. Generation time varies with the model size, hardware, and regeneration settings.