返回资讯中心
外部精选
软件工程
#开发者实时流

3DCodeVerse

Yipeng Gao1,*,†Ziyao Zeng2,*Jinfa Huang3,*Yijiang Li4,*

Hacker News 最新ilreb2026/10/16 · 2026/10/1618 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

Yipeng Gao1,*,†Ziyao Zeng2,*Jinfa Huang3,Yijiang Li4,

Haolin Xiong1Wang Qin1Xinlei Yu1Youheng Yao2Jie Zhang5Kun-Yu Lin6Rong Liu1Yinyuan Zhao7

Yajie Zhao1Meiqi Guo8Jiebo Luo3Lei Shu8Yunhao Ge1Xinyu Hu9Laurent Itti1

1University of Southern California2Yale University3University of Rochester4University of California, San Diego5Columbia University6Nanyang Technological University7University of California, Santa Cruz8Google DeepMind9Philo Labs

PaperarXiv soonCodeDatasetVideo

Jade GuardianLantern RiverOutpost

▶ Run live in your browser

The Jade Guardian, running live. Every mesh, material and light, and the water surface, is generated by a few hundred lines of Three.js and GLSL. Nothing is downloaded except the source. Drag to orbit, or switch scenes.

Expand figure3DCodeVerse showcase: scenes, graphics, articulated objects and 3D objects, all built by code

Close

Original image

01code → 3D

One case per language across all four tracks. The left pane is the actual source file that produced the render on the right; the GLSL shader is compiled and running in your browser, and the URDF assets move between their real joint states.

57 chambers, nacreous layers and fine growth lines, on a stand that actually touches the shell.

Programs come from public datasets, from converting specialized representations (DeepCAD sequences and Articraft geometry become standalone CadQuery), and from running the harness itself. Each program ships with the sources and assets it needs to execute, renders, three kinds of captions (instruction, visual description, construction procedure) and its verification record.

Expand figureLeft: programs collected from public datasets and generated with the harness. Middle: a coding agent with modeling skills and inspection tools (3D IoU, collision, overlap, distance) builds, judges and refines each program. Right: the corpus spans objects, articulated objects, scenes and graphics.

Curation and composition of 3DCodeVerse-2M

Close

Left: programs collected from public datasets and generated with the harness. Middle: a coding agent with modeling skills and inspection tools (3D IoU, collision, overlap, distance) builds, judges and refines each program. Right: the corpus spans objects, articulated objects, scenes and graphics.

Original image

Packaged catalogue

269,597 programs across ten collections, expanded into 2,013,629 render and caption entries: the 2.03M code–condition pairs. 3DCodeBench factories, instances and test splits are held out for evaluation.

Visually plausible is not the same as geometrically valid. The harness pairs a VLM judge with deterministic measurements, turns every finding into a targeted edit, and loops without human feedback, scaling at inference time to projects with thousands of lines of code.

Expand figureAn LLM planner defines the target and a DAG coordinates sub-agents to code and assemble the program (left). Execution checks, spatial measurements and rubric-guided VLM judgments feed a fix agent (center, bottom). Successive rounds are scored and the best program is kept (right).

Overview of the 3DCodeVerse Harness

Close

An LLM planner defines the target and a DAG coordinates sub-agents to code and assemble the program (left). Execution checks, spatial measurements and rubric-guided VLM judgments feed a fix agent (center, bottom). Successive rounds are scored and the best program is kept (right).

Original image

01

Plan

An LLM planner turns the prompt into components, dimensions, materials, spatial relations and acceptance criteria.

The plan becomes a DAG of coding and assembly tasks. Sub-agents own separate files and write them in parallel.

Execution logs, then spatial tools: contacts, collisions, joint sweeps and placement, each reported per part in millimetres.

A VLM judge reads labelled multi-view renders against the plan. Measurements cap the score and veto claims they rule out.

Findings become localized repair tasks for a fix agent. Every round is kept, and the highest-scoring program wins.

“Belly plates sink 3.2 mm into the clouds (≤ 2 mm).” “Scales read as domed buttons; flatten the dome to 0.05w.”

During development the judge and the generator are refined in turns. The judge is tuned on a fixed bank of stored artifacts to reduce score variance and contradictions with measurements; then, with the judge frozen, harness designs are compared on matched prompts, trading judged quality against workflow failures.

Across four backbones, outputs from each model's native harness are rarely preferred, while the same model inside the 3DCodeVerse harness wins most comparisons. Fine-tuning open Qwen models on 3DCodeVerse-2M turns mostly failing programs into runnable ones.

Share of blind side-by-side comparisons won; ties split equally

Gemini 3.1 Pro rubric score, 0–100; failed generations score 0

Execution rate after fine-tuning on 3DCodeVerse-2M

% of programs that run in the dialect's pinned runtime; T = 0.7, one seed

The preference and judge charts cover 100 text-to-3D object prompts. Human preference also reaches 78–88% on articulated objects, 50 scene prompts and 50 graphics prompts.

Expand figure(a) The same prompts and backbones with the native harness (Without) and with the 3DCodeVerse harness (With). (b) Selected rounds of automated refinement; numbers are rubric scores on [0, 1].

Qualitative comparisons and automated refinement

Close

(a) The same prompts and backbones with the native harness (Without) and with the 3DCodeVerse harness (With). (b) Selected rounds of automated refinement; numbers are rubric scores on [0, 1].

Original image

Expand figureApplications. Code-built articulated assets and scenes go to robot simulation (MuJoCo) and act as spatial and temporal guides for video generation models.

Applications of 3D code in robotics and video generation

Close

Applications. Code-built articulated assets and scenes go to robot simulation (MuJoCo) and act as spatial and temporal guides for video generation models.

Original image

05Showcase film

Introducing 3DCodeVerse: a 45-second tour of ten code-authored worlds. Sound on.

Code-authored showcase scenes. Geometry, procedural materials, lighting, camera and animation are all kept as executable source. Hover to play a clip, and click to read the code next to the render.

These are showcase studies written with the harness and polished over several rounds. They are not the random samples or model comparisons behind the reported results.

Copy@article{gao2026_3dcodeverse, title = {3DCodeVerse: Exploring 3D Worlds as Code}, author = {Gao, Yipeng and Zeng, Ziyao and Huang, Jinfa and Li, Yijiang and Xiong, Haolin and Qin, Wang and Yu, Xinlei and Yao, Youheng and Zhang, Jie and Lin, Kun-Yu and Liu, Rong and Zhao, Yinyuan and Zhao, Yajie and Guo, Meiqi and Luo, Jiebo and Shu, Lei and Ge, Yunhao and Hu, Xinyu and Itti, Laurent}, journal = {arXiv preprint}, year = {2026} }

Related: 3DCodeBench: Benchmarking Agentic Procedural 3D Modeling via Code.

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →