From task planning and perception to 3D generation and scene design.
Reality-aligned urban world generation
UrbanWorld2.0 A Multimodal Agentic Framework for Reality-Aligned 3D World Generation at City-Scale
1 College of AI, Tsinghua University
2 Shenzhen International Graduate School, Tsinghua University
3 Department of Electronic Engineering, Tsinghua University
4 Zhongguancun Academy
ACM MM 2026
Abstract
UrbanWorld2.0 is a reality-aligned intelligent multimodal synthesis engine for generating detailed, high-fidelity 3D worlds at city scale. An agent plans the generation process, curates real-world street-view and geospatial evidence, invokes multimodal foundation tools, and iteratively reflects on intermediate results. The resulting mesh-based worlds preserve real layouts while supporting scalable scene construction, standard computer graphics workflows, embodied intelligence, and world-model research.
Human preference against representative city-scale generation baselines.
Textured mesh assets remain editable in standard graphics pipelines.
Road topology, fine-grained objects, people, and vehicle traffic at city scale.
Framework
An agent that turns real-world evidence into complete 3D cities.
The framework separates a complex generation request into controllable stages. Each stage calls specialized tools while the planning module manages data flow, decisions, and quality feedback.
Planning
Decompose the city-generation objective and control execution.
Perception
Retrieve, curate, and assess street-view and geospatial evidence.
Imagination
Complete occluded structures with multimodal world knowledge.
Reflection
Critique intermediate outputs and regenerate weak candidates.
3D Generation
Create and refine textured, reusable mesh assets.
Scene Design
Align assets, roads, urban details, and traffic in one world.
Tool orchestration
Specialized tools, one controlled workflow.
- OpenStreetMap
- Street-view APIs
- Vision-language models
- Image generation & editing
- Hunyuan3D
- Blender & MOSS
Results
Higher perceptual quality without giving up real layouts.
UrbanWorld2.0 combines detailed building assets with geospatially controlled scene assembly. Evaluations cover region-level alignment, street-level quality, automated judging, and expert human preference.
86.9%Human preference win rate
6.05GPT-5.4 quality score
5.9833LAION aesthetics score
Reflection & refinement
Quality control is part of generation.
A vision-language quality critic identifies hallucination, structural anomalies, implausible textures, and instruction failures. Weak candidates are paired with a diagnosis and regenerated until they satisfy the quality threshold or reach the iteration limit.
Applications
Worlds built for reasoning, simulation, and embodied systems.
The generated environments are more than visual assets. They expose coherent geometry, navigable roads, dynamic traffic, and city-scale context for downstream intelligence.
Spatial reasoning
Object count, relative direction, distance, and scale.
Navigation
Route planning grounded in landmarks and urban topology.
Scene understanding
Multiview interpretation of buildings, roads, and context.
Traffic simulation
Lane-level networks with city-scale people and vehicles.
Citation
Cite UrbanWorld2.0
@inproceedings{wang2026urbanworld2,
title={UrbanWorld2.0: A Multimodal Agentic Framework for Reality-Aligned 3D World Generation at City-Scale},
author={Wang, Shengyuan and Zheng, Zhiheng and Shang, Yu and He, Lixuan and Yu, Yangcheng and Fan, Hangyu and Feng, Jie and Liao, Qingmin and Li, Yong},
booktitle={Proceedings of the 34th ACM International Conference on Multimedia},
year={2026},
doi={10.1145/3767308.3836009}
}