OpenCity: A Scalable Platform to Simulate Urban Activities with Massive LLM Agents

Abstract: Agent-based models (ABMs) have long been employed to explore how individual behaviors aggregate into complex societal phenomena in urban space. Unlike black-box predictive models, ABMs excel at explaining the micro-macro linkages that drive such emergent behaviors. The recent rise of LLMs has led to the development of LLM agents capable of simulating urban activities with unprecedented realism. However, the extreme high computational cost of LLMs presents significant challenges for scaling up the simulations of LLM agents. To address this problem, we propose OpenCity, a scalable simulation platform optimized for both system and prompt efficiencies. Specifically, we propose a LLM request scheduler to reduce communication overhead by parallelizing requests through IO multiplexing. Besides, we deisgn a "group-and-distill" prompt optimization strategy minimizes redundancy by clustering agents with similar static attributes. Through experiments on six global cities, OpenCity achieves a 600-fold acceleration in simulation time per agent, a 70% reduction in LLM requests, and a 50% reduction in token usage. These improvements enable the simulation of 10,000 agents' daily activities in 1 hour on commodity hardware. Besides, the substantial speedup of OpenCity allows us to establish a urban simulation benchmark for LLM agents for the first time, comparing simulated urban activities with real-world data in 6 major cities around the globe. We believe our OpenCity platform provides a critical infrastructure to harness the power of LLMs for interdisciplinary studies in urban space, fostering the collective efforts of broader research communities. Code repo is available at https://anonymous.4open.science/r/Anonymous-OpenCity-42BD.

• The paper presents a scalable platform that simulates urban activities using massive LLM agents, achieving a 600-fold speedup on standard hardware.
• The paper introduces a Group-and-Distill Meta-Prompt Optimizer that reduces LLM requests by 73.7% and token usage by 45.5%, optimizing simulation efficiency.
• The paper demonstrates high simulation accuracy in reproducing urban dynamics and enables counterfactual experiments, enhancing urban planning studies.

OpenCity: A Scalable Platform to Simulate Urban Activities with Massive LLM Agents

Introduction

The paper presents "OpenCity," a scalable platform for simulating urban activities using large-scale LLMs agents. The work addresses the computational challenges of deploying numerous LLM agents in urban simulations by introducing novel optimization strategies to significantly enhance system efficiency. The platform leverages the inherent reasoning capabilities of LLM agents to model complex human behaviors, offering a robust tool for urban studies and planning.

Architectural and Algorithmic Innovations

LLM Request Scheduler

OpenCity incorporates a scalable LLM request scheduler to optimize network communication in agents' simulation. This scheduler uses I/O multiplexing to parallelize LLM requests, reducing idle waiting times typically encountered during network communication.

Figure 1: The functionality of the proposed LLM Request Scheduler.

Group-and-Distill Meta-Prompt Optimizer

The Group-and-Distill Meta-Prompt Optimizer reduces computational overhead by decreasing the number of LLM prompts. By clustering agents with similar static attributes and utilizing batch prompting, this strategy minimizes redundancy and effectively compresses input tokens, sustaining agent independence concerning dynamic properties.

Figure 2: Overview of Group-and-Distill Meta-Prompt Optimizer.

Experimental Results

Performance Evaluation

The experimental evaluation conducted across six major global cities demonstrates that OpenCity achieves substantial improvements in simulation efficiency. The platform can simulate 10,000 agents' activities in just one hour on standard hardware, achieving a 600-fold speedup and reducing LLM requests by 73.7% and token usage by 45.5%.

Figure 3: Acceleration experiment results and scalability experiments showcase significant performance improvements.

This performance indicates the platform's strong scalability, as demonstrated by the reduced simulation times per agent as the number of agents increases from one to 10,000.

Faithfulness and Simulation Accuracy

To assess accuracy, OpenCity was tested for its ability to reproduce urban dynamics such as the radius of gyration and the origin-destination matrix. The system demonstrates high fidelity in simulating human-like behaviors, maintaining consistency in agents' dynamic attributes and minor discrepancies in urban dynamics reproduction metrics across cities.

Counterfactual Simulations

The platform’s capabilities extend to conducting counterfactual experiments, such as examining experienced urban segregation under different urban planning scenarios. By evenly distributing income groups within simulations, researchers can observe and analyze the resulting impact on urban segregation metrics.

Figure 4: The distribution of income segregation index for counterfactual experiment.

Practical and Theoretical Implications

Infrastructure for Urban Studies

OpenCity acts as infrastructure for interdisciplinary urban studies, supporting simulations that integrate social, economic, and environmental factors. Its user-friendly web interface allows researchers from various fields to engage with complex simulations without extensive programming knowledge.

Future Developments

The platform sets a precedent for future research by highlighting the utility of LLM agents in large-scale simulations. There remains potential for enhancing LLM agents' scalability and multi-agent interactions further, integrating real-time adaptive models that better replicate dynamic urban systems.

Figure 5: A detail case of interpreting simulation results through communication.

Conclusion

OpenCity offers a critical advancement in the simulation of urban activities through LLM agents, drastically improving computational efficiency while maintaining high fidelity in behavioral modeling. The platform's design opens avenues for a broad range of applications in urban planning and policy-making, urging further exploration of LLM capabilities in complex system simulations.