A Multi-LLM-Agent-Based Framework for Economic and Public Policy Analysis

Abstract: This paper pioneers a novel approach to economic and public policy analysis by leveraging multiple LLMs as heterogeneous artificial economic agents. We first evaluate five LLMs' economic decision-making capabilities in solving two-period consumption allocation problems under two distinct scenarios: with explicit utility functions and based on intuitive reasoning. While previous research has often simulated heterogeneity by solely varying prompts, our approach harnesses the inherent variations in analytical capabilities across different LLMs to model agents with diverse cognitive traits. Building on these findings, we construct a Multi-LLM-Agent-Based (MLAB) framework by mapping these LLMs to specific educational groups and corresponding income brackets. Using interest-income taxation as a case study, we demonstrate how the MLAB framework can simulate policy impacts across heterogeneous agents, offering a promising new direction for economic and public policy analysis by leveraging LLMs' human-like reasoning capabilities and computational power.

• The paper presents a multi-agent framework using various LLMs to capture heterogeneous economic decision-making.
• The study evaluates five advanced LLMs through a two-period consumption-savings problem with CRRA utility optimization.
• Results reveal that distinct LLM reasoning patterns produce nuanced insights for economic modeling and policy analysis.

A Multi-LLM-Agent-Based Framework for Economic and Public Policy Analysis

This essay provides a detailed summary of the paper titled "A Multi-LLM-Agent-Based Framework for Economic and Public Policy Analysis" (2502.16879). This paper explores an innovative approach to economic and public policy analysis by leveraging multiple LLMs as heterogeneous agents. The research introduces the Multi-LLM-Agent-Based (MLAB) framework to simulate policy impacts across diverse socioeconomic groups. By using LLMs with varied capabilities and reasoning patterns, the paper aims to capture the complexity and heterogeneity inherent in real-world economic decision-making.

Introduction to Multi-LLM-Agent-Based Framework

The traditional approaches in macroeconomic modeling often employ strong assumptions such as rational expectations and representative agents, which may not capture the full spectrum of human behavior and economic dynamics. These models simplify complex processes into mathematical formulations that sometimes fail to reflect the nuanced behaviors observed in real economies. Agent-Based Models (ABMs) have emerged as alternative methods to incorporate boundedly rational and heterogeneous agents, offering insights into emergent phenomena. However, traditional ABMs rely heavily on predetermined and rigid behavioral rules.

Recent advances in artificial intelligence, particularly the development of LLMs, provide an opportunity to enhance agent-based modeling. These models exhibit human-like reasoning capabilities, making them suitable for modeling diverse cognitive traits in economic agents. By utilizing multiple LLMs, each with different analytical capabilities, this paper pioneers a two-dimensional approach to modeling heterogeneity. This method accounts for variations in economic circumstances and cognitive abilities, better reflecting the diverse nature of real-world decision making.

Evaluating LLMs' Optimization Capabilities

The research evaluates five state-of-the-art LLMs: DeepSeek-V3, GPT-4o-20241120, Gemini-1.5-pro-002, Claude-3.5-sonnet-20241022, and Llama-3.1-405B. The evaluation involves solving a two-period consumption-savings problem, a canonical framework in macroeconomic analysis. In this context, a representative middle-aged agent in urban China makes consumption decisions over a working and a retirement period.

The analysis with explicit utility functions involves maximizing CRRA utility, constrained by an intertemporal budget. The optimal solutions are derived via the Euler equation, showcasing the relationship between consumption smoothing and intertemporal substitution.

Figure 1: Consumption choices by LLMs. The diagonal line represents the budget constraint, and the dashed lines intersect at the theoretical optimal point. Each point represents one trial response.

Performance analysis indicates significant differences between LLMs in terms of optimal consumption choices and adherence to budget constraints. DeepSeek-V3 consistently clusters around the theoretical optimum, demonstrating strong optimization capabilities. In contrast, Llama-3.1-405B exhibits high variability, with many responses indicating under-consumption.

Assessing LLMs' Economic Intuition

To better simulate real-world conditions where decisions are influenced by intuition and social factors, the study assesses LLMs' economic intuition absent explicit optimization guidelines. The modified experimental design encourages LLMs to employ intuitive reasoning to determine consumption decisions.

Analysis reveals common tendencies, such as a generalized emphasis on consumption smoothing across all LLMs. However, each model demonstrates distinct characteristics in reasoning and decision-making. For example, DeepSeek-V3 maintains a professional economic reasoning approach, while Gemini-1.5-pro incorporates additional considerations like inflation and cultural values.

Figure 2: Consumption choices by LLMs without utility function specification. The diagonal line represents the budget constraint.

This dual approach to evaluating LLM performance—both with and without explicit optimization parameters—provides valuable insights into the models' inherent economic reasoning capabilities. Despite the absence of guidance, LLMs display varying levels of economic rationality rooted in human-like traits.

Policy Analysis with the MLAB Framework

The MLAB framework operationalizes the heterogeneous agent model by mapping different LLMs to corresponding educational and income groups. This mapping reflects both objective economic circumstances and subjective cognitive dimensions.

A case study on interest income taxation illustrates the application of the MLAB framework. By simulating a calibrated population of 100 individuals, the study demonstrates how different socioeconomic groups respond to taxation policy. The results reveal nuanced behavioral patterns consistent with expectations of varied tax sensitivities across demographics.

Figure 3: Saving Rate Analysis (1-c1/(w0+y1)).

The framework's two-dimensional approach provides a robust model for capturing heterogeneity in economic decision-making, presenting a promising direction for future research and policy evaluation.

Conclusion

The paper contributes to the field by presenting the Multi-LLM-Agent-Based framework as a significant advancement in economic modeling and policy analysis. By combining economic parameters with the LLMs' diverse reasoning patterns, the research offers a more comprehensive representation of heterogeneity in real-world decision-making. This methodology enhances simulation exercises, providing policymakers with a nuanced tool to understand different societal responses to policy interventions.

Future developments in LLM capabilities could lead to even more finely tuned agent simulations, paving the way for sophisticated, customized economic models. As these developments continue, the transformative potential of the MLAB framework in economic analysis and public policy research may become increasingly prominent.