RTBAgent: A LLM-based Agent System for Real-Time Bidding

Abstract: Real-Time Bidding (RTB) enables advertisers to place competitive bids on impression opportunities instantaneously, striving for cost-effectiveness in a highly competitive landscape. Although RTB has widely benefited from the utilization of technologies such as deep learning and reinforcement learning, the reliability of related methods often encounters challenges due to the discrepancies between online and offline environments and the rapid fluctuations of online bidding. To handle these challenges, RTBAgent is proposed as the first RTB agent system based on LLMs, which synchronizes real competitive advertising bidding environments and obtains bidding prices through an integrated decision-making process. Specifically, obtaining reasoning ability through LLMs, RTBAgent is further tailored to be more professional for RTB via involved auxiliary modules, i.e., click-through rate estimation model, expert strategy knowledge, and daily reflection. In addition, we propose a two-step decision-making process and multi-memory retrieval mechanism, which enables RTBAgent to review historical decisions and transaction records and subsequently make decisions more adaptive to market changes in real-time bidding. Empirical testing with real advertising datasets demonstrates that RTBAgent significantly enhances profitability. The RTBAgent code will be publicly accessible at: https://github.com/CaiLeng/RTBAgent.

• The paper introduces RTBAgent, which leverages LLM-driven dynamic reasoning and strategic memory retrieval to optimize real-time bidding.
• It employs a two-step decision-making process that combines expert strategies with insights from environment, bidding, and reflection memories to adapt bids effectively.
• Performance evaluations using the iPinYou dataset demonstrate RTBAgent's superior adaptability, stability, and profitability compared to traditional and RL-based methods.

RTBAgent: A LLM-based Agent System for Real-Time Bidding

The paper "RTBAgent: A LLM-based Agent System for Real-Time Bidding" (2502.00792) introduces an innovative approach to optimizing real-time bidding (RTB) for online advertising. Leveraging the capabilities of LLMs, RTBAgent aims to address the challenges posed by dynamic bidding environments and discrepancies between offline and online conditions.

Introduction to RTBAgent

RTB is crucial in digital advertising, allowing advertisers to bid for ad impressions in real time. Traditional methods such as rule-based and reinforcement learning often struggle with adaptability and interpretability in complex and rapidly changing environments. RTBAgent offers a novel solution by integrating LLMs to perform dynamic reasoning, memory retrieval, and strategic decision-making.

The system is designed with multiple components: tools for data and strategy analysis, various memory types, and a two-step decision-making process. The framework is tailored to maximize profitability by simulating competitive bidding scenarios and dynamically adjusting bidding strategies.

Figure 1: The workflow of our RTBAgent, which is equipped with 4 tools, 3 types of memory, two-step decision-making to execute actions and output bidding prices.

Key Components

Decision-Making Process

RTBAgent employs a two-step decision-making process:

1. Insight Reasoning: The agent gathers information from its three memories, combines them with expert bidding strategies, and generates insights into potential decisions.
2. Action Making: Based on the insights, it calculates an optimal bidding price, allowing for adjustments in response to observed market conditions.

This process enhances the agent's bid optimization by enabling real-time adjustments and strategic flexibility, thus outperforming traditional bidding methods.

Memory Systems

RTBAgent's memory system, consisting of environment, bidding, and reflection memories, plays a vital role in its effectiveness. These memories store historical data, decision rationale, and reflection outcomes, which are critical for contextualizing current decisions and optimizing future strategies.

Figure 2: An example of RTBAgent using LLMs to assist reasoning process.

Performance Evaluation

Empirical evaluations conducted using the iPinYou dataset demonstrate RTBAgent's superiority over traditional methods and RL models across various budget constraints. The agent achieved notable improvements in click counts, highlighting its capacity for effective real-time bid optimization.

Performance Comparison:

• RTBAgent outperformed traditional models like MCPC, LIN, and LP, showcasing its adaptability to fluctuating market conditions.
• Compared to RL-based models like DRLB and USCB, RTBAgent maintained higher stability and interpretability in decision-making.
• Analysis revealed that RTBAgent's two-step decision-making process significantly enhanced performance metrics.

Ablation Studies:

Experiments with different LLMs confirmed the agent's adaptability, allowing deployment with various model sizes without sacrificing efficacy.

Implications and Future Directions

The integration of LLMs in RTB posits significant implications for the advertising industry. RTBAgent's capability to deliver high-performance outcomes in dynamic environments make it a valuable tool for enhancing ad profitability and strategic insight. The research opens avenues for further exploration of multi-agent systems and the potential refinement of LLM integration in competitive scenarios.

Limitations:

• Response time may be an issue compared to traditional methods, suggesting scope for optimization in the deployment environment.
• The current generation lacks comprehensive bidding knowledge, indicating potential for further enhancements through techniques like Retrieval-Augmented Generation (RAG).

Conclusion

RTBAgent represents a substantial advancement in utilizing LLMs for real-time bidding challenges. Its system architecture, combining strategic reasoning and adaptive memory retrieval, positions it as a robust solution for optimizing advertising efficiency in competitive markets. As digital advertising continues to evolve, RTBAgent sets a precedent for exploring sophisticated AI applications in real-time decision environments.