Papers
Topics
Authors
Recent
Search
2000 character limit reached

Know in AdVance: Linear-Complexity Forecasting of Ad Campaign Performance with Evolving User Interest

Published 17 May 2024 in cs.IR | (2405.10681v1)

Abstract: Real-time Bidding (RTB) advertisers wish to \textit{know in advance} the expected cost and yield of ad campaigns to avoid trial-and-error expenses. However, Campaign Performance Forecasting (CPF), a sequence modeling task involving tens of thousands of ad auctions, poses challenges of evolving user interest, auction representation, and long context, making coarse-grained and static-modeling methods sub-optimal. We propose \textit{AdVance}, a time-aware framework that integrates local auction-level and global campaign-level modeling. User preference and fatigue are disentangled using a time-positioned sequence of clicked items and a concise vector of all displayed items. Cross-attention, conditioned on the fatigue vector, captures the dynamics of user interest toward each candidate ad. Bidders compete with each other, presenting a complete graph similar to the self-attention mechanism. Hence, we employ a Transformer Encoder to compress each auction into embedding by solving auxiliary tasks. These sequential embeddings are then summarized by a conditional state space model (SSM) to comprehend long-range dependencies while maintaining global linear complexity. Considering the irregular time intervals between auctions, we make SSM's parameters dependent on the current auction embedding and the time interval. We further condition SSM's global predictions on the accumulation of local results. Extensive evaluations and ablation studies demonstrate its superiority over state-of-the-art methods. AdVance has been deployed on the Tencent Advertising platform, and A/B tests show a remarkable 4.5\% uplift in Average Revenue per User (ARPU).

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. Flamingo: a Visual Language Model for Few-shot Learning. Advances in Neural Information Processing Systems (NIPS) 35 (2022), 23716–23736.
  2. Layer normalization. arXiv preprint arXiv:1607.06450 (2016).
  3. Guy E Blelloch. 1990. Prefix sums and their applications. (1990).
  4. A Unified Framework for Campaign Performance Forecasting in Online Display Advertising. arXiv preprint arXiv:2202.11877 (2022).
  5. Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 785–794.
  6. Wide & deep learning for recommender systems. In Proceedings of the 1st workshop on deep learning for recommender systems. 7–10.
  7. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014).
  8. Ying Grace Cui and Ruofei Zhang. 2013. Campaign Performance Forecasting for Non-guaranteed Delivery Advertising. US Patent App. 13/495,614.
  9. Flashattention: Fast and memory-efficient exact attention with io-awareness. Advances in Neural Information Processing Systems 35 (2022), 16344–16359.
  10. Dentsu. 2022. Global Ad Spend Forecast. https://www.dentsu.com
  11. Deep session interest network for click-through rate prediction. arXiv preprint arXiv:1905.06482 (2019).
  12. Hungry Hungry Hippos: Towards Language Modeling with State Space Models. In The Eleventh International Conference on Learning Representations.
  13. Albert Gu and Tri Dao. 2023. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752 (2023).
  14. Efficiently Modeling Long Sequences with Structured State Spaces. In International Conference on Learning Representations.
  15. DeepFM: a factorization-machine based neural network for CTR prediction. arXiv preprint arXiv:1703.04247 (2017).
  16. Query-dominant User Interest Network for Large-Scale Search Ranking. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. 629–638.
  17. Practical lessons from predicting clicks on ads at facebook. In Proceedings of the eighth international workshop on data mining for online advertising. 1–9.
  18. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735–1780.
  19. Transformer quality in linear time. In International Conference on Machine Learning. PMLR, 9099–9117.
  20. Predicting the Performance of an Advertising Campaign. US Patent App. 14/292,277.
  21. Method and system for forecasting a campaign performance using predictive modeling. US Patent App. 14/747,706.
  22. Deephit: A deep learning approach to survival analysis with competing risks. In Proceedings of the AAAI conference on artificial intelligence, Vol. 32.
  23. Time interval aware self-attention for sequential recommendation. In Proceedings of the 13th International Conference on Web Search and Data Mining. 322–330.
  24. FAN: Fatigue-Aware Network for Click-Through Rate Prediction in E-commerce Recommendation. In International Conference on Database Systems for Advanced Applications. Springer, 502–514.
  25. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 1930–1939.
  26. Entire space multi-task model: An effective approach for estimating post-click conversion rate. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 1137–1140.
  27. Mega: Moving Average Equipped Gated Attention. In The Eleventh International Conference on Learning Representations.
  28. Exploring the limits of weakly supervised pretraining. In Proceedings of the European conference on computer vision (ECCV). 181–196.
  29. Ad Impression Forecasting for Sponsored Search. In Proceedings of the 22nd International Conference on World Wide Web. 943–952.
  30. Search-based user interest modeling with lifelong sequential behavior data for click-through rate prediction. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2685–2692.
  31. User behavior retrieval for click-through rate prediction. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2347–2356.
  32. Product-based neural networks for user response prediction. In IEEE 16th international conference on data mining (ICDM). IEEE, 1149–1154.
  33. Deep Landscape Forecasting for Real-time Bidding Advertising. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 363–372.
  34. Steffen Rendle. 2010. Factorization machines. In 2010 IEEE International conference on data mining. IEEE, 995–1000.
  35. Predicting clicks: estimating the click-through rate for new ads. In Proceedings of the 16th international conference on World Wide Web. 521–530.
  36. High-resolution Image Synthesis with Latent Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10684–10695.
  37. Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th international conference on World Wide Web. 285–295.
  38. Self-Attention with Relative Position Representations. In Proceedings of NAACL-HLT. 464–468.
  39. Simplified State Space Layers for Sequence Modeling. In The Eleventh International Conference on Learning Representations.
  40. Sequence to sequence learning with neural networks. Advances in neural information processing systems 27 (2014).
  41. Attention is all you need. Advances in neural information processing systems 30 (2017).
  42. Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature 575, 7782 (2019), 350–354.
  43. Pointer networks. Advances in neural information processing systems 28 (2015).
  44. Deep & cross network for ad click predictions. In Proceedings of the ADKDD’17. 1–7.
  45. A Search-based Method for Forecasting Ad Impression in Contextual Advertising. In Proceedings of the 18th International Conference on World Wide Web. 491–500.
  46. CLOCK: Online Temporal Hierarchical Framework for Multi-scale Multi-granularity Forecasting of User Impression. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. 2544–2553.
  47. Deep censored learning of the winning price in the real time bidding. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2526–2535.
  48. Predicting winning price in real time bidding with censored data. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1305–1314.
  49. Efficient Ad-level Impression Forecasting based on Monotonicity and Sampling. In 2021 7th International Conference on Big Data Computing and Communications (BigCom). IEEE, 180–187.
  50. Predicting advertiser bidding behaviors in sponsored search by rationality modeling. In Proceedings of the 22nd International Conference on World Wide Web (Rio de Janeiro, Brazil) (WWW ’13). Association for Computing Machinery, New York, NY, USA, 1433–1444. https://doi.org/10.1145/2488388.2488513
  51. Multi-task learning for bias-free joint ctr prediction and market price modeling in online advertising. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2291–2300.
  52. Metaformer is Actually What You Need for Vision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10819–10829.
  53. Effectively Modeling Time Series with Simple Discrete State Spaces. In The Eleventh International Conference on Learning Representations.
  54. Leaving No One Behind: A Multi-Scenario Multi-Task Meta Learning Approach for Advertiser Modeling. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining (Virtual Event, AZ, USA) (WSDM ’22). Association for Computing Machinery, New York, NY, USA, 1368–1376. https://doi.org/10.1145/3488560.3498479
  55. Bid-aware gradient descent for unbiased learning with censored data in display advertising. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. 665–674.
  56. Deep interest evolution network for click-through rate prediction. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33. 5941–5948.
  57. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 1059–1068.
Citations (1)
View on Semantic Scholar

Summary

  • The paper presents AdVance, a framework that integrates auction-level and campaign-level modeling to forecast ad performance with linear complexity.
  • It employs a Transformer Encoder and a conditional State Space Model to capture dynamic user interests and ad fatigue effectively.
  • Empirical results on Tencent data demonstrate reduced forecasting errors and a 4.5% increase in ARPU, underscoring its practical impact.

Linear-Complexity Forecasting of Ad Campaign Performance

The paper "Know in AdVance: Linear-Complexity Forecasting of Ad Campaign Performance with Evolving User Interest" presents a robust framework called AdVance, designed to enhance the prediction of ad campaign performance by integrating both auction-level and campaign-level modeling while accounting for evolving user interests. The framework is particularly pertinent in the Real-time Bidding (RTB) landscape, where advertisers need to forecast the impact and cost of campaigns efficiently.

Overview of AdVance Framework

AdVance tackles the forecasting challenge by combining two types of modeling: local auction-level details and global campaign-level insights. The core innovation is utilizing a Transformer Encoder to convert auctions into embeddings and summarizing these embeddings with a conditional State Space Model (SSM). This approach ensures that the entire procedure maintains global linear complexity, a significant improvement over the traditional quadratic complexity encountered in self-attention mechanisms. Figure 1

Figure 1: AdVance disentangles user interests as time-stamped click sequences representing user preference and fatigue vectors compressing all displayed items.

User Interest and Fatigue

User interest evolves over time as users interact with ads, accumulating preference for certain items while developing fatigue towards others. AdVance models these dynamics using a combination of click sequences and fatigue vectors. Click records provide a timestamped sequence reflecting preferences, while the fatigue vector, generated via an SSM, accounts for non-clicked items. These components are crucial in representing user behavior accurately, an approach that has shown to be superior to traditional static or incomplete models.

Modeling Auctions and Campaigns

The local auction modeling employs self-attention and cross-attention mechanisms. This dual-attention strategy models the interaction between competing ads and extracts relevant information from user features, forming a comprehensive view of auction dynamics. Subsequently, the auction representation undergoes a supervised learning process, optimizing predictions of auction-level performance metrics such as win rates and yields.

Globally, the AdVance framework incorporates these auction-level insights into a sequence via the conditional SSM. The SSM's parameters are made data-dependent, allowing the system to adapt to the irregular intervals typical of auction sequences in real-world scenarios. Figure 2

Figure 2: The AUC of three baselines and AdVance on five campaigns from various industries.

Performance and Evaluation

Empirical evaluations on large-scale datasets from the Tencent Advertising platform indicate that AdVance outperforms numerous state-of-the-art methods. The framework achieves lower Weighted Mean Absolute Percentage Error (WMAPE) across different forecasting horizons (1H, 6H, 12H, and 24H) compared to methods like CPF, GMIF, and MTAE, illustrating its robustness and efficacy in handling long sequences and evolving dynamics.

The conditional SSM significantly contributes to this performance, providing a scalable solution with linear complexity, contrasting the quadratic demands of self-attention in typical Transformer models.

Deployment and Real-World Impact

AdVance is deployed on Tencent's advertising platform, demonstrating practical utility with a reported 4.5% increase in Average Revenue per User (ARPU) during A/B testing. Its ability to predict campaign performance in real time, efficiently handling vast streams of auction data, underlines its value in large-scale online advertising ecosystems. Figure 3

Figure 3: Real-time bidding workflow and the funnel-shaped structure.

Conclusion

The AdVance framework provides a significant advancement in ad performance forecasting by effectively combining evolving user interest modeling with scalable sequence summarization through conditional SSMs. This dual approach not only improves prediction accuracy but also ensures operational efficiency, positioning AdVance as a vital tool in the RTB domain with substantial economic implications. Future work could explore integrating game-theoretic approaches to cater to even more dynamic advertising environments.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up