Learning to Move Like Professional Counter-Strike Players

Abstract: In multiplayer, first-person shooter games like Counter-Strike: Global Offensive (CS:GO), coordinated movement is a critical component of high-level strategic play. However, the complexity of team coordination and the variety of conditions present in popular game maps make it impractical to author hand-crafted movement policies for every scenario. We show that it is possible to take a data-driven approach to creating human-like movement controllers for CS:GO. We curate a team movement dataset comprising 123 hours of professional game play traces, and use this dataset to train a transformer-based movement model that generates human-like team movement for all players in a "Retakes" round of the game. Importantly, the movement prediction model is efficient. Performing inference for all players takes less than 0.5 ms per game step (amortized cost) on a single CPU core, making it plausible for use in commercial games today. Human evaluators assess that our model behaves more like humans than both commercially-available bots and procedural movement controllers scripted by experts (16% to 59% higher by TrueSkill rating of "human-like"). Using experiments involving in-game bot vs. bot self-play, we demonstrate that our model performs simple forms of teamwork, makes fewer common movement mistakes, and yields movement distributions, player lifetimes, and kill locations similar to those observed in professional CS:GO match play.

• The paper presents MLMove, a transformer-based movement controller achieving real-time inference with less than 0.5 ms per game step.
• The paper leverages the CSKnow dataset, comprising 123 hours of professional CS:GO gameplay, to train and rigorously evaluate the model through human studies and quantitative metrics.
• The paper demonstrates that MLMove’s behavior closely mimics human gameplay, outperforming rule-based bots and setting a foundation for immersive AI in FPS games.

Learning to Move Like Professional Counter-Strike Players

In this paper, the authors present a data-driven approach to generate human-like movement for bots in the first-person shooter game Counter-Strike: Global Offensive (CS:GO). The bot, named MLMove, leverages a novel movement model trained on an extensive dataset of professional game play, addressing the long-standing challenge of creating realistic bot behavior in multiplayer FPS games.

Key Contributions

1. Efficient Transformer-Based Movement Controller: The core of MLMove is a transformer-based model trained to predict player movement actions based on game state. This model operates efficiently, meeting the stringent real-time performance requirements of commercial FPS games. The model achieves an amortized inference cost of less than 0.5 ms per game step on a single CPU core, which is within the acceptable limits for deployment in commercial game servers.
2. CSKnow Dataset Curation: The authors curated a comprehensive dataset named CSKnow, comprising 123 hours of professional CS:GO gameplay, sampled at 16 Hz. This dataset includes detailed player movements, game state information, and event logs, providing a rich resource for training and evaluating movement models.
3. Human-Like Movement Evaluation: The performance of the MLMove bot was evaluated through both human studies and quantitative analysis of in-game metrics. Human evaluators consistently ranked MLMove's behavior as more human-like compared to rule-based bots and the bots currently deployed in CS:GO. Quantitatively, MLMove's movement distributions, player lifetimes, and kill locations closely matched those of professional human players.

Technical Approach

Transformer-Based Movement Model

The movement model uses a transformer architecture inspired by the Scene Transformer but is optimized for the specific requirements of FPS games. Key features include:

• Input Representation: The model inputs include player states (position, velocity, team association), map states (bomb location, time remaining), and derived features approximating visibility and team strategy information.
• Output Representation: The model outputs discrete movement commands, including direction, speed, and jump states, encoded as probabilities.
• Attention Mechanism: The transformer architecture leverages self-attention to capture complex inter-player interactions, essential for realistic team-based movement.
• Compute Efficiency: Design choices such as restricting the model to one game map and optimizing the number of input tokens contribute to the model's high computational efficiency.

Training and Integration

The model was trained using a supervised learning approach on the CSKnow dataset, with standard cross-entropy loss for movement command prediction. The integrated bot, MLMove, combines the learned movement model with a rule-based execution module for aiming and firing, ensuring efficient and realistic gameplay.

Evaluation and Results

Human Study

A user study involving 15 evaluators with varying levels of CS:GO experience was conducted to assess the realism of bot movements. Evaluators watched videos of gameplay and ranked the bots based on human-likeness. MLMove significantly outperformed rule-based bots and the commercially available CS:GO bots, achieving TrueSkill ratings that were closer to those of actual human players.

Quantitative Self-Play Analysis

Extensive quantitative analysis was performed on self-play rounds, comparing distributions of player positions, kill locations, shots per kill, and player lifetimes against human gameplay data. MLMove matched human gameplay distributions more closely across all metrics. Notably, the model demonstrated key movement strategies like using cover effectively and coordinating attacks, aspects where rule-based bots fell short.

Implications and Future Work

The success of MLMove demonstrates the effectiveness of transformer-based models in capturing complex, human-like behavior in team-based FPS games. This approach has significant implications for the development of more realistic and immersive AI opponents and teammates in commercial games. Future work could extend this model to multiple maps and incorporate additional game actions beyond movement, such as firing strategies and advanced team tactics.

Furthermore, the creation of comprehensive datasets like CSKnow provides a valuable resource for the broader research community, opening avenues for further advancements in AI for interactive games.

In conclusion, the paper presents a robust solution to the challenge of creating realistic bot behavior in CS:GO, providing a foundation for future research and practical implementations in the gaming industry. The combination of technical innovation and thorough evaluation sets a high standard for future work in this domain.