Deep Gait Recognition: A Survey

Abstract: Gait recognition is an appealing biometric modality which aims to identify individuals based on the way they walk. Deep learning has reshaped the research landscape in this area since 2015 through the ability to automatically learn discriminative representations. Gait recognition methods based on deep learning now dominate the state-of-the-art in the field and have fostered real-world applications. In this paper, we present a comprehensive overview of breakthroughs and recent developments in gait recognition with deep learning, and cover broad topics including datasets, test protocols, state-of-the-art solutions, challenges, and future research directions. We first review the commonly used gait datasets along with the principles designed for evaluating them. We then propose a novel taxonomy made up of four separate dimensions namely body representation, temporal representation, feature representation, and neural architecture, to help characterize and organize the research landscape and literature in this area. Following our proposed taxonomy, a comprehensive survey of gait recognition methods using deep learning is presented with discussions on their performances, characteristics, advantages, and limitations. We conclude this survey with a discussion on current challenges and mention a number of promising directions for future research in gait recognition.

• The paper presents a comprehensive survey of deep gait recognition methods, introducing a novel taxonomy based on body, temporal, feature, and neural dimensions.
• It evaluates state-of-the-art neural architectures like CNNs, GANs, and RNNs using benchmarks such as the CASIA-B and OU-MVLP datasets.
• The study highlights challenges including adversarial robustness and cross-dataset generalization, outlining promising future research directions.

Deep Gait Recognition: A Survey

The paper, "Deep Gait Recognition: A Survey" (2102.09546), provides a comprehensive review of the advancements in gait recognition using deep learning methodologies. The focus lies on vision-based systems, analyzing datasets, test protocols, state-of-the-art deep learning solutions, challenges, and potential future research directions in this domain.

Introduction to Gait Recognition

Gait recognition is a unique biometric modality relying on the distinct manner of an individual's walk. It has significant applications in security, healthcare, and more. Deep learning architectures have largely transformed gait recognition by autonomously learning discriminative features, surpassing traditional methods.

Figure 1: The number of gait recognition papers published after 2015 using non-deep and deep gait recognition methods.

The dominance of deep learning in recent research is clearly depicted in Figure 1.

Evolution and Taxonomy in Gait Recognition

Evolution of Deep Gait Recognition

The evolution of deep learning methodologies in gait recognition is graphically depicted, showcasing the transition and improvement over time.

Figure 2: The evolution of deep gait recognition methods.

Taxonomy

The authors propose a novel taxonomy for deep gait recognition, segmented into four dimensions: body representation, temporal representation, feature representation, and neural architecture (Figure 3). This taxonomy provides a structured framework for understanding the technology landscape in this field.

Figure 3: Our taxonomy consisting of 4 dimensions: body representation, temporal representation, feature representation, and neural architecture.

Core Components and Approaches

Body and Temporal Representation

Gait recognition primarily adopts silhouettes and skeleton-based representations. Silhouettes are widely used due to their ease of computation. Temporal representations involve either static templates like Gait Energy Images (GEI) or dynamic sequence volumes.

Figure 4: Overview of temporal representations in gait recognition using various stages of neural networks.

Neural Architectures

Multiple deep learning structures are utilized, including CNNs, GANs, RNNs, and hybrid models that combine these architectures. These networks are tailored for learning both spatial and temporal characteristics of gait data.

Figure 5: Different approaches for using RNNs in the context of deep gait recognition systems.

Performance Evaluation

A detailed examination of test protocols across various datasets is provided (Figure 6), notably distinguishing between subject-dependent and independent evaluations.

Figure 6: An overview of test protocols in gait recognition.

Datasets

Gait datasets are critical for assessing deep learning solutions, with the most commonly used being CASIA-B and OU-MVLP. They significantly contribute to benchmarking performance and generalization.

Challenges and Future Directions

The paper highlights several challenges:

• Disentanglement and Self-supervised Learning: Effective representation requires disentangling identity from other confounding factors and leveraging unlabeled data.
• Adversarial Robustness: There is a challenge in addressing gait recognition's vulnerability to adversarial attacks.
• Synthesized Data and Multi-task Learning: Generating realistic synthetic data and employing multi-task learning paradigms offer promising avenues for enhancing robustness and applicability.
• Cross-Dataset Evaluation: Ensuring recognition systems generalize across different data domains is crucial.

Conclusion

This paper systematically explores the advancements and state-of-the-art methodologies in deep gait recognition. It categorizes existing technology, evaluates against popular datasets, and outlines unresolved challenges and innovative directions that can facilitate enhanced real-world applications and robustness in gait recognition systems.

Figure 7: Visualization and frequency of different neural architectures, loss functions, and gait datasets used in literature.