Sample RGB images from the BRIAR dataset, showing challenging real-world conditions.
Gait is becoming popular as a method of person re-identification because of its ability to identify people at a distance. However, most current works in gait recognition do not address the practical problem of occlusions. Among those which do, some require paired tuples of occluded and holistic sequences, which are impractical to collect in the real world. Further, these approaches work on occlusions but fail to retain performance on holistic inputs. To address these challenges, we propose RG-Gait, a method for residual correction for occluded gait recognition with holistic retention. We model the problem as a residual learning task, conceptualizing the occluded gait signature as a residual deviation from the holistic gait representation. Our proposed network adaptively integrates the learned residual, significantly improving performance on occluded gait sequences without compromising the holistic recognition accuracy. We evaluate our approach on the challenging Gait3D, GREW and BRIAR datasets and show that learning the residual can be an effective technique to tackle occluded gait recognition with holistic retention.
The core challenge in occluded gait recognition is the absence of complete information about a subject's walking pattern. Our method, RG-Gait, addresses this through a three-module architecture that dynamically adapts to different levels of occlusion. RG-Gait consists of three key components: (1) an Occlusion Evaluation Module (OEM) for detecting and quantifying occlusions, (2) a Feature Restoration Network (FRN) for compensating missing information, and (3) a Gait Signature Extractor (GSE) for capturing identity-specific gait features. We model the occluded gait features as a residual deviation from the holistic features. The FRN learns this residual, and it is adaptively integrated into the final gait signature based on the occlusion level detected by the OEM. This allows the model to rely on direct gait features when silhouettes are complete, while incorporating more restorative residual features when occlusions are significant.
Following previous works, we experiment with synthetic occlusions of different types. Specifically, we randomly occlude the top, bottom, or middle part of the subject in the input silhouette. The size and position of the occlusion are chosen randomly during training and testing. We also introduce a moving occlusion, where a black mask of varying size moves laterally across the video at different speeds, occluding different parts of the subject within the same sequence. We follow settings where up to 60% of the subject may be occluded. Importantly, we introduce these occlusions randomly, without the need for paired (occluded, complete) data during training.
We evaluate our method using multiple metrics. Rank-Retrieval accuracy (Rank-1, Rank-5) measures how well an unseen probe sample matches an identity in a gallery. To address the issue that occlusion training can reduce performance on non-occluded (holistic) data, we introduce the Holistic Performance Retention (HPR) evaluation, which measures how much performance is retained on complete data after occlusion training. We also use the Relative Performance (RP) metric, which evaluates a model's accuracy on occluded data relative to the upper-bound performance on holistic data, providing a standardized measure of how effectively a method handles occlusions.
Furthermore, we evaluate the generalizability and adaptability of our approach by testing on new, unseen occlusion types. Generalizability refers to a model's ability to handle new occlusions without any retraining, while adaptability measures how well a model can perform on new occlusions after being fine-tuned on them.
Our main results show that RG-Gait consistently performs better than other baseline methods on the occluded gait recognition task. This holds true across challenging datasets like Gait3D, GREW, and BRIAR, and when using different model backbones such as GaitBase and DeepGaitV2. We also tested our method with a transformer-based backbone, SwinGait, and observed that our approach was still more effective than the baselines. These results demonstrate that learning a corrective residual feature is an effective strategy for occluded gait recognition.
A key goal of our work was to handle occlusions without sacrificing performance on complete, non-occluded (holistic) data. When we evaluate models trained for occlusion on holistic data, we observe that other methods experience a significant performance drop compared to the original backbone's capability. In contrast, our method, RG-Gait, is specifically designed to retain most of the holistic performance of the original backbone, even after being trained on occluded data.
For a gait recognition method to be practical, it must handle diverse occlusion types, but it is not feasible to train a model on all possible variations. We therefore test our model on new kinds of occlusions it has not seen during training. Generalizability refers to how well a model performs on new occlusions in a zero-shot scenario, without any retraining. Adaptability refers to the model's performance on new occlusions after it has been fine-tuned on them. Our experiments, which involve training on top and bottom occlusions and evaluating on middle and moving occlusions, show that RG-Gait can generalize and adapt to new occlusion types better than other methods.
We perform ablation studies to analyze the effect of the different components in our framework: occlusion residual learning, adaptive feature integration, and occlusion awareness. Our experiments show that the Feature Restoration Network (FRN), which is responsible for learning the residual, is essential for performance on occluded data. Furthermore, adaptively integrating the residual features based on the occlusion score from the OEM improves results, indicating that the learned residual is more important in some scenarios than others. Finally, providing occlusion awareness to the FRN in the form of a feature from the OEM further improves the quality of the generated residual. We find that all components complement each other to boost overall performance when used together.
@inproceedings{gupta2025ijcb,
title={Mind the Gap: Bridging Occlusion in Gait Recognition via Residual Gap Correction},
author={Ayush Gupta and Siyuan Huang and Rama Chellappa},
booktitle={2025 IEEE International Joint Conference on Biometrics (IJCB)},
year={2025}
}