Naser Jawas, Tardi Tjahjadi
Accurate hand tracking is a fundamental requirement for sign language recognition and human computer interaction. However, reliable tracking remains challenging due to the high degrees of freedom of hand motion, frequent shape variations, and the need to localise both hands simultaneously under varying imaging conditions. This paper proposes an unsupervised, model free hand tracking framework that exploits dense optical flow and temporal consistency to estimate hand locations without requiring any training data. The method employs a multi-scale dense optical flow representation in which a colour-encoded motion map separates dominant hand motion from the background. Clustering and contour analysis are then applied to generate hand candidates, followed by motion refinement and validation using tracking information from previous frames to ensure temporal robustness. The proposed approach is evaluated on four public sign language datasets: RWTH-Boston-50, RWTH-Boston-104, RWTH-PHOENIX-Weather, and the BBC-Oxford British Sign Language (BOBSL) dataset. Quantitative comparisons are conducted against state-of-the-art learning-based detectors, including YOLOv3, YOLOv11, MediaPipe, and MMPose, as well as classical tracking baselines, using centre-of-location error, tracking error rate, and computational speed. Results show that although learning-based methods achieve the highest accuracy, the proposed unsupervised tracker attains competitive performance, stable tracking under fast motion and motion blur, and consistent cross-dataset generalisation without training or domain adaptation, offering a favourable trade-off between accuracy and computational complexity for data-limited or resource-constrained sign language recognition systems. © 2013 IEEE.
Faculty of Computer Science, Universitas Brawijaya, East Java, Malang, 65145, Indonesia; School of Engineering, University of Warwick, West Midlands, Coventry, CV4 7AL, United Kingdom
