SUT · Teaching Iranian Sign Language via VR

Developed a real-time AI-driven VR educational system for teaching Iranian Sign Language (ISL) to hearing individuals, addressing the communication barrier faced by the hearing-impaired community through an immersive game-based learning environment. The system recognizes 8 ISL hand gestures from a standard webcam feed and translates classifications into in-game commands in real time — enabling natural, interaction-driven language acquisition without requiring specialized hardware.

Two deep learning architectures were trained and benchmarked: a Convolutional Neural Network (CNN) achieving 99.7% test accuracy and a Long Short-Term Memory (LSTM) network achieving 99.1% test accuracy, both implemented in Python/TensorFlow. The recognition pipeline processes each video frame live and communicates classified gestures to the Unity game engine, with the VR environment deployed on Oculus Quest 2. The AI module and VR runtime were synchronized over a local socket connection for sub-frame latency.

The system was validated through a controlled user study tracking participant performance across play sessions. Statistical analysis of the recorded progression data revealed that players sustained active gameplay 195.36% longer in the second half of their sessions compared to the first — demonstrating significant, quantifiable learning improvement. This work was published and delivered as an oral presentation at IEEE ICRoM 2022 (10th RSI International Conference on Robotics and Mechatronics), Sharif University of Technology.