Real-Time Collaborative
Sign Language To Speech
Collect 30-frame holistic gesture sequences collaboratively with your team in Supabase, train LSTM models in Google Colab, and translate sign language to speech live in your browser.
1. Collaborative Data Studio
Record 30-frame sequences containing 258 landmarks (Pose + Both Hands) directly to Supabase. Multiple teammates can record simultaneously from different devices.
2. Live Sentence Translator
Run real-time LSTM sequence predictions in browser. Aggregates recognized gestures into full sentences, reorders grammar, and speaks with Web Speech TTS.
3. Session Log & Replay
Review past translated conversations, replay speech audio, export logs, and analyze translation history over time.
How SignSpeak Works
Track (30 Frames)
MediaPipe Holistic extracts 258 normalized features per frame (132 Pose + 63 Left Hand + 63 Right Hand) with face landmarks explicitly disabled.
Train (Supabase + Colab)
Sequences are stored in Supabase in real-time. Export the full JSON dataset to train an LSTM model in Python / Google Colab for client deployment.
Translate (TFJS + Speech)
Run rolling 30-frame sequence inference using TensorFlow.js in your browser. Finalized sentences are converted to speech via Web Speech API.