.png)
Sign to Text
Real-time ASL hand gesture recognition via webcam, converts signs to text output
Sign to Text translates American Sign Language (ASL) gestures into text in real time using the device webcam. The system uses MediaPipe hand landmark detection on the frontend and a gesture classification model on the backend, capturing frames every 300 ms and committing a sign only after it has been held consistently for a configurable number of readings.
Supported Signs
Letters
The following 16 static ASL letter signs are supported:
A B C D F I K L O R S U V W X Y
Note: J and Z require motion strokes and are not supported in static-frame mode.
Word Gestures
| Gesture | Sign Description |
|---|---|
| HELLO | All fingers spread open |
| GOOD | Thumbs up |
| BAD | Thumbs down |
| I LOVE YOU | Thumb + index finger + pinky |
| STOP | Open flat palm facing forward |
| ROCK ON | Index finger + pinky extended |
| OK | Thumb and index finger form a circle |
How It Works
- Camera initialises on page load — allow camera access when prompted.
- Click Start Recognition — the system begins capturing frames every 300 ms.
- Each frame is sent to the backend
/sign-language/detectendpoint. - A stability filter requires the same sign to appear
3consecutive times before it is committed to the text output. This prevents accidental or partial gestures from being written. - Word gestures are added with surrounding spaces; letters are appended directly.
Text Output Controls
| Button | Action |
|---|---|
| Space | Manually insert a space between words |
| Delete | Remove the last character |
| Clear | Clear the entire text buffer |
| Copy | Copy the full text to clipboard |
Best Practices
- Ensure your hand is clearly visible and well-lit against a contrasting background.
- Hold each sign steady for at least 1 second for reliable detection.
- Position your hand roughly in the centre of the camera frame.
- For letter-by-letter spelling, pause briefly between each letter to allow the stability counter to reset.
Limitations
- Only one hand is tracked at a time.
- Signs requiring motion (J, Z) are not supported.
- Accuracy may decrease in low-light conditions or with fast hand movement.