The announcement
Google DeepMind introduced SL2T, a massively multilingual sign-language-to-text model, and said it will power new sign language features on Pixel 11 through Gboard and Live Transcribe. The first rollout starts with American Sign Language to English, with more devices and additional languages planned later.
The product idea is straightforward: Deaf and hard of hearing users should be able to sign to a phone anywhere they would otherwise type. In Gboard, that can mean signing a web search, drafting a message or document, or asking Gemini to answer a query or carry out a task. In Live Transcribe, users can sign a response during a conversation instead of typing back and forth. According to DeepMind, testers found ASL signing faster and more natural than typing in English.
Why sign language translation is different
DeepMind frames SL2T as a response to a gap in language technology. Spoken-language AI has enabled dictation, automatic translation, and conversational interfaces for hearing users, but the same progress has not reached the world’s more than 200 sign languages or the estimated 70 million Deaf and hard of hearing people who use them.
A key point is that sign languages are not manual versions of English. They are independent natural languages with their own grammar and vocabulary. That makes the task closer to machine translation than transcription. Speech transcription maps sound to text in the same language; sign-language-to-text must understand visual language and produce a different written language.
The visual problem is also demanding. Meaning can be conveyed through simultaneous movement of the hands, arms, torso, head, and face. This is why glove-based approaches have been limited: they cannot fully capture whole-body motion, facial expression, non-manual markers, and spatial grammar.
How SL2T is built
DeepMind says SL2T was trained on more than 100,000 hours of data across over 50 sign languages, with roughly a quarter of the data in ASL. Training across multiple languages, dialects, and proficiency levels helped the model learn shared structure and, in DeepMind’s experiments, outperform single-language models.
The system uses an on-device MediaPipe Holistic model to track body landmark locations. Instead of sending raw camera video to the server, it sends geometric coordinate sequences for translation, allowing the original video to be discarded immediately. This design is intended to reduce privacy exposure while still giving the model the motion information it needs.
SL2T also translates directly from body landmarks to text, rather than relying on intermediate “glosses.” A gloss is a written label often used to annotate a sign, but gloss-based pipelines can miss rich aspects of sign languages, including facial grammar and spatial constructions. Direct translation removes that artificial bottleneck.
On the FLEURS-ASL sd-test benchmark, which evaluates ASL-to-English translation, SL2T reaches a zero-shot score of 70 BLEURT. BLEURT is a metric used to estimate how semantically close generated text is to reference text. DeepMind says this score is significantly above previously reported results.
From benchmark to product
DeepMind notes that benchmark performance alone does not ensure usability. The team worked on streaming latency, preventing hallucinated text when users are not signing, fairness for the roughly 10% of signers who are left-handed, and one-handed signing, which matters when someone is holding a phone in the other hand.
Examples from FLEURS-ASL show the model producing fluent English translations for topics such as the Cook Islands, income tax, and rugby. The company also lists remaining errors, including rare signs, rapid fingerspelling, passive constructions, classifier depictions, and tense when context is limited. In one example, “prey” becomes “grey,” showing that fine-grained visual recognition remains a hard problem.
What it could change
SL2T is important because it moves sign-language AI from research settings into everyday mobile input. Gboard and Live Transcribe are high-frequency interfaces; if the experience is reliable, signing could become a practical way to interact with search, messaging, documents, and AI assistants.
The next phase will depend on community involvement, data quality, fairness, and privacy. DeepMind says Deaf perspectives shaped the project, including concept work by Deaf Googler Sam Sepah, data collection with Deaf partners, user studies, and impact assessment. The larger test is whether the approach can expand beyond ASL-to-English to more devices and more sign languages without treating highly diverse Deaf communities as a single use case.