Deep Dive — Google ships the first sign-language AI in a consumer product
For the first time, a sign language is a first-class input on a phone. DeepMind's new SL2T model translates American Sign Language into English text inside Gboard and Live Transcribe on the Pixel 11 — the biggest accessibility milestone in AI this year, and a far harder engineering problem than the launch-day gloss suggests. It is also a story with open questions: whose data trained it, whose benchmark proves it, and who actually gets to use it.
The milestone: sign language finally gets its dictation moment
Speech AI spent a decade turning spoken language into a commodity interface — dictation, live translation, voice assistants — while the estimated 70 million Deaf and hard of hearing people who use one of the world's more than 200 sign languages got research demos. That gap was never for lack of effort. Early attempts like sign-language gloves failed because they treated signing as "English on the hands," when sign languages are independent languages that carry meaning simultaneously through hands, face, head, and torso. The problem was always full-body visual perception plus real translation, not gesture recognition.
SL2T — sign-language-to-text — is DeepMind's answer, and it is the first model of its kind to reach a consumer product. Announced August 12, it powers sign-to-text dictation in Gboard and Live Transcribe on the Pixel 11, starting with ASL to English, at no extra cost. A user signs at the camera and gets streaming text they can use anywhere they would normally type: web searches, drafts, documents, even prompts to Gemini. In Live Transcribe, a Deaf user can sign replies during a live conversation instead of typing back and forth. Google says its testers found signing in ASL faster and more natural than typing in English.
The launch is part of the Pixel 11 event, where Google's entire pitch is AI — we covered the agentic Gemini push this morning — but SL2T is the rare feature in that lineup that is both genuinely useful and genuinely hard to copy. It was also built differently: conceptualized by Sam Sepah, a Deaf Googler, with an AI Sign Language Advisory Committee (AISLAC) of global Deaf organizations shaping the release, and a joint impact report published alongside SL2T 1.0 that spells out capabilities and limitations.
Landmarks, not video — and no glosses
Three design decisions separate SL2T from the research that came before it.
First, scale and multilingualism. SL2T was trained on more than 100,000 hours of signing across more than 50 sign languages, roughly a quarter of it ASL. Google says training jointly on many languages, dialects, and proficiency levels teaches the model shared underlying structure, and that it outperformed single-language models in its own experiments. That data advantage is the moat: nobody else has a comparable multilingual sign corpus.
Second, privacy by architecture. An on-device model (MediaPipe Holistic) tracks pose landmarks — points on the signer's face, hands, arms, and torso — and only those geometric coordinates are sent to the server for translation. The raw camera feed is discarded immediately. Video never leaves the phone; what leaves it is a skeleton of coordinates.
Third, direct translation. Prior sign-language systems typically translated through "glosses" — intermediate sign-by-sign word annotations. Glosses flatten the language: they miss non-manual markers (facial expression, head movement) and spatial constructions that carry grammatical meaning. SL2T translates straight from the landmark sequence to text, which removes the artificial vocabulary ceiling and lets quality scale with data.
The headline number is 70 BLEURT, zero-shot, on the FLEURS-ASL benchmark (sd-test) — a score Google says is far beyond anything previously reported. The unglamorous engineering is just as important: minimized streaming latency, hallucination suppression when the input isn't signing at all, fairness tuning for the roughly 10% of signers who are left-handed, and support for one-handed signing so users can hold the phone in the other hand.
Google is also candid about failure modes, which is rare and welcome. Its own examples show errors on rare signs, rapid fingerspelling (a "prey" that came out as "grey"), passive constructions, classifier handshapes (a dropped "claws"), and tense without context. The system is good; it is not fluent.
Who wins, and who waits
The clear winners are Deaf users, who get the dictation parity hearing users have had for a decade — signing to search, write, and command an agent, plus two-way conversation through Live Transcribe. That is a real capability, not an accessibility patch.
Google wins on strategy. SL2T lands in the same week Apple is preparing a rebuilt, Gemini-powered Siri — Google remains one of Apple's biggest AI suppliers while competing with it head-on — and accessibility is the one axis where Google can define the category before Apple can respond. The data moat matters too: a 100,000-hour, 50-plus-language training set plus the on-device vision stack plus the devices themselves is an integrated advantage no rival currently matches. Google says it is the first sign-language model in a consumer product, and nothing comparable is on the market.
The waitlist is long, though. SL2T launches with ASL only; "additional languages will follow" with no dates or roadmap. It launches on Pixel 11 only; "more devices coming soon," again undated. And it launches on a phone that starts at $899 — the accessibility milestone of the year arrives first on the most expensive device, which is a choice worth noticing. The people with the most at stake are the least likely to be early adopters of a flagship.
The skeptical case
The benchmark is partly Google's own yardstick. FLEURS-ASL is a Google dataset hosted on Kaggle, and the 70 BLEURT claim appears in a blog post, not a peer-reviewed paper. Zero-shot scores also say little about real conditions: a noisy café, a moving bus, a signer with a strong regional dialect, or lighting that confuses pose tracking.
The data question is the one Google has not answered. One hundred thousand hours of signing — collected with Deaf partners, the company says — but the composition, provenance, and consent terms are undisclosed. The AISLAC committee and the impact report are genuine governance signals; they are not veto power, and they were published alongside a model that ships regardless. "Nothing about us without us" is the standard the Deaf community has set, and shipping first while reporting alongside is a reasonable first step — but only a first step.
And privacy-by-design has limits worth naming. Coordinates are not video, but they are still a record of how a person moves — a behavioral signature — and cloud translation means Google's servers observe your signing patterns. Sending a skeleton instead of footage is real progress; it is not the same as on-device translation, which Google does not claim.
Finally, the error stakes scale with context. For a text message, a mistranslated classifier is a nuisance. In a hospital or a courtroom, a wrong sign is a wrong answer with consequences — and Google's own examples show meaning-changing errors are still routine. "Good enough for texting" is a fine first milestone. Where the line gets drawn next is the real test.
What to watch: which sign languages follow ASL and when; sign-language generation (the reverse direction, text to signing, which is substantially harder); the promised expansion beyond Pixel; whether FLEURS-ASL becomes the field's standard benchmark or stays Google's home turf; and how Apple, Meta, and Nvidia respond — none of them has shipped anything comparable. The deeper question is whether Google keeps publishing honest limitations reports for every major release, and whether the Deaf community's trust grows faster than the inevitable bad translations.
Sign-language AI is finally in the wild. The question is whether it earns the Deaf community's trust — tell us in the comments.
Sources: Google DeepMind · AISLAC joint impact report (PDF) · SiliconANGLE · Techmeme · AI Midday — Pixel 11 brief · CNBC
