WhatsApp Begins a Limited On-Device AI Anti-Scam Test
WhatsApp is running a limited test of Scam Alert, an optional anti-scam feature. Its central design choice is that scam classification for messages from non-contacts happens locally on the device, with message content kept on the device during classification. Users can choose whether to enable it.
Key technical highlights:
- On-device processing: A small machine learning model is downloaded and runs locally
- Differential privacy: Minimum group thresholds and differential privacy are applied to aggregated telemetry
- Confidential computing: A Confidential VM, a form of trusted execution environment, handles performance measurement
- Transparency mechanism: Production and experimental model versions, along with their SHA-256 hashes, are published to a third-party append-only transparency ledger before deployment
- Model verification: Clients verify ledger entries, model signatures, freshness, and hashes before loading a model
Workflow and Privacy Architecture
Scam Alert can be understood in two parts: real-time detection and performance evaluation.
Detection phase: Once enabled, when a non-contact sends a message, the on-device model evaluates it using conversational structure and language signals. Meta says the model is trained on patterns observed in scam conversations previously reported to WhatsApp by users. If a message is flagged as suspicious, the user sees a warning that is invisible to the sender and can choose to block, report, or continue the conversation. Users can also mark a chat as “trusted,” after which Scam Alert will no longer flag that conversation. They may also choose to share the five most recent messages from trusted chats with WhatsApp to help improve the feature.
Performance evaluation phase: Devices locally aggregate warning events and user actions into counts. These metrics are transmitted with anonymous credentials through an Oblivious HTTP (OHTTP) relay and processed inside a Confidential VM. Before transmitting data, the client verifies the code running in the confidential environment and checks privacy parameters. The aggregation process applies minimum group thresholds and differential privacy, after which WhatsApp receives only approximate, population-level statistics.
A notable part of the design is that model distribution is treated as a security boundary. Each production or experimental model version and its SHA-256 hash are published to a third-party append-only transparency ledger before deployment. Before loading a model, clients verify the ledger entry, signature, freshness, and hash. Model downloads use anonymous credentials and OHTTP, while experiment assignment happens locally, preventing the server from selecting a specific model variant for an individual user.
Comparison: Google Messages and WhatsApp On-Device Detection
The original report notes that Google Messages uses a similar approach for real-time spam protection against scams and phishing, along with privacy-preserving mechanisms. However, the architectural boundaries differ by feature, including whether processing happens locally or through Google services.
| Aspect | WhatsApp Scam Alert | Google Messages |
|---|---|---|
| Detection location | Message classification happens on device | Varies by specific safety feature |
| Privacy mechanisms | Confidential computing + differential privacy + OHTTP + transparency ledger | Privacy-preserving mechanisms; some features use on-device detection |
| Training data source | Patterns from user-reported scam conversations | Not specified in the source |
| Model verification | Client verifies SHA-256 hash, signature, and transparency ledger entry | Not specified in the source |
The shared goal is to improve scam and phishing detection without relying on direct cloud-side scanning of message content. Still, each platform draws different boundaries between on-device processing, cloud services, and telemetry handling.
What It Means for Users
For people who often receive messages from unknown contacts, Scam Alert could provide an additional layer of risk warning if it becomes more broadly available. It does not automatically report conversations: users remain in control and can choose to block, report, continue chatting, or mark chats as trusted.
The feature is still in limited testing. Users should keep two caveats in mind: models can produce false positives or miss some scams, and while classification happens locally, users who opt to help improve the feature may voluntarily share the five most recent messages from trusted chats.
Final Thought
Anti-scam features are expanding from “cloud-side real-time scanning” toward “on-device pattern recognition,” reflecting a new balance between safety and privacy in messaging platforms. Meta says the limited test will let the company and its Bug Bounty community stress-test the implementation before broader rollout. The company is also expanding Bug Bounty coverage to include the confidential federated analytics pipeline and plans to release Confidential VM binaries and privacy-related source-code components for independent security review.