A newsletter spotlight on two accountability gaps

MIT Technology Review’s latest edition of The Download centers on two related questions: do we actually know how people use AI systems, and what kinds of public-safety systems are police-tech vendors choosing to build?

The first issue concerns the limits of company-published AI usage reports. Firms such as Anthropic and OpenAI regularly describe how people use their products, but researchers quoted in the newsletter argue that these disclosures show only what the companies decide to release. Without independent data, the public has no clear way to verify whether those reports reflect the full range of user behavior.

A research effort called AI Observatory is trying to address that gap. Its analysis suggests that real-world AI use includes more sensitive behavior than is typically visible in reports from major AI companies, which tend to emphasize workplace and productivity use. In this context, “sensitive” refers to areas where privacy, identity, emotion, relationships, education, or minors may be involved, rather than simply office tasks.

Model choice appears to shape behavior

The project also found meaningful differences across AI systems. According to the newsletter, users were more likely to turn to Anthropic for coding, Gemini for social and roleplay uses, and ChatGPT for homework help. That pattern matters because it challenges the idea that general-purpose chatbots are interchangeable.

Key points from the report include:

• Company AI usage reports lack independent corroboration;
• AI Observatory found more personal and sensitive behaviors;
• Anthropic was more associated with coding;
• Gemini was more associated with social and roleplay use;
• ChatGPT was more associated with homework assistance.

Those distinctions have policy implications. If AI is framed mainly as a workplace productivity tool, debate will focus on business efficiency, enterprise risk, and intellectual property. If many people also use AI for schoolwork, companionship, roleplay, or private decision-making, the discussion expands to children’s safety, emotional reliance, misleading advice, and data protection.

Flock’s design choices, not just its benefits

The second major item examines Flock Safety, a police-technology company known for a US network of roughly 120,000 automatic license plate readers. Automatic license plate readers are camera systems that capture vehicle plates and connect them to time and location data, making them searchable for law-enforcement or security purposes.

Flock recently announced platform changes intended to prevent officers from using the system for illegal or illegitimate purposes, including stalking. Defenders of the company often argue that the cameras may help solve crimes, or even help catch a kidnapper, while for most drivers they merely capture images that no one will ever examine.

The newsletter argues that this misses the deeper question: what kind of crime-fighting architecture has Flock chosen to create? The system works as it does because of decisions about what data to collect, who can search it, how long information is retained, and how broadly it can be shared. Those product and governance choices define the bargain between security and civil liberties.

The broader test: auditability

The AI usage debate and the Flock controversy belong to different sectors, but they reveal the same structural problem. When platforms control data, access, interfaces, and public narratives, outsiders struggle to determine how systems are actually used and where boundaries should be set.

The same newsletter also points to wider technology pressures: a multi-state child privacy case against Meta, Nvidia’s commitment of up to $105 billion to OpenAI’s Ohio data center, a facility expected to cost up to $500 billion and come online in 2028, and a report that women made up only 26% of new AI hires last year. Together, these items show that technology competition is increasingly about infrastructure, governance, labor, privacy, and power.

The next phase will not be defined only by stronger AI models or larger sensor networks. It will depend on whether powerful systems can be independently examined. AI companies will face growing pressure to support privacy-preserving research into real usage, while surveillance vendors will need to demonstrate enforceable limits on access, retention, and sharing. Trust will depend less on reassuring claims and more on whether these systems can be questioned, audited, and corrected.