AI Observatory: Anthropic work filter would drop 48% of chats
This digest was compiled by AI from multiple sources — links to the originals are below.
The AI Observatory applied the Anthropic Economic Index's work-related filter to seven existing datasets of consented AI conversations with Claude and Gemini from 2023 to 2025 and found 48% of conversations would have been filtered out. The excluded non-work conversations carried higher rates of health and relationship topics, adult content, harassment and sexual content than Anthropic's published analysis. OpenAI's 2025 ChatGPT report similarly counted only 30% of consumer use as work-related.
Key Facts
- The AI Observatory analyzed real AI conversations with Claude and Gemini gathered through seven existing datasets with user consent, covering 2023–2025.
- Applying Anthropic Economic Index's work-related filter excluded 48% of conversations in the AI Observatory dataset.
- Excluded non-work chats showed higher rates of health and relationships (44.2% vs 31.2%), adult or illicit topics (7.9% vs 2.1%), harassment and hate (27.5% vs 5.66%), and sexual content (16.7% vs 2.4%) than Anthropic's analysis.
- OpenAI's 2025 report on ChatGPT found only 30% of consumer use was work-related.
- In the WildChat dataset, prompt tokens, response tokens, and conversation turns increased, while AI assistants' self-disclosure decreased and small talk rose.
AI Observatory Scope
The AI Observatory is a public platform co-led by Reuel that aggregates and analyzes real AI conversations with models such as Claude and Gemini. Its dataset draws on seven existing collections built with users' consent and covers conversations from 2023 through 2025. The project aims to give researchers and policymakers independent data on how people actually use generative AI, beyond vendor reports. Reuel argues that consequential decisions about AI benefits and risks are currently made on the basis of very limited data.
Anthropic Index Blind Spots
The Anthropic Economic Index is widely cited but focuses on work- and productivity-related uses of Claude and filters out other conversations. When AI Observatory researchers applied Anthropic's filtering method to their dataset, 48% of conversations would have been removed. The removed non-work conversations were more likely to involve health and relationships (44.2% versus 31.2%) and adult or illicit topics (7.9% versus 2.1%). They were also more likely to include harassment and hate (27.5% versus 5.66%) and sexual content (16.7% versus 2.4%). OpenAI's 2025 ChatGPT report similarly found that only 30% of consumer use was work-related.
Conversation Trends
WildChat, one of the largest datasets in the AI Observatory study, showed conversations got longer and more elaborate over time. Growing numbers of prompt tokens, response tokens, and conversation turns indicate that AI companionship was increasing. Small talk rose significantly, while AI assistants' self-disclosure, such as admitting to being a chatbot, decreased. Exchanges labeled sensitive, including sexual harassment and hate speech, became less frequent over the study period.
1 source
AI Observatory: Anthropic work filter would drop 48% of chats
