Anthropic and Accenture plan $2bn AI safety effort

Anthropic said Friday it is partnering with Accenture to have outside evaluators work inside the AI company, moving toward a pledge by chief executive Dario Amodei to let independent observers assess its most powerful models.

The work will be led by Faculty, Accenture's specialist AI business, and will include evaluating and red-teaming Anthropic's models. Red-teaming means deliberately trying to make AI systems misbehave to expose flaws.

Anthropic and Accenture each expect to invest at least $1 billion over the next five years. Anthropic said it would fund Accenture's work directly because of the “importance and urgency” of the effort, but said longer-term funding should come from pooled or government sources.

Anthropic said it is also in talks with Model Evaluation and Threat Research, a California-based nonprofit known as METR, to take part in some embedded evaluation work using METR's own funding.

Critics, including former White House AI adviser David Sacks, have questioned METR's independence because of its ties to Anthropic's investors and staff. METR says it takes no funding from AI companies or their executives.

More than 100 AI researchers, including Geoffrey Hinton, signed a public letter Friday calling for evaluators embedded in AI companies to be meaningfully independent, including by not being owned or governed by frontier AI companies and not accepting rewards tied to their findings.