Anthropic is giving outside AI safety researchers access to its internal systems at a level typically reserved for employees.
The company announced Oct. 15 that it has onboarded six employees from METR, a nonprofit that evaluates frontier AI models, providing them with company hardware, office space, and access to internal tools comparable to those available to Anthropic's own risk assessment teams.
Outside a narrow set of categories (customer data, attorney-client privileged information and model architecture details), Anthropic employees are encouraged to share information with METR staff just as they would with colleagues working on alignment risk, the company said in a blog post.
METR can publish its findings on Anthropic's risks, incidents and safety practices without the company's prior approval.
Anthropic retains the right to require redactions in those predefined categories, but METR can publicly dispute whether those redactions were justified, giving the evaluators some ability to challenge restrictions on what they disclose.
"In line with our transparency policies, our report will openly describe the scope, access, and other limitations of our work," METR said, per Reuters.
A tiered system for scrutiny
Anthropic is introducing three levels of access for external AI safety evaluators, with different rules governing what researchers can examine and publish.
The first tier covers narrowly scoped investigations that require approval before publication. The second gives evaluators broader access and allows them to publish without prior sign-off, subject to certain redactions.
METR is currently the only organization in the third tier, which provides access to internal systems comparable to that of Anthropic employees working on AI risk assessments.
SecureBio, a biosecurity nonprofit, sits at the first tier and is reviewing the chemical and biological weapons chapter of Anthropic's August risk report. Anthropic is also working with Accenture's AI business Faculty and is in early discussions with Hugging Face.
"We're embedding evaluators voluntarily, and we believe every frontier AI company should do the same," Anthropic said. The company compared the arrangement to banking regulation, where resident examiners work inside the largest banks because those institutions understand their own risk positions better than outsiders do.
The announcement follows Reuters reporting that Anthropic and OpenAI had sought significant control over what outside reviewers could publish, concerned about leaking trade secrets. A key sticking point is that information companies consider intellectual property can also be relevant to assessing model risks.
Experts have raised concerns about evaluator independence, since AI companies themselves choose who conducts the reviews.
Why this sets a precedent
Anthropic's move could provide an early test of whether voluntary third-party evaluations can deliver meaningful scrutiny of advanced AI systems without formal regulatory requirements.
The White House accord signed in late September by Anthropic, Google, Meta, OpenAI, SpaceX and Nvidia encouraged companies to "partner with an independent external auditor or evaluator," but set no binding requirements.
If METR's embedded evaluation produces credible, critical findings, it could become a template competitors feel pressured to match. If it produces soft assessments or gets bogged down in redaction disputes, it may fuel arguments that self-regulation is insufficient.
The graduated tier structure offers a practical blueprint for regulators: rather than demanding full access immediately, oversight could expand as trust and expertise develop. That flexibility may prove essential as AI governance standards evolve.
What independent evaluations could mean for AI security
If you use Anthropic's Claude chatbot, this arrangement won't change your experience directly. But it could affect what you eventually learn about the model's capabilities and limitations.
METR's published assessments may offer the most candid outside view of how Claude handles risky tasks or where it falls short. Users who care about transparency now have a new source of information that Anthropic cannot block, only redact.
For organizations using Claude or evaluating AI systems for sensitive workloads, independent assessments could provide additional insight into how developers identify, test, and address security risks.
METR's access may allow researchers to examine safety practices more closely than external testing alone would permit. However, the value of those assessments will depend on what researchers can investigate, how much they can disclose, and whether their findings lead to meaningful changes.
For security teams, the reports could become another source of evidence when assessing AI providers. They should not replace an organization's own security testing, vendor due diligence, or controls governing how AI systems access sensitive information.
Other news: Firefox 158 addresses security vulnerabilities, including sandbox escape risks that could undermine the browser's protections against malicious code.