'In-House Evaluation Is Not Enough' — third-party AI flaw disclosure framework
- Type
- Public reporting channel
- Place
- International — Global (norm-setting, not binding)
- Last checked
- 2026-09-22
- Next check due
- 2027-03-21
Ways to reach them
- Information only · 2026-09-22
Submission page
- Recipient
- 'In-House Evaluation Is Not Enough' — third-party AI flaw disclosure framework
- Who may use it
- Members of the public
- What it accepts
- AI model flaws; Systemic AI risks
Information only. This link provides background or evidence; it is not a place to send a message.No contact action is available for this entry. - Information only
Homepage
- Recipient
- 'In-House Evaluation Is Not Enough' — third-party AI flaw disclosure framework
- Who may use it
- Members of the public
- What it accepts
- AI model flaws; Systemic AI risks
Information only. This link provides background or evidence; it is not a place to send a message.No contact action is available for this entry.
What it does
N/A — it is the proposal document that defines the emerging norm. Read it to understand the vocabulary that labs and coordinators are converging on.
Honest assessment
This is where the coordinated flaw disclosure push landed: as of September 2026 the standardised report and routing layer exist (FLARE-AI, live July 2026), but broad statutory or contractual safe harbour for AI model-behaviour research does not. Labs still grant safe harbour only inside their own programs, on their own terms.
How to file
Three proposals: (1) standardised flaw reports and published rules of engagement, so researchers know what they may test and how to report; (2) broadly-scoped disclosure programs with legal safe harbours modelled on bug bounties; (3) coordinated distribution infrastructure, because jailbreaks and flaws transfer across providers. Its practical successor is FLARE-AI, which implements the standardised report and the distribution layer.
Format
Academic position paper (arXiv 2503.16861, March 2025).
Timing
N/A
What happens after
N/A
What it accepts
N/A — it is the proposal document that defines the emerging norm. Read it to understand the vocabulary that labs and coordinators are converging on.
What it does not accept
N/A
Operated by
Shayne Longpre, Kevin Klyman, Ruth E. Appel and 30+ co-authors (MIT, Stanford HAI/CRFM, Princeton and others); endorsed by HackerOne