Gray Swan AI Welcomes U.S. AI Safety Institute to the UK AISI Agent Red-Teaming Challenge

We're excited to announce that the U.S. Al Safety Institute (US AISI) has officially joined the UK AISI Agent Red-Teaming Challenge as a co-judge.

Gray Swan
April 2, 2025

We're excited to announce that the U.S. Al Safety Institute (US AISI) has officially joined the UK AISI Agent Red-Teaming Challenge as a co-judge. Alongside the UK AISI, US AISI will help evaluate submissions focused on Al agent failures, instruction bypass, misuse risk, and over-refusals-helping ensure the challenge maintains the highest standards of fairness and transparency.

A Global, Multi-Stakeholder Effort

This challenge is now supported by some of the most influential organizations in Al:

  • Al Security Institute
  • OpenAl|
  • Anthropic Al
  • Google DeepMind

The prize pool has grown to $170,000, making this the largest Al red-teaming challenge of its kind.

What Is the UK AISI Agent Red-Teaming Challenge?

The challenge tasks participants with identifying vulnerabilities in anonymous Al agents-testing their ability to:

  • Breach confidentiality
  • Override aligned goals
  • Trigger disallowed actions
  • Expose model weaknesses under pressure
  • Identify over-refusals in edge-case scenarios

Participants use both direct and indirect exploit techniques, simulating the kinds of threats real-world agents may face in production.

We're currently in Wave 4, the final phase of the month-long challenge. New behaviors have been introduced, and submissions remain open through April 6.

Jump into the Arena

Heading 1

Heading 2

Heading 3

Heading 4

Heading 5
Heading 6

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur.

Block quote

Ordered list

  1. Item 1
  2. Item 2
  3. Item 3

Unordered list

  • Item A
  • Item B
  • Item C

Text link

Bold text

Emphasis

Superscript

Subscript