Humane Intelligence Algorithmic Bias Bounty
CHALLENGE 1
Stop bad LLM output before it happens!
Humane Intelligence is thrilled to be launching the first of 10 "algorithmic bias bounty" programs, which will be unfolding over the coming year. With the support of Google.org, we are building themed programs that aim to build community and professionalize the practice of algorithmic assessment.
This challenge is now closed. Thank you to all who participated.
Bias Bounty 1 Winners!
We are thrilled to announce the winners of our very first Bias Bounty Challenge! This challenge was designed to fine-tune automated red teaming models and explore issues like bias, factuality, and misdirection in Generative AI.
A huge thank you to all participants who contributed their expertise and congratulations to our winners!.
We are so proud of the innovative approaches and dedication shown by all participants.
Advanced:
Yannick Daniel Gibson (Factuality)
Elijah Appelson (Misdirection)
Gabriela Barrera (Bias)
Intermediate:
AmigoYM (Factuality)
Mayowa Osibodu (Factuality)
Simone Van Taylor (Bias)
Beginner:
Blake Chambers (Bias)
Eva (Factuality)
Lucia Kobzova (Misdirection)
The Challenge
Our first challenge builds on the evaluation and dataset from our Generative AI Red Teaming Challenge: Transparency Report. Challenge participants can win from a pool of about $10,000 in prizes for beginner, intermediate, and advanced submissions.
The final task? Create a probability estimation model that determines whether the prompt provided to a language model will elicit an outcome that demonstrates factuality, bias, or misdirection, inspired by the data from our Generative AI Red Teaming Challenge: Transparency Report.
Regardless of your skill or ability in algorithmic assessment, we have a competition for you.
Beginner
Pick one of the three datasets. Identify gaps in the data and suggest new categories of data that would make the dataset more representative. Generate five prompts per subject area that will elicit a bad outcome. You will be graded both on the number of new topics as well as the diversity of the prompts produced.
Intermediate
After completing the beginner task, generate synthetic data to fill in the gaps in the dataset that you’ve identified. You’ll generate the synthetic data instead of manually writing five prompts per topic. You will be graded both on the number of new topics as well as the diversity of the prompts produced.
Advanced
With your new dataset, generate a likelihood estimator. This model should provide a likelihood (in other words, a probability) that a given prompt would elicit a bad outcome in your topic area. You will be graded against a holdout dataset to determine the accuracy of your model.
Choose wisely! We have nine prizes, but you can only enter one challenge.
The Details
Data: We’ve split the GRT data into three separate and broad categories: factuality, bias, and misdirection. You can only choose ONE dataset to work on per competition.
These are defined as:
1. Factuality
Factuality refers to the model's ability to discern reality from fiction and provide accurate outcomes. For the purposes of the challenge, we focus on examples that could be harmful, rather than simply humorous. These include challenges on political misinformation, defamatory information, and economic misinformation.
2. Bias
Bias analysis demonstrates and explores model biases. That is, we asked the user to elicit scenarios that would broadly be considered defamatory or socially unacceptable by perpetuating harmful stereotypes. This topic includes data on: demographic negative biases, demographic stereotypes, and Human rights violations.
3. Misdirection
Misdirection analyses include incorrect outputs and hallucinations that could misdirect or mislead the user. Our misdirection dataset includes contradictions/internal inconsistencies, multilingual inconsistencies, citizen rights misinformation, and overcorrection.
You can find the data here. Again, we strongly suggest you look at how the data was collected, by reading the Transparency Report before you start answering the challenge. Note that we did not include all the GRT questions in the datasets for the challenge, so please use our datasets for the challenge and not the original GRT dataset.
Sign up below, under "Sign up to participate!" for more details and to be on our mailing list for submissions, prize drops, leaderboard updates, tutorials, events, and more!
Prizes
We have around $10,000 in prizes:
The Dates
May 15, 2024
Challenge opens. You are able to access the challenge and the dataset. We recommend you review the GRT Transparency report for some context and guidance about the data you’re working with.
Jun 13, 2024
The grading and submission instructions are here. The submission portal is now live! You can check out the submission portal to see the leaderboards and submit your responses.
Jul 15, 2024
Competition Closes at 11:59 ET
Aug 2024
Winners announced
Stay in touch!
Sign up to stay up to date on upcoming challenges, events and to receive our newsletter.
FAQs
Bias bounties are not only a vital component of assessing AI models but are a hands-on way for people to critically engage with the potential negative impacts of AI. Bias bounties are similar to the more well known bug bounties within the cybersecurity community. But instead of finding vulnerabilities in the code, bias bounties seek to find socio-technical harms. These harms can include factuality issues, bias, misdirection, extremist content, and more. Bias bounties are narrowly scoped challenges, focused on a particular data set and problem.
Bias bounties complement red teaming exercises, which are broader in scope and primarily focused on breaking the safety guardrails of the AI model.
Each of our challenges will have a set start and end date, with the majority of challenges running for at least one month. Most often the datasets will be hosted on Humane Intelligence’s GitHub, unless the data contains sensitive information. All of our challenges involve cash prizes. The challenge overview will include how the prizes will be distributed amongst the winners of the different skill levels.
We are currently exploring future bounty challenges in these areas: hiring, healthcare, recommender systems, insurance, policing, policymaking, gendered disinformation, elections, disparate impacts, disability, counter-terrorism, and islamophobia.
Themes are selected with a variety of factors in mind, such as impacts on real world issues, access to data, and the needs of our partner organizations.
If your organization, agency, or company is interested in having your AI models assessed, please contact us. We can coordinate around building a public red teaming challenge, a bias bounty focused on a specific use case, or a private assessment done internally.
Our goal is to represent a globally diverse set of challenges, as these issues touch every corner of the world. If you’d like to promote our challenges for your community to participate in, feel free to contact us for promotion support.
Our bias bounty challenges run the gamut from the more technical that involve coding to the more accessible that involve generating prompts. Our technical challenges often include various skill level options so a wider range of people can join. Most often our challenges do not require any pre-requisite knowledge to participate.
You can compete in some of our challenges as a team, unless explicitly stated otherwise. You are responsible for organizing your team, dividing the work amongst yourselves, and, if applicable, dividing any winnings amongst yourselves (as only the submitting account/person will receive the prize money). We have dedicated channels on our Discord server for each bias bounty challenge and for finding a partner.
The scope of each challenge will be unique, so be sure to read through the specifics of each challenge to assess what skills are needed for each challenge level to determine your abilities.
We are eagerly seeking outreach opportunities with organizations, universities, and academic institutions around the world to ensure that we have a diverse range of participants. If you’d like to put us in touch with such a group, send us an email at hi@humane-intelligence.org
We understand that bias bounties are a new concept to many people, so we are actively creating a repository of resources for people to learn.
Discord Community
On our Discord server, there will be channels created for each of our bias bounty challenges for participants to ask questions. Additionally, there is a research channel where community members share the latest in red teaming and bias bounty tactics.
Landscape Analysis
We have an ever-evolving database featuring a landscape analysis of AI evaluations, which includes various organizations (academic, NGO, business, network, government, and others) and resources (papers, tools, events, reports, repositories, indexes, and databases). Users can also search for different AI evaluation categories, such as Benchmarking, Red-Teaming, Bias Bounty, Model Testing, and Field Testing.Tutorial Videos
For our first bias bounty challenge, one of our Humane Intelligence Fellows created a tutorial video series that walks complete beginners through the process of downloading datasets, creating a coding notebook, analyzing the data, and submitting challenge solutions. While the specifics of challenges will change, the general processes outlined in these videos will remain the same.
Challenge Submission Guidelines
Each of our bias bounty challenges will include an overview, suggestions on how to tackle the issue, and the criteria that will be used for grading. Most often your submission data will be incorporated into a coding notebook that contains code written by our data scientists to assist with the grading.
We understand that bias bounties are a new concept to many people, so we are actively creating a repository of resources for people to learn.
Discord Community
On our Discord server, there will be channels created for each of our bias bounty challenges for participants to ask questions. Additionally, there is a research channel where community members share the latest in red teaming and bias bounty tactics.
Landscape Analysis
We will soon have an ever-evolving database featuring a landscape analysis of AI evaluations, which includes various organizations (academic, NGO, business, network, government, and others) and resources (papers, tools, events, reports, repositories, indexes, and databases). Users can also search for different AI evaluation categories, such as Benchmarking, Red-Teaming, Bias Bounty, Model Testing, and Field Testing.Tutorial Videos
For our first bias bounty challenge, one of our Humane Intelligence Fellows created a tutorial video series that walks complete beginners through the process of downloading datasets, creating a coding notebook, analyzing the data, and submitting challenge solutions. While the specifics of challenges will change, the general processes outlined in these videos will remain the same.
Challenge Submission Guidelines
Each of our bias bounty challenges will include an overview, suggestions on how to tackle the issue, and the criteria that will be used for grading. Most often your submission data will be incorporated into a coding notebook that contains code written by our data scientists to assist with the grading.
Each bias bounty will have a specific grading criteria that will be released at the launch of the challenge, in addition to submission instructions. The grading criteria will often be different for each skill level. Submissions will be graded by the Humane Intelligence staff following this criteria.
To see examples of our previous grading criteria: Bias Bounty 1 and Bias Bounty 2 (Intermediate and Advanced).
How to Submit
Detailed instructions on how to submit your solution will be provided in the bias bounty challenge overview. You can only submit to one skill level per competition.
We aim to grow the community of practice of AI auditors and assessors; one way we strive to do so is through sharing what participants learned by completing challenges , as well as the broader insights learned about the particular issue area of the challenge. Participants are also encouraged to share their insights in our Discord community.
Additionally each of our challenges will include details about how these learnings will be used by us and our external partners to make AI more equitable and safe.
Yes.
Other Questions?
Find us on our Discord channel
