We present an intersectional framework for impact assessment, designed to be useful and adaptable to other contexts as well. Impact assessment is a systematic process that identifies, analyzes, and anticipates the potential effects of an action — such as the design, use, or governance of an AI system — on people and the environment.
Intersectional Framework
The proposed intersectional framework goes beyond a simple technical risk review. It emphasizes how structural inequalities — related to gender, race, class, ability, or origin, among others — can be reproduced or amplified in the creation and implementation of AI technologies.
The goal is to turn impact assessment into a transformative practice that questions and modifies these dynamics, promoting fairer and more equitable outcomes.
The Audit Tools are designed to assess intersectional biases in vision and language foundation models.
Their goal is to make AI system auditing accessible to individuals and organizations that are usually excluded from these technical processes, with a particular focus on those most affected by AI's impacts.
The tools are being developed in collaboration with diverse participants, ensuring they respond to real experiences and needs.
Methodology
Their methodology combines technical analysis with participatory and intersectional approaches, making it possible to identify how biases affect people differently depending on gender, background, age, or other factors.
The result is a set of tools that not only detect bias, but also distribute the capacity to audit it.
What Makes Them Different?
- Co-designed with diverse participants, not only technical experts
- Include an intersectional analysis to capture multiple and overlapping biases
- Accessible to non-technical users
- Focused on real-world impacts on vulnerable communities
The workshops are co-creation spaces where directly affected communities and civil society organizations actively participate in designing the project's technical tools. Their purpose is to ensure that both the participatory auditing tool and the algorithmic justice benchmark are grounded in real experiences, rather than solely in technical criteria.
Through a participatory and intersectional methodology, the workshops help identify how discrimination in vision and language models affects people differently depending on their gender, background, age, or situation of vulnerability.
At the same time, they contribute to raising awareness of situated risks and strengthening communities' capacity to defend themselves.
- Designed with and for affected communities
- Incorporate intersectionality as a methodological core, not as an add-on
- Directly connect civic participation with technical development
- Generate collective knowledge and critical capacity in relation to AI
Measuring fairness, not just accuracy
The Benchmark is a technical tool designed to assess intersectional biases in vision and language foundation models. Unlike conventional benchmarks, which only measure technical performance, this one focuses on algorithmic justice: how models affect diverse people depending on gender, background, age, or other combined identity dimensions.
Its main indicators make it possible to compare model behavior in terms of fairness and bias mitigation, and to evaluate the social impact of training datasets.
The benchmark is built on the outcomes of participatory workshops, ensuring that its categories and evaluation criteria reflect the real experiences of affected communities.
Its goal is to become a reference standard for AI researchers and developers who want to incorporate justice as a core criterion, rather than an add-on.
- Assesses intersectional fairness, not just technical accuracy
- Built from data generated through participatory workshops
- Directly connected to the Audit Tool for a comprehensive evaluation
- Aimed at becoming an industry standard