Subgroup Discrimination

Main Idea

This 90 minute learning scenario introduces students to the concept of subgroup discrimination in AI systems. Students explore how AI models, while seemingly fair at a broad level, can reinforce or amplify discrimination when applied to more specific intersections of identity (e.g., race + gender + income level). Through an interactive, inquiry-based approach, students analyze example datasets and simulated algorithmic outputs to develop awareness and critical thinking around AI fairness.

Creator Nang May Myat Moe
Subject

Computer Science, Social Studies

Length 90 minutes
Pedagogical Approach

Inquiry-based learning

Project-based learning

Collaborative learning

Competences

Understand the basic concepts of bias, discrimination, and subgroup discrimination in algorithmic systems.

Recognize how AI systems can unintentionally harm specific marginalized subgroups despite appearing fair overall.

Reflect on the social and ethical impacts of algorithmic decisions and propose inclusive strategies to mitigate harm.

Competencies from the AIWARE framework:

C70 – Biases

C72 – Algorithmic bias

C98 – Subgroup discrimination

C66 – Ethical issues

Grades 10-12th Grade
Technologies

Padlet

Teachable Machine

Canva

Kahoot

Evaluation

Class Participation: Assess individual contributions to discussions.

  • Teacher uses guiding questions during the initial case study discussion to evaluate students’ understanding.

  • Assessment criteria: clarity of reasoning, ability to connect fairness to social consequences, engagement in group dialogue.

Kahoot! 

  • Students’ score, common misconceptions
   

Learning Activities

Lesson plan:

Introduction with the title : When AI goes Unfair (15 minutes)

  • Activity: Students are shown a real-world case: an AI passport checker that checks if the person in the photo is blinking or not. 
  • Show a short video clip of it and make a group dicussion. 
  • Group discussion: “Why do you think the system couldn’t detect the Asian face?”, “Would you call this fair or unfair?”, “What kind of harm could come from this?

2.Core Concepts (25 minutes)

      Define and give examples of:

  • Bias: Systematic preference in data
  • Discrimination: Unfair outcomes from bias
  • Subgroup Discrimination: Fails specific identity intersections (e.g., older Black women)

3.Hands-On Task: Train Your Own Classifier (25 minutes)

Experience how subgroup discrimination can emerge in AI models.

  • Tool: Teachable Machine by Google

  • Activity Steps:

    • Create 3 categories: light, dark, and Asian faces

    • Upload 8–10 curated images per class

    • Train the model

    • Test it using new images or webcam

 Observe: Are some groups misclassified more than others? Why?

4.What Does Fairness Mean? (15 minutes)

  1. Present a short thought experiment:
    “Imagine this model is used in passport photo approval or job hiring. What could happen?”

  2. Display a 2-minute clip or quote from Gender Shades or real news article on AI bias.

  3. Guide a group reflection using Padlet, Jamboard, or worksheet prompts.

5.Design for Fairness (10–15 minutes)

Encourage problem-solving and ethical thinking.

  • Introducing a design challenge: How would you fix it?
  • Complete Fairness Design Plan worksheet:

    • Identify the unfair outcome

    • Suggest a solution (e.g., more diverse training data)

    • Add a creative sketch or policy rule

Worksheet: Designing for Fair AI

Title: My Plan to Make AI Fairer

Section A – What I Observed

  1. Which group was treated unfairly by the model?
    → ____________________________

  2. What kind of mistake did the AI make?

    • ☐ Mislabeling

    • ☐ Ignoring some faces

    • ☐ Lower confidence scores

    • ☐ Other: __________________

Section B – My Fairness Plan

3. What would you change in the system?

Write one idea to improve fairness:
______________________________________________________

Why do you think this would work?
______________________________________________________

6.Conclusion ( 10 minutes)

  • Kahoot quiz: Bias, subgroup discrimination, fairness concepts

  • Summarize key takeaways from the session. 
  • Assign optional homework: Read simplified COMPAS case and answer

 

COMPAS and AI in Criminal Justice.pdf

https://www.canva.com/design/DAGrSKPYIdo/vNVg9hJ_PK_5HuzETrglog/view?utm_content=DAGrSKPYIdo&utm_campaign=designshare&utm_medium=link2&utm_source=uniquelinks&utlId=h1f2bdb5cd5