Subgroup Discrimination
Main Idea
This 90 minute learning scenario introduces students to the concept of subgroup discrimination in AI systems. Students explore how AI models, while seemingly fair at a broad level, can reinforce or amplify discrimination when applied to more specific intersections of identity (e.g., race + gender + income level). Through an interactive, inquiry-based approach, students analyze example datasets and simulated algorithmic outputs to develop awareness and critical thinking around AI fairness.
| Creator | Nang May Myat Moe |
| Subject |
Computer Science, Social Studies |
| Length | 90 minutes |
| Pedagogical Approach |
Inquiry-based learning Project-based learning Collaborative learning |
| Competences |
Understand the basic concepts of bias, discrimination, and subgroup discrimination in algorithmic systems. Recognize how AI systems can unintentionally harm specific marginalized subgroups despite appearing fair overall. Reflect on the social and ethical impacts of algorithmic decisions and propose inclusive strategies to mitigate harm. Competencies from the AIWARE framework: C70 – Biases C72 – Algorithmic bias C98 – Subgroup discrimination C66 – Ethical issues |
| Grades | 10-12th Grade |
| Technologies |
Padlet Teachable Machine Canva Kahoot |
| Evaluation |
Class Participation: Assess individual contributions to discussions.
Kahoot!
|
Learning Activities
Lesson plan:
Introduction with the title : When AI goes Unfair (15 minutes)
- Activity: Students are shown a real-world case: an AI passport checker that checks if the person in the photo is blinking or not.
- Show a short video clip of it and make a group dicussion.
- Group discussion: “Why do you think the system couldn’t detect the Asian face?”, “Would you call this fair or unfair?”, “What kind of harm could come from this?
2.Core Concepts (25 minutes)
Define and give examples of:
- Bias: Systematic preference in data
- Discrimination: Unfair outcomes from bias
- Subgroup Discrimination: Fails specific identity intersections (e.g., older Black women)
3.Hands-On Task: Train Your Own Classifier (25 minutes)
Experience how subgroup discrimination can emerge in AI models.
- Tool: Teachable Machine by Google
- Activity Steps:
- Create 3 categories: light, dark, and Asian faces
- Upload 8–10 curated images per class
- Train the model
- Test it using new images or webcam
- Create 3 categories: light, dark, and Asian faces
Observe: Are some groups misclassified more than others? Why?
4.What Does Fairness Mean? (15 minutes)
- Present a short thought experiment:
“Imagine this model is used in passport photo approval or job hiring. What could happen?” - Display a 2-minute clip or quote from Gender Shades or real news article on AI bias.
- Guide a group reflection using Padlet, Jamboard, or worksheet prompts.
5.Design for Fairness (10–15 minutes)
Encourage problem-solving and ethical thinking.
- Introducing a design challenge: How would you fix it?
- Complete Fairness Design Plan worksheet:
- Identify the unfair outcome
- Suggest a solution (e.g., more diverse training data)
- Add a creative sketch or policy rule
- Identify the unfair outcome
Worksheet: Designing for Fair AI
Title: My Plan to Make AI Fairer
Section A – What I Observed
- Which group was treated unfairly by the model?
→ ____________________________ - What kind of mistake did the AI make?
- ☐ Mislabeling
- ☐ Ignoring some faces
- ☐ Lower confidence scores
- ☐ Other: __________________
- ☐ Mislabeling
Section B – My Fairness Plan
3. What would you change in the system?
Write one idea to improve fairness:
______________________________________________________
Why do you think this would work?
______________________________________________________
6.Conclusion ( 10 minutes)
- Kahoot quiz: Bias, subgroup discrimination, fairness concepts
- Summarize key takeaways from the session.
- Assign optional homework: Read simplified COMPAS case and answer
COMPAS and AI in Criminal Justice.pdf


