Algorithm audit

Assignment Overview

For this assignment, youll audit a classification system that sorts people into outcomes. As you work, draw on ideas from our algorithms unit, especially how institutions translate messy lives into categories, how objectivity gets built into forms and metrics, and how ideal types quietly set the standard for what counts as normal.

Your submission will consist of several components which include writing (between 1800 and 2400 words total), evidence, and a step map. Your project should meaningfully integrate at least concept or reading from class, moving beyond mere summary to demonstrate your ability to apply ideas and arguments.

Getting Started

Choose a system you can document with evidence. These are a few examples you might consider, but we invite you to come up with your own:

• financial aid verification
• GPA policy, major eligibility, probation rules
• credit scoring, background checks, housing applications
• clinic intake forms or patient portal workflows
• visa/immigration forms (or university international travel paperwork)
• job application portals / automated screening
• social media feed ranking or content moderation
• tracking apps

Note: Your system must (a) collect inputs, (b) classify people, and (c) produce outputs/consequences. We will make time to work on generating ideas in class, but you are always welcome to reach out or set up a meeting to discuss potential choices.

Assignment Components

1) System snapshot (300 words)

Briefly describe:

• What the system is and where it lives (institution/platform).
• Who it sorts, what it sorts for, and what problem it claims to solve. What is it actually good at (speed, consistency, fraud control, resource management, safety, convenience)?
• What outcomes it produces (approved/denied, flagged/unflagged, boosted/suppressed, eligible/ineligible, etc.).

2) Evidence (minimum 3 items)

Provide evidence that demonstrates how your system works. Use any mix:

• screenshots (you dont have to include sensitive information) of interfaces, or photographs of physical sites connected to your system
• excerpts from policy language, self-descriptions
• observation notes (what the interface prompts, how people move through it)
• a blank form/template or sample prompts

Add a 12 sentence caption for each item explaining what it shows.

3) Step map (one diagram + brief explanation)

A flowchart or numbered sequence that makes the systems logic legible:

1. inputs the system requests
2. decision points (if/then)
3. categories it produces
4. outputs and consequences
5. where humans intervene, if at all
6. where the system forces ambiguity into a checkbox

Write a brief, ca. 150-word description of your map.

4) The ideal type (400600 words)

Describe the ideal user the system assumes:

• traits it rewards (stability, documentation, time, money, literacy, language, predictable life path)
• what it treats as risk, deviation, or noise
• what it assumes about bodies, family forms, work schedules, housing, tech access, citizenship, etc.

5) Misread / exclusion analysis (500800 words)

Identify two groups the system is likely to misread or exclude (overlap is fine). For each:

• what gets misunderstood (identity, situation, intent, need)
• the consequences (delay, denial, extra burden, stigma, invisibility)
• what the system treats as objective when it isnt

6) Reflection (400600 words)

Write a brief reflection on what this system does to people. Use any of these prompts; you dont need to cover all of them.

• Who benefits from the system working as designed, and who absorbs the costs (time, documentation, delays, denials, stigma, invisibility)?
• Where do you locate yourself in relation to the systems assumed user (close to the ideal, legible-but-friction-filled, treated as risk, forced into mismatched categories)?
• Is meaningful bias reduction possible here? What would less biased look like in practice, or what makes bias structurally hard to remove?
• What inequities would likely remain even after reform?
• What does the system prioritize (speed, standardization, compliance, risk control, resource allocation), and what does it sacrifice (context, nuance, access, privacy, dignity)?