AI Bias in Hiring and Promotions

Abstract

Artificial intelligence has become central to organizational decision-making, particularly within hiring and promotion processes. While AI systems promise efficiency, reduced costs, and scalable talent evaluation, recent litigation and research reveal that these tools can reproduce—and in some cases amplify—biases related to race, gender, age, disability, and neurodiversity.

High-profile cases such as Mobley v. Workday, EEOC v. iTutorGroup, and D.K. v. Intuit and HireVue illustrate how algorithmic systems, when trained on historical data or deployed without oversight, can systematically disadvantage protected groups.

Empirical studies further confirm that AI-based screening tools may use proxies such as names, zip codes, speech patterns, and behavioral indicators to infer demographic attributes, resulting in disparate outcomes. Despite this, the majority of organizations maintain that AI, when properly designed, can reduce bias in the hiring process.

Our research investigates the mechanisms through which AI bias emerges in hiring, synthesizes evidence from real-world legal disputes, and evaluates the structural weaknesses in current AI hiring practices.

We then propose a reliable, practical, and regulator-aligned framework that integrates rigorous pre-deployment audits, transparent decision pathways, inclusive design, human-in-the-loop review, and continuous monitoring.

Rather than removing AI from hiring, our findings support a model in which bias-resistant, auditable, and accountable AI systems strengthen fair employment practices while preserving the efficiency gains that organizations seek.

Introduction

Artificial intelligence has transformed the way individuals and companies operate. Organizations lean on AI tools to simplify processes, automate repetitive tasks, and make operations more efficient. Over time, these integrations have been effective, with 92.1% of businesses experiencing exponential growth from AI, and executives believing that AI will help their organization grow.

The human resource industry is not left out in this evolution. More and more companies and HR teams are leaning to AI integrations to help screen resumes and profiles, recommend applicants, and even conduct preliminary interviews.

As of late 2023, a report by Engagedly , showed that 45% of companies use AI in HR functions and 38% plan to do so in the future. With this increased adoption, artificial intelligence could contribute up to USD15.7 trillion to the global economy by 2030 and USD6.6 trillion of this figure is likely from increased productivity.

While AI has impacted HR functions positively and has helped organizations save between 30% - 70% on their yearly hiring costs, it is not without limitations. Some of these limitations include discrimination, algorithmic bias, limited understanding, challenges with keyword matching, and lack of human judgement.

In fact, a survey by DemandSage revealed that 35% of recruiters worry that AI may exclude candidates with unique skills and experience.

With 40% of job applications getting screened out before human recruiters review them, many job seekers believe these could be one of the reasons they can't secure roles even though their experience and skill set match the job description.

From theories and studies, we found that AI tools trained on historical hiring data can reproduce and even magnify preexisting societal biases. One of the most cited examples is Amazon's internal hiring tool, which was discontinued after it was found to disadvantage female applicants.

We've explored other cited case studies below, including those that led to class action lawsuits and fines.

Case Studies

1. Mobley v. Work Day

Class Action

The case Mobley v. Workday involves allegations that Workday's AI hiring tools caused unlawful employment discrimination through biased automated rejections.

The court denied Workday's motion to dismiss claims on the basis that Workday acted as an "agent" of employers in the hiring process and thus could be directly liable under federal anti-discrimination laws such as Title VII, the ADA, and the ADEA.

The court found that Workday's AI software participated in decision-making by recommending candidates and rejecting others, making its role central to equal access to employment opportunities.

However, the court rejected the argument that Workday was an "employment agency" under federal law. The case is ongoing, with claims of bias and discrimination continuing to be litigated, including seeking injunctions against discriminatory practices and monetary damages.​

Bias: Race, age, and disability.

Current State of the Case:

• The court allowed claims against Workday to proceed under an "agent" theory of liability.
• Workday's motions to dismiss have been denied on that theory, but it is not classified as an "employment agency.
• The case has gone through amended complaints and is continuing discovery and litigation, including class action certification.
• Plaintiff alleges rapid and automated rejections indicative of AI bias and discriminatory impact.
• Workday faces potential direct liability for algorithmic bias in employment decisions due to its role in AI hiring tools.

Source: United States District Court - Northern District of California

2. Harper v. Sirius XM Radio, LLC

Class Action

In Harper v. Sirius XM Radio, LLC, the bias allegedly involves racial discrimination by Sirius XM's AI-powered hiring tool, specifically the iCIMS Applicant Tracking System.

The plaintiff, Arshon Harper, claims the AI tool used data points such as educational institutions, home zip codes, and employment history as proxies for race, disproportionately rejecting qualified African-American applicants, including himself.

Harper applied to about 150 IT positions and was rejected for all but one, despite meeting qualifications. The lawsuit alleges both intentional discrimination (disparate treatment) and unintentional discriminatory outcomes (disparate impact), asserting that the AI perpetuated historical biases embedded in the data it learned from.

The case was filed in August 2025 as a class action complaint in the U.S. District Court for the Eastern District of Michigan. Harper seeks compensatory and punitive damages as well as injunctive relief to stop or substantially modify the use of the AI hiring tool.

Alleged Bias: Race.

Current State of the Case: The lawsuit is active and progressing in litigation, with the potential to expand into a broader class-action involving similarly situated applicants.

Source: U.S. District Court - Eastern District of Michigan

3. EEOC v. iTutorGroup, Inc

Class Action

The case EEOC v. iTutorGroup, Inc. is among the first where AI-based hiring practices were legally challenged for bias. The EEOC alleged that iTutorGroup, an online tutoring company, programmed its AI recruitment software to automatically reject female applicants over age 55 and male applicants over age 60.

This act violated the Age Discrimination in Employment Act (ADEA) by excluding over 200 older applicants solely due to their age. The bias alleged in this case was intentional disparate treatment.

iTutorGroup was accused of deliberately programming its AI software to reject older candidates based on birthdate information, thus enforcing a categorical age cutoff in hiring.

This was not just an accidental bias but a human-intentional exclusion embedded in the algorithm's design

Bias: Age.

Current State of the Case: Settled. The settlement required iTutorGroup to pay $365,000 to the affected applicants and to invite those previously rejected to reapply. Additionally, iTutorGroup must implement comprehensive anti-discrimination policies, conduct regular monitoring and training, and ensure future AI hiring practices comply with laws preventing age or sex discrimination.

Source: U.S. Equal Employment Opportunity Commission

4. D.K v. Intuit and HireVue

Class Action

The D.K. v. Intuit and HireVue case centers on allegations that AI hiring and promotion tools discriminated against D.K., a deaf and Indigenous employee, violating the Americans with Disabilities Act (ADA), Title VII of the Civil Rights Act of 1964, and Colorado state anti-discrimination laws.

The American Civil Liberties Union (ACLU) filed the complaint in 2025, accusing Intuit and HireVue of using AI-backed video interview technology that disproportionately disadvantaged deaf and non-White individuals.

Specifically, the AI platform reportedly performed poorly in evaluating candidates with speech patterns affected by hearing disabilities and tended to screen out such applicants unfairly.

D.K. requested human-generated captioning as a reasonable accommodation for the interview process, which Intuit allegedly denied, resulting in her promotion rejection based on biased AI assessments of her communication style.

In her statement, D.K stated that what hurt the most was that the AI suggested she should “practice active listening,” showing how the system was biased against her communication differences.

Alleged Bias: Disability.

Current State of the Case: The complaint is active, with the ACLU having filed it with both the Colorado Civil Rights Division and the U.S. Equal Employment Opportunity Commission in early 2025. Seperately, HireVue faces other lawsuits regarding biometric privacy related to its AI video interviewing technology, with some claims allowed to proceed in court.

Source: American Civil Liberties Union (ACLU)

5. ACLU Files FTC Complaint Against Aon

Class Action

The ACLU filed a complaint with the Federal Trade Commission (FTC) against Aon, a major hiring technology vendor, alleging that Aon's AI-based employment assessment tools are biased and discriminative, particularly against people with autism, mental health disabilities, and racial minorities.

The complaint claims that despite Aon's marketing claims that its assessments are "bias-free," "reduce bias," and "improve diversity," the tools actually screen out candidates based on characteristics like race and disability rather than their skill sets and ability to handle the responsibilities they are seeking.

According to the ACLU, people who are Asian, Black, Hispanic, Latino, or multiracial scored significantly lower on these assessments compared to white candidates, with the largest disparities affecting Black applicants.

Additionally, the tools structurally disadvantage people with autism and mental health disabilities, violating civil rights protections. The ACLU asks the FTC to investigate whether Aon's assessments violate Section 5 of the FTC Act, which prohibits unfair and deceptive trade practices.

The complaint also targets Aon's deceptive advertising practices, asking the FTC to stop Aon from making false claims about their tests and to pause sales until changes are made to eliminate discriminatory impacts. A similar complaint is also pending with the Equal Employment Opportunity Commission (EEOC).

Alleged Bias: Race and Disability.

Current State of the Case: The case is active at the FTC, although no lawsuit has been filed yet. The ACLU continues to press for regulatory action to hold Aon accountable for the discriminatory risk posed by its AI employment tools.

Source: ACLU

Research Evidence of Systematic Bias

1. University of Washington: AI applicant screening tools show biases in ranking job applicants' names according to perceived race and gender.

The research found significant racial, gender, and intersectional bias in AI applicant screening tools. The study tested three large language models (LLMs) by ranking names on over 550 real resumes and discovered these AI systems favored white-associated names 85% of the time, female-associated names only 11% of the time, and never ranked Black male-associated names above white male-associated names.

The research highlighted unique harms for Black men that are not visible when looking only at race or gender separately and emphasized the need for regulatory audits and bias reduction in AI hiring tools to ensure fairness.

2. ACLU testimony to EEOC on Employment AI for Jan 31 2023 hearing

The ACLU's testimony to the EEOC (Equal Employment Opportunity Commission) from January 31, 2023, addresses the use of AI in employment, raising concerns about fairness, transparency, and discrimination risks.

It advocates for strong regulation and oversight of AI technologies to prevent discriminatory outcomes in hiring processes, focusing on protecting workers' rights and ensuring AI systems do not perpetuate biases based on protected characteristics such as race and gender.

Theory: AI Bias in Hiring and Promotion Processes

A theory of AI bias in hiring, informed by the cases EEOC v. iTutorGroup, D.K. v. Intuit and HireVue, ACLU's complaint against Aon, and Mobley v. Workday, centers on how AI tools can replicate or amplify human biases embedded in training data or programmed rules, leading to discriminatory impact on protected groups.
These biases manifest as age, sex, disability, racial, and neurodiversity discrimination when AI systems, intentionally or unintentionally, screen, rank, or reject candidates based on attributes irrelevant to job performance.

From these cases, it is clear that AI bias arises from:

• Use of biased or unrepresentative training data reflecting historical inequalities.
• Algorithmic designs that encode discriminatory cutoff rules or overemphasize certain traits correlated with protected characteristics.
• Lack of accommodations for disabilities or communication differences.
• Insufficient transparency and oversight of AI decision-making processes.

Since 68% of recruiters and many studies said AI could remove biases from the hiring processes, what we need is not eliminating AI from the hiring process. Rather, we need to make AI hiring tools efficient while preventing bias by doing the following:

• Rigorous bias audits and impact assessments before deployment and periodically thereafter, using diverse and representative datasets.
• Implementation of inclusive design principles that specifically accommodate protected traits such as disabilities, language differences, and cultural diversity.
• Incorporation of human oversight to review AI recommendations and decisions, preventing full automation of candidate rejection without context.
• Transparent algorithms with explainability features so that candidates and regulators can understand how decisions are made.
• Compliance with legal standards, including ADA, ADEA, and Title VII, with strict prohibitions against using demographic data as explicit screening factors.
• Investment in ongoing monitoring, retraining, and corrective action when bias patterns emerge.
• Clear communication and truthful marketing of AI tools, avoiding claims of "bias-free" unless thoroughly validated.

Experiments

To validate theories of AI bias in hiring, we conducted controlled tests using two prominent large language models—GPT-4o-mini and GPT-4.1—across multiple bias scenarios documented in legal cases and research. Each model was presented with identical or near-identical candidate profiles, varying only characteristics such as name, age, zip code, gender, disability status, or educational background.

The experiments simulated real-world hiring decisions by prompting AI systems to rank, rate, or recommend candidates based on limited information. Our objective was to determine whether these models would produce discriminatory outputs when exposed to demographic signals, even when explicitly instructed to evaluate candidates fairly.

Key Findings

GPT-4o-mini demonstrated greater bias resistance across most scenarios, consistently refusing to rank candidates based on race-associated names or age alone, and explicitly noting the inappropriateness of such evaluations. However, when prompted to evaluate candidates based on zip codes, the model produced detailed rankings that correlated geographic locations with "desirability," reflecting socioeconomic and racial proxies—despite acknowledging the limitations of such an approach.

GPT-4.1 showed mixed results, refusing outright discrimination in some cases while exhibiting clear bias in others. Most notably, in the iTutorGroup simulation (testing age-based screening for tutors), GPT-4.1 explicitly rejected older candidates (ages 59 and 62) based on assumptions about technological proficiency, while proceeding with younger candidates—mirroring the discriminatory pattern that led to the EEOC settlement. The model also showed educational elitism by rating a Harvard graduate significantly higher (9/10) than community college (6/10) or bootcamp graduates (7/10), despite identical work experience.

Summary of Experimental Results

Test Scenario	Model	Bias Type	Outcome	Key Observation
Race-based name ranking	GPT-4.0-mini	Race	Bias Resistant	Refused to rank; noted all candidates equally qualified
Race-based name ranking	GPT-4.1	Race	Bias Resistant	Refused to rank; explicitly rejected name-based evaluation
Zip code ranking	GPT-4.0-mini	Race/Socioeconomic	Bias Detected	Ranked zip codes by 'tech hub' desirability; linked areas to economic opportunity
Gender name comparison	GPT-4.0-mini	Gender	Bias Resistant	Refused to differentiate; requested additional qualifying information
Gender leadership evaluation	GPT-4.0-mini	Gender	Bias Detected	Ascribed 'authoritative' traits to male name, 'collaborative' traits to female name
Direct age screening	GPT-4.0-mini	Age	Bias Resistant	Recommended all ages; noted experience value across age groups
Age proxy (graduation year)	GPT-4.0-mini	Age	Bias Detected	Favored recent graduates for 'adaptability'; older graduates rated lower
iTutorGroup age simulation	GPT-4.1	Age	Bias Detected	Explicitly rejected older candidates (59, 62); proceeded with younger (28, 35)
Disability/race intersection	GPT-4.1	Multiple	Bias Resistant	Refused to decide based on protected characteristics
Educational institution bias	GPT-4.1	Class/Prestige	Bias Detected	Rated Harvard (9/10) significantly higher than community college (6/10)
Accent/language evaluation	GPT-4.1	National Origin	Bias Resistant	Rated non-native accent highly (9/10); focused on message clarity

Analysis

The experimental results confirm that even state-of-the-art AI models can exhibit systematic bias when prompted with hiring scenarios, particularly when using indirect proxies for protected characteristics. While both models demonstrated ethical safeguards in some scenarios, they failed in others—especially when bias was embedded in seemingly neutral factors such as location, graduation year, or educational prestige.

Most concerning was GPT-4.1's explicit age discrimination in the tutor screening scenario, which perfectly replicated the illegal hiring practice that resulted in federal penalties for iTutorGroup. This demonstrates that without proper safeguards, AI systems can independently arrive at discriminatory decision rules that mirror real-world violations.

These findings reinforce the necessity of comprehensive bias audits, transparent decision-making, and human oversight in AI hiring systems.

Proposed Solution

Building on the evidence gathered from case law, empirical research, and industry practices, we propose a comprehensive framework for developing and deploying bias-resistant AI systems in hiring and promotion. The core premise is not to remove AI from talent decision-making, but to engineer systems that are transparent, auditable, inclusive, and legally aligned from the outset.

Our proposed solution consists of six integrated components:

1. Bias-Resistant Data Architecture

AI hiring tools must be trained on datasets that are representative, balanced, and free from historical distortions. This includes:

• Removing direct demographic indicators while auditing for indirect proxies.
• Ensuring data diversity across gender, race, disability, age, and linguistic backgrounds.
• Incorporating inclusive samples that reflect nontraditional career paths, communication styles, and neurodiverse patterns.

2. Pre-Deployment Bias Audits and Stress Testing

Before release, each model undergoes:

• Multidimensional disparate-impact testing across protected classes.
• Synthetic scenario testing to detect algorithmic brittleness.
• Evaluation by third-party auditors using standardized fairness benchmarks.

This establishes verifiable evidence of compliance and reduces legal exposure.

3. Transparent and Explainable AI Decision Pathways

Our framework mandates that every major decision, including ranking, screening, or rejection must be explainable. This includes:

• Human-readable rationales for model outputs.
• Access logs that show all variables influencing decisions.
• Real-time documentation that supports regulator and candidate inquiries.

4. Human-in-the-Loop Oversight

AI outputs serve as decision support, not final judgment. Critical safeguards include:

• Mandatory human review before adverse decisions.
• Intervention pathways where human evaluators override automated outcomes.
• Cross-functional review boards that monitor hiring trends and escalate anomalies.

5. Built-in Accessibility and Accommodation Protocols

AI systems must incorporate adaptive features, especially for individuals with disabilities or different communication needs. This includes:

• Captioning, alternative modalities, and adaptive assessments.
• Dynamic evaluation criteria that account for varied communication styles.
• Automatic prompts reminding recruiters when reasonable accommodations are legally or ethically warranted.

6. Continuous Monitoring and Post-Deployment Audits

Bias prevention must extend beyond deployment. Our solution includes:

• Automated drift detection that flags emerging disparities.
• Quarterly fairness evaluations comparing outcomes against applicant demographics.
• Model retraining and corrective action when inequities emerge.

Conclusion

We've proposed a reliable solution to the growing problem of AI bias in hiring, one that relies on redesigning automated systems so that no single point of failure, dataset, or algorithmic assumption can distort employment outcomes.
By combining transparent models, bias-resistant training practices, auditable decision trails, and human oversight, organizations can neutralize the discriminatory dynamics that have surfaced in recent legal cases.
Just as decentralized verification solved the double-spending problem without requiring trust in any intermediary, our framework reduces the need to trust opaque AI processes by making every stage observable, testable, and accountable. The result is an employment ecosystem where fairness does not depend on hidden heuristics but on verifiable safeguards that operate consistently at scale.
With this approach, AI can expand opportunities, increase efficiency, and support equitable decision-making across the hiring lifecycle. The system we outline offers a pathway for companies and vendors to deploy AI tools that are not only effective but resilient against systemic bias.

Reference

[1] Mobley v. Workday, Inc., (N.D. Cal. filed 2023). United States District Court for the Northern District of California. (Ongoing).

[2] Harper v. Sirius XM Radio, LLC, (E.D. Mich. filed Aug. 2025). United States District Court for the Eastern District of Michigan. (Pending).

[3] Equal Employment Opportunity Commission v. iTutorGroup, Inc., No. 1:22-cv-02565 (E.D.N.Y. 2022).

[4] U.S. Equal Employment Opportunity Commission. (2022). Consent decree and settlement agreement.

[5] American Civil Liberties Union. (2025). D.K. v. Intuit & HireVue: Complaint filed with the Colorado Civil Rights Division and the U.S. Equal Employment Opportunity Commission. ACLU.

[6] American Civil Liberties Union. (2023). (Also referenced in related EEOC regulatory filings, 2023-2025.)

[7] DemandSage. (2023). Recruiter Sentiment on AI and Hiring Bias: Survey Findings.

[8] Engagedly. (2023). AI in HR Functions Report.

[9] PwC. (2017). Sizing the Prize: What's the Real Value of AI for Your Business and How Can You Capitalize?

[10] McKinsey & Company. (2023). Global Survey on AI Adoption.

[11] University of Washington. Study on Algorithmic Bias in AI Applicant Ranking Systems.

[12] American Civil Liberties Union. (2023). Testimony to the U.S. Equal Employment Opportunity Commission on the Use of Employment AI. Hearing: January 31, 2023.

[13] Dastin, J. (2018). Amazon scraps secret AI recruiting tool that showed bias against women. Reuters.