TL;DR: What is a bias audit? A bias audit is an independent, impartial evaluation of whether a hiring tool selects or scores groups of candidates at different rates. Under NYC Local Law 144, the auditor calculates selection or scoring rates and impact ratios by sex, race/ethnicity and their intersections, and the employer publishes a summary. The law sets no threshold score.
General information, not legal advice. Have counsel confirm how this applies to you.
The term has two meanings. Under New York City law, a bias audit is a narrow, defined calculation; in our reading, vendors often use the phrase more broadly, to cover training data and model behaviour. The legal audit reports selection rates, scoring rates and impact ratios, and Local Law 144 requires no specific action based on them, and the 0.80 figure comes from the EEOC's four-fifths rule of thumb.
Updated 21 October 2026. First publication.
Key takeaways
- Under NYC Local Law 144, a bias audit is an impartial evaluation by an independent auditor that, at a minimum, calculates selection or scoring rates and impact ratios by sex, race/ethnicity and intersectional categories (DCWP FAQ II.1).
- In our reading, vendors use "bias audit" more broadly, covering training data and model behaviour; the legally defined NYC audit is narrower.
- The audit reports numbers. Local Law 144 requires no specific action based on the results of a bias audit (DCWP FAQ II.2).
- The auditor must be independent of the tool's use, development and distribution, and DCWP keeps no approved-auditor list (DCWP FAQ IV.1–IV.2).
- The 0.80 figure comes from the EEOC's four-fifths rule of thumb (29 CFR § 1607.4(D)), not from Local Law 144.
The legal definition comes from NYC Local Law 144, and it is narrower than the general one.
What does a bias audit measure?
In general use, a bias audit is a statistical review of whether a decision tool or process produces materially different outcomes for demographic groups. NYC's version is the narrow, legally defined one, and this page uses it.
The NYC audit rests on three calculations defined in 6 RCNY § 5-300 (DCWP Notice of Adoption, April 2023):
- Selection rate: individuals in the category moving forward or classified, divided by individuals in the category who applied or were considered. The rule's example: 100 Hispanic women apply, 40 are selected, and the rate is 40%.
- Scoring rate: the share of individuals in the category scoring above the sample median score.
- Impact ratio: a category's rate divided by the rate of the most-selected (or highest-scoring) category. The three-table walk-through is on how impact ratios are calculated.
Under § 5-301(b)–(c) (same notice), the audit reports these categories:
- Sex
- Race/ethnicity: Hispanic or Latino, White, Black or African American, Native Hawaiian or Pacific Islander, Asian, Native American or Alaska Native, and Two or More Races
- Sex and race/ethnicity combined (the intersectional categories)
The count of individuals in an "unknown" category is reported, not dropped.
How does a bias audit work in numbers?
The numbers below are illustrative, not real data. Three groups apply to the same tool, and the audit computes a selection rate and an impact ratio for each.
| Category | Applicants | Advanced | Selection rate | Impact ratio |
|---|---|---|---|---|
| Group A | 1,000 | 300 | 30.0% | 1.00 |
| Group B | 800 | 200 | 25.0% | 0.83 |
| Group C | 500 | 100 | 20.0% | 0.67 |
Illustrative, not real data.
Read it column by column:
- Applicants: individuals in each category who applied or were considered.
- Advanced: those in the category moving forward.
- Selection rate: Advanced divided by Applicants. Group A: 300 ÷ 1,000 = 30.0%.
- Impact ratio: each group's selection rate divided by the highest rate, which is Group A's 30.0%. Group B: 25.0 ÷ 30.0 = 0.83. Group C: 20.0 ÷ 30.0 = 0.67. Group A divides by itself, so its ratio is 1.00.
When the tool scores candidates instead of advancing them, the audit uses the scoring rate. A second illustrative example: ten candidates are scored, and the median score of the whole sample is the dividing line. Sorted, the scores are 91, 84, 80, 78, 72, 66, 64, 61, 58, 52. The median is (72 + 66) ÷ 2 = 69.
| Category | Scores | Above median (69) | Scoring rate | Impact ratio |
|---|---|---|---|---|
| Group A | 91, 84, 78, 66, 61 | 3 of 5 | 60.0% | 1.00 |
| Group B | 80, 72, 64, 58, 52 | 2 of 5 | 40.0% | 0.67 |
Illustrative, not real data.
Group B's ratio is 40.0 ÷ 60.0 = 0.67.
Group C's 0.67 sits below the EEOC's 0.80 rule of thumb. Local Law 144 does not tell the employer what to do about it; it requires the number to be calculated and published. What the employer does next is a question for its lawyer under federal, state and city anti-discrimination law.
What does a bias audit mean under NYC Local Law 144?
DCWP's FAQ (II.1) defines it as "an impartial evaluation by an independent auditor." The audit must be no more than one year old when the tool is used (§ 5-301(a), Notice of Adoption). The summary is posted on the employment section of the employer's site, or linked through an active hyperlink (FAQ II.3), and kept up for at least six months after the tool's last use (§ 5-303). Candidates who are NYC residents receive notice 10 business days before use (FAQ VI). The employer, not the vendor, is responsible, though a vendor may commission the audit (FAQ V.1–V.2).
On data, the audit uses the tool's historical data by default. Test data is allowed only if historical data is insufficient for a statistically significant audit, and inferred demographic data cannot be used (§ 5-302; FAQ III.5–III.6).
New York City is one of several jurisdictions with duties of this kind. The AI hiring laws by state tracker shows where audit or notice duties apply elsewhere.
Who can conduct a bias audit?
An independent auditor. Under § 5-300 (DCWP Notice of Adoption) and DCWP FAQ IV.1–IV.2, a bias auditor is not independent if the person:
- is or was involved in using, developing or distributing the tool;
- has an employment relationship with the employer or the vendor during the audit; or
- has a direct or material indirect financial interest in either.
DCWP keeps no list of approved auditors (FAQ IV.1–IV.2).
How much does a bias audit cost?
One provider publishes a list price: Bias Audit USA lists $5,500 for one tool, delivered in 5 business days from clean data (pricing page, fetched 15 October 2026). That is one provider's price, not the market. For other auditors, check each one's own pricing page or ask for a quote. The dated cost comparison sits on the page on how to conduct an AI bias audit.
Is there a passing score for a bias audit?
No. DCWP's FAQ (II.2) states: "The Law does not require any specific actions based on the results of a bias audit." Anti-discrimination laws still apply (DCWP FAQ).
The 0.80 figure comes from the EEOC's Uniform Guidelines. Under 29 CFR § 1607.4(D) (2024 edition), a rate less than four-fifths of the highest group's rate "will generally be regarded by the Federal enforcement agencies as evidence of adverse impact." The same provision states that smaller differences can still be adverse impact, and that larger ones may not be where numbers are small. It is a rule of thumb, not a threshold.
DCWP's statement of basis calls its calculations "consistent with Section 1607.4" (Notice of Adoption). The consistency is in the arithmetic, not in a mark that decides anything.
On the federal posture: Executive Order 14281 (23 April 2025), Sec. 4, directs agencies to deprioritize disparate-impact enforcement; it does not change the text of Title VII. As of 15 October 2026, the EEOC page titled "Select Issues: Assessing Adverse Impact in Software, Algorithms, and AI" did not load when we checked it. What either means for a specific employer is a question for counsel.
How is a bias audit of an AI hiring tool different from an audit of an AI model?
In our reading, vendors often define an AI bias audit more broadly, to include training data, model behaviour and outcomes. The NYC audit is outcome-based: it looks at selection and scoring rates, not training data or model internals, which makes it narrow.
Its required categories are sex and race/ethnicity only. Age and disability are not in the required tables, so other laws and other tests apply. The cases that show why bias audits exist are collected on a separate page.
Whether a scored AI interview platform falls inside the law is a scope question for counsel. DCWP's FAQ lists, as one of three cumulative conditions, that the tool "substantially assists or replaces discretionary decision-making" (DCWP FAQ).
What does a bias audit report look like?
The published summary lists the audit date; the data source and an explanation of it; the count in the unknown category; the number of applicants; the rates and impact ratios for all categories (DCWP FAQ II.3–II.4), and the justification for any excluded category (§ 5-301, § 5-303).
On exclusions, § 5-301(d) provides that an independent auditor "may exclude" a category under 2% of the data from impact-ratio calculations (Notice of Adoption). The summary must then give the justification and the excluded category's applicant count and rate. An open question: whether "2% of the data" is judged table by table, since a group can be over 2% in the race/ethnicity table and under 2% in the intersectional table. The rule text we read does not settle this. Ask the auditor and counsel.
A finished example is Pfizer's Bias Audit Summary of the HiredScore Grade System, published May 2026. It covers data from August 2022 to February 2025, reports impact ratios under three definitions (grade A; grade A or B; grade A, B or C), and excludes groups under 2%.
As a baseline, a June 2024 study of 391 employers found 18 with posted audit reports and 13 with posted notices (Wright et al., FAccT 2024).
FAQ
What is a bias audit in simple terms?
A statistical check of whether a hiring tool advances groups of candidates at materially different rates.
What is an impact ratio?
A category's rate divided by the rate of the most-selected or highest-scoring category. See the impact ratio and four-fifths rule page for the full calculation.
Who can do a bias audit?
An independent auditor who meets the three independence tests in § 5-300: no involvement in using, developing or distributing the tool; no employment relationship with the employer or vendor during the audit; no direct or material indirect financial interest in either. DCWP keeps no approved list (FAQ IV.1–IV.2).
How often is a bias audit required?
The audit must be no more than one year old when the tool is used (§ 5-301(a), Notice of Adoption).
How much does a bias audit cost?
One published list price is $5,500 for one tool (Bias Audit USA, fetched 15 October 2026); for other auditors, check their pricing pages or request a quote. See bias audit steps and cost for the dated comparison.
Is there a passing score for a bias audit?
No, not under Local Law 144, which requires the numbers to be calculated and published (DCWP FAQ II.1, II.3) but requires no specific action based on the results (DCWP FAQ II.2). The 0.80 figure is the EEOC's rule of thumb (29 CFR § 1607.4(D)).
Can a vendor's bias audit cover my use?
The employer remains responsible for the audit requirement, and a vendor may commission an audit (DCWP FAQ V.1–V.2). Whether a particular vendor audit can be relied on for your use depends on the data it used: the rules set the tool's historical data as the default (§ 5-302, FAQ III.5–III.6). Confirm with counsel.
Does an AI interview tool need a bias audit?
That is a scope question for counsel, and this page does not answer it for any product. One of the three cumulative conditions in DCWP's FAQ is that the tool "substantially assists or replaces discretionary decision-making."
See an evidence pack in a demo to see how ZeaHire documents the basis for each score. Whether a bias audit applies to your use is a question for your counsel.
Comments
No comments yet — start the conversation.