Agent workspace · Fairness test
Who gets shortlisted, by group
The vendor's scoring rules are applied to 1,000 synthetic applicants and the top half are shortlisted. Selection rates are then compared across groups. A group selected at less than 0.80 times the rate of the best group is flagged. That line is the US "four-fifths" rule of thumb, used here as a reference standard; India has no statutory equivalent.
Shortlisted
500 / 1000
Shortlist line
63.3
Groups below 0.80
College tier, Language of posts, Religion (synthetic label), Social media presence
Gender
Lowest ratio 0.88- Man (n=580)53% · 1.00
- Woman (n=420)46% · 0.88
College tier
Lowest ratio 0.54- Tier 1 (n=141)72% · 1.00
- Tier 2 (n=410)55% · 0.77
- Tier 3 (n=449)39% · 0.54
Language of posts
Lowest ratio 0.59- English (n=371)65% · 1.00
- Hindi / Hinglish (n=240)47% · 0.72
- Other Indian language (n=389)38% · 0.59
Home region
Lowest ratio 0.92- East (n=210)50% · 0.95
- North (n=252)52% · 1.00
- Northeast (n=46)52% · 1.00
- South (n=345)49% · 0.93
- West (n=147)48% · 0.92
Religion (synthetic label)
Lowest ratio 0.77- Majority (n=816)52% · 1.00
- Minority (n=184)40% · 0.77
Social media presence
Lowest ratio 0.35- Active (n=585)51% · 0.93
- Minimal (n=136)54% · 0.99
- None (n=72)19% · 0.35
- Private (n=207)55% · 1.00
Bars show the share of each group shortlisted. The ratio divides a group’s rate by the best group’s rate on the same attribute. Groups under 15 people are shown but not judged.
A pass is not proof of fairness
Under the vendor configuration, gender’s lowest ratio is 0.88, above the line. Yet signal S11 penalises women’s engagement posts directly. Women’s private accounts hide some negative signals, which offsets that penalty on average, so the group test passes while individual women, like Priya in the sample, are marked down for their marital plans. A group-level test can miss direct discrimination.
What this test cannot show
Whether the tool predicts who joins or stays. That needs real outcome data, and the vendor has provided no validation study. The pool is synthetic and the result depends on the assumptions below. It shows what the scoring rules do to a realistic mix of applicants, not what happened to real people.
Modelling assumptions
- The aptitude test is equally distributed across every group. This is built in, so any gap comes from the vendor's signals, not from chance. Kestrel's aptitude test itself was not audited.
- Women more often keep accounts private, so less of their content is visible and the no-social-media penalty reaches them more often.
- The language people post in follows their home region; English posting is more common at tier-1 colleges.
- Network overlap with employees, follower reach and professional portfolios are higher at tier-1 colleges.
- The vendor's culture-fit model scores minority-faith festival content as dissimilar, and its attrition model applies the engagement-post penalty to women.
The pool is generated from a fixed seed, so it is identical in the browser, on the server and in the automated tests.