Why explainability is non-negotiable
Under the Digital Personal Data Protection Act, an automated decision that affects an individual must be explainable to that individual. That is a hard requirement for any candidate-scoring system, including ours. The ATS we ship cannot be a black box.
We chose six default scoring parameters (experience, technical depth, education, domain fit, career stability, resume quality) and held the line on letting customers add their own. Every score has a per-parameter breakdown the candidate can request, and every override by the AI is auditable in the candidate timeline.
The bias-detection harness
Bias never shows up in unit tests. It shows up in slow drift across cohorts. We built a CI job that takes a corpus of synthetic resumes (carefully labelled for gender-coded language, name-based proxies, and education tier) and runs scoring twice, swapping a single attribute. If the score moves more than the configured tolerance, the build fails.
It is not perfect. But it caught two regressions in the last six months that we would not have spotted from production telemetry alone.
What the recruiter sees
Recruiters see the score, the per-parameter breakdown, and a one-line summary of why the AI ranked the candidate where it did. They can override the AI in one click, and the override is logged with the recruiter's user ID and a free-text reason field.
Candidates can request the same breakdown via the privacy portal. We respond within seven working days, well inside the DPDP timeline.
What we will not ship
We have explicitly chosen not to score on inferred attributes (gender from name, age from graduation year, religion from name patterns). We do not buy or pull data from third-party enrichment vendors. The model sees only what the candidate submits and what the recruiter explicitly inputs. If the customer wants a parameter we do not support, they can add it as a custom criterion (covered in this companion post) and own the bias surface themselves.