AI outperforms human recruiters
A 70,000-applicant experiment shows AI-led interviews boost hiring outcomes, retention, and candidate satisfaction
Welcome to FullStack HR, and an extra welcome to the 12 people who have signed up since last edition.
If you haven’t yet subscribed, join the 9700+ smart, curious and like-minded future of work people by subscribing here:
Happy Tuesday!
If last week was my first week back, this feels like my first week back.
Chicago felt like something else, but this week I’m behind my usual desk, with the everyday things.
This is also the third week that I’ve been working full-time with AI-related tasks, and that feels like a luxury. Tomorrow I’ve set aside time to record a tutorial about a new (and cool!) feature that will help us all in HR.
But that’s for another day. Late yesterday evening, I came across a study. And that’s what today is all about.
There aren’t many studies that directly relate to our field when it comes to AI and its use within HR. However, a new field experiment has just delivered one of the most significant findings for HR in the era of AI, more specifically on AI voice agents, conducting job interviews, outperforming (!) human recruiters.
N.B. the AI did not select candidates, it “only” interviewed them and then a human decided who to proceed with or not.
Yes, you read that right. Not screening resumes. Not sending automated scheduling emails. Actual job interviews.
The study, conducted with over 70,000 applicants at a major global recruiting firm, tested three scenarios:
Applicants interviewed by a human recruiter
Applicants interviewed by an AI voice agent
Applicants given the choice
The results are striking:
Job offers: Applicants interviewed by AI were 12% more likely to receive an offer.
Job starts: They were 18% more likely to actually start.
Retention: They were 17% more likely to stay at least 30 days.
Preference: When given the choice, 78% of applicants preferred AI over a human recruiter.
Recruiters expected the opposite. Before the trial, most recruiters predicted AI-led interviews would be lower quality and lead to worse hires. The data proved them wrong.
Why did AI interviews work better?
The transcripts hold the key here.
AI interviews were:
More structured. AI covered more required topics than humans (42% vs. 39% “comprehensive” interviews).
Better at eliciting information. Applicants gave responses with higher linguistic richness, more clarity, and less filler.
Fairer. Reports of gender discrimination almost halved when interviews were AI-led (!!!)
Applicants themselves rated the experience about the same as human interviews, slightly less “natural,” but equally fair, comfortable, and even a bit more positive in open-ended feedback.
This isn’t a small productivity hack. This challenges a core assumption that interviews require humans. (I’ve lost count of the number of times I’ve heard that “sure AI is great, but it can never replace a human interviewer” over the last 2,5 years.)
If you’ve attended any of my trainings, you may recognize the implications this could have, as I’ve discussed this topic several times before, because it will have an impact on HR.
Scalability: High-volume hiring (like call centers, retail, logistics) could shift interviews to AI, cutting costs and time without harming candidate experience.
Fairness: If AI interviews reduce bias, HR has a new lever for building fairer processes. But vigilance is needed—bias doesn’t disappear, it shifts.
Redefining recruiter roles: Recruiters may move from interviewing to evaluating and advising, raising the bar for where human expertise is most valuable.
Candidate expectations: With nearly 8 in 10 choosing AI, HR needs to rethink the assumption that “candidates won’t accept it.” They already are.
It’s not perfect, though. Around 5% of applicants refused to continue with AI, and in 7% of cases, the AI had technical failures. Recruiters also changed how they weighed interview vs. test scores, placing less weight on AI-led interviews. However, it’s still significant enough to have an impact on our profession, one way or the other.
There’s also a sorting effect; applicants with weaker test scores were more likely to choose AI interviews. That could mean AI opens doors for candidates who feel judged by humans, or it could mean lower-quality applicants try to game the system.
For years (especially since the launch of GenAI), AI in hiring has meant resume screening, chatbots, or assessment platforms. This study presents the first large-scale real-world evidence that AI can replace humans in the interview stage, one of the most resource-intensive and high-touch parts of the hiring process.
For HR leaders, the question is no longer whether this will be tried.
It already has been, at scale, with better results than humans. The question is how quickly your organization is ready to test, adapt, and set the guardrails.
The future of interviews might not be between a recruiter and a candidate. It might be between a candidate and an AI agent, while the recruiter decides what to do with the results.
And if this study is right, that future could lead to better hires, happier candidates, and fewer dropouts.
Yeah, I smell a lot of BS in this though not to blame the author of course. I enjoy the article very much. I just believe that there’s a lot of outcomes in recruiting with AI that don’t have specifics that we need to review and know about before we are impressed with statistics like this. The reason why this doesn’t really make sense to me and I called BS on it, is because candidates engaged with automated interactive voice response systems controlled by AI have universally poopooed their interactions with IVR chatbots, just don’t feel a connection with the company because of the impersonal AI chatbot. They don’t always necessarily report a positive experience with the brand at all- many candidates have reported that they don’t feel important to the company at all!! They just put up with it because they need . And that shouldn’t surprise anyone. So this flies in the face of that basic knowledge. I’d have to check the specifics of the social science techniques behind the metrics to actually know if this was an effective study. Moreover, it doesn’t really say what level of hires these are that they’re interviewing for through the AI voice response system, but I’m certain they’re lower level probably the lowest level of hires companies can make. However, I would advocate that even the “lowest level” of hires deserves the best candidate experience with more human interaction! I’m all for AI automating the areas in recruitment that are BRAINLESS like filing organizing taking care of the ATS updates, but the face-to-face areas of recruitment need to be kept to the Recruiter in my opinion.
This is so fascinating. I’m curious iof the demographics of the applicants, particularly age. As AI voice improves I suspect we won’t be able to tell the difference.