What I tell recruiting teams who are skeptical about AI interviewing
10 minute read
We’ve just launched Ashby AI Interviewer to the public, and I want to address the questions I hear most from recruiting teams. Most are practical: what will this mean for their time, their candidates, and the hiring decisions they’re responsible for making?
My co-founder Ben Chaikin and I asked ourselves the same questions while building Talent Llama, and again when we rebuilt the AI Interviewer inside Ashby. We set three goals: interviews that feel practical for candidates, outputs hiring teams can use, and a workflow connected to the rest of the hiring process.
The skeptical questions below are the ones we debate internally, and several of them changed what we built.This post goes one level deeper on the questions I hear most.
1. “If my team re-checks every answer, haven’t we just added a step?”
It’s a fair question. In Ashby's own hiring, our recruiting team estimates that reviewing a completed first screen takes 5 to 7 minutes, compared with a 30-minute live screen plus 15 minutes of prep.
Human review is the safeguard, and Ashby’s terms require a qualified person to review the output and remain responsible for every hiring decision. The goal is for that review to take less time than a live screen.
Reviewers see a summary and competency-level evaluations backed by observations and direct quotes, with the transcript and recording available to check the evidence. There’s no overall candidate rating. Every interview needs a human review. Reviewers should go deeper where the evidence is mixed or incomplete before deciding what happens next.
Compare the whole workflow, including coordination, preparation, review, feedback, and no-shows. If reviewing AI interviews consistently takes as long as live screens, revisit the criteria and whether the step fits the role.
2. “Candidates won’t want to talk to an AI”
Some won’t, and they should have another way to continue. Others may welcome an interview they can complete on their own schedule, especially when time zones or work commitments make a live screen difficult to arrange.
You can offer the AI interview alongside a direct-booking link for a human conversation in the same invitation. We’ve used this approach in our own APAC hiring, where aligning with US recruiter hours can be difficult.
James Herriotts, Senior Director, Global Talent Acquisition at Oviva highlighted the flexibility for clinical candidates:
“For our clinical candidates, the real benefit is a two-way conversation they can complete around their fixed work schedules, with the chance to ask their own questions.”
Candidate feedback from our design partners and early access customers has been encouraging: 92.9% of candidates who left an optional rating gave their AI interview a thumbs-up.¹
Completion is a useful signal, too. Sofie Roberge, Director of Talent Acquisition at A1 Garage / TMII shared:
"So far, we’ve completed 106 AI interviews and made 10 hires, with a 55% interview completion rate. Other AI interviewer platforms we tried only resulted in 10-15% completion rates. How you introduce the experience matters. Explain what the conversation involves, how your team will review it, and what happens next. Make the alternative path clear and keep meaningful human conversations in the process. Then review completion, advancement, and candidate feedback to understand how the experience is working."
¹ Among 689 candidates across 36 organizations who left an optional rating between July 1 and November 5, 2026.
3. “Will it reject good candidates or introduce bias?”
A person on your team makes every hiring decision. Ashby AI Interviewer never advances or rejects anyone on its own. It evaluates candidates against requirements you define and shows the evidence behind each competency’s assessment, so reviewers can weigh it before deciding what happens next.
That starts with interview design. Each competency needs observable requirements. Your team, as the experts in the role, sets observable requirements for each competency. These shape the rubric bands and example answers. Before publication, Ashby is designed to flag vague questions, criteria that can’t be evaluated, and potential conflicts with EEOC guidelines. Your team must correct or explicitly accept those flags before publishing. Publishing over a raised flag is your team’s decision. Candidates are assessed against the same criteria, while follow-up questions can adapt to their answers.
Interviews can also include minimum qualifications; clear, objective, yes-or-no requirements, such as whether a candidate has the legal right to work in the country or a required professional qualification. If an answer doesn’t meet one of your set minimum qualifications, the AI repeats it back and asks the candidate to confirm before ending the interview. Ending the interview does not reject the candidate. Your team reviews the assessment and decides what happens next, and a recruiter can reset the interview.
Warden AI independently audited the evaluation process for bias using 65,880 synthetic interview transcripts across 15 demographic and protected-characteristic categories. Its September 1, 2026 audit reported Clear results across every category tested. Those results may not reflect every customer’s configuration or candidate population, so teams should continue reviewing outcomes for their own roles and candidates.
For the first 10 to 20 interviews, I’d have the recruiter and hiring manager review the same candidates and compare judgments. Use disagreements to sharpen the requirements. If you offer AI and human paths, compare pass-through rates across both.
4. “What about privacy, consent, and regulation?”
What candidates consent to
Candidates are told in writing in the onboarding flow before recording starts, and again out loud when it begins, that they’re interacting with AI. It explains what will be recorded, including video when enabled, which integrity signals are observed, how the hiring team uses the results, and how to decline or request an accommodation or alternative.
Candidates must affirmatively consent to continue. Teams can customize the terms while retaining the core disclosures and consent step.
What the evaluation uses
Used for post-interview evaluation:
Transcript text
Criteria your team defines during setup
Not used for post-interview evaluation:
Audio or voiceprints
Facial recognition
Emotion or affect
Accent or first language
Pitch, tone, pace, or volume
Pause length or fluency
Pauses are noted in the transcript for human reviewers but aren’t used in the evaluation. The live minimum-qualification check described above is separate from the post-interview evaluation and remains subject to human review. If video is enabled, reviewers can watch it, but it isn’t sent to a model.
How candidate data is handled
Ashby’s AI Features Terms and Product Specific Terms prohibit Ashby and its AI providers from using customer data, including candidate audio, video, and transcripts, to train, fine-tune, or improve models.
Recordings, transcripts, and evaluations remain in Ashby for authorized hiring team members to review. Retention and deletion follow your agreement with Ashby. The AI Model Cards and subprocessor list explain processing and the providers involved.
Candidate data is protected by Ashby’s SOC 2 Type 2-audited security program. Your team can also delete individual AI interviews, removing the recording, transcript, evaluations, review scorecards, and candidate feedback and rating.
The Warden audit covers Ashby’s evaluation process. It doesn’t replace any bias audit your team may need for its own hiring. Our Product Specific Terms explain those responsibilities, so involve your legal and security partners early.
5. “Can’t candidates cheat?”
It’s possible, as is with a live screen, and a polished answer can leave you wondering how much someone actually knows. AI Interviewer surfaces integrity signals for a reviewer to weigh, and the interview design gives your team enough detail to probe what a candidate knows.
Ashby shows reviewers integrity signals, including when candidates leave the interview tab or window and response delays of more than 60 seconds. A candidate fraud check runs automatically at the start using device fingerprint and IP information, and it doesn't use AI Credits.
Before the interview begins, candidates are informed about integrity monitoring and its informational purpose. These signals do not affect competency assessments or trigger automated hiring actions, and they are not proof of misconduct. They give reviewers additional context to consider alongside the candidate’s answers.
Interview design helps, too. Ask candidates to explain specific work, decisions, and tradeoffs. You can set follow-up depth to Minimal, Probing, or Deep Dive, using deeper follow-ups for key skills so reviewers have more than a rehearsed first answer to assess.
If your assessment requires a lockdown browser or identity verification, use a tool that provides those controls. Your hiring team still needs to weigh the answers, integrity signals, and other candidate evidence before making a decision.
6. “AI is wrong for relationship-heavy roles”
For certain conversations, I agree. Building rapport, persuading someone to join, and helping them understand the team are important parts of recruiting. I’d keep people responsible for those conversations.
The boundary depends on what you need to learn. AI Interviewer is conversational; it does not include coding tests or embedded exercises. If a work sample is the clearest way to assess a skill, use one. For executive searches and small, high-touch pipelines, I’d keep the first conversation human.
Start with the interview’s purpose. Use AI where a structured conversation gives more candidates a chance to show what they know. Keep live conversations where building a relationship is central.
7. "How is Ashby AI Interviewer different from other AI interviewing tools?"
Ashby AI Interviewer is a native step in your Ashby recruiting workflow, not a third-party integration or white-labeled product. The biggest difference is how the interview connects to everything around it. We rebuilt AI Interviewer inside Ashby so candidate context, scheduling, evaluation, and follow-up are all part of the same hiring process.
When a candidate finishes the AI interview, the transcript, recording, and evaluation sit on their record alongside their application and other interview feedback. Completion, candidate feedback, and advancement appear in Ashby reporting. There’s no separate candidate record or interview integration to maintain.
Jordan Duffy, Director of People Systems and Technology at Neo Financial Technologies Inc. described why that mattered to its team: "We were already using AI interviews for high-volume hiring, but wanted the conversation to be part of the same recruiting funnel as everything else."
When comparing AI interviewers, I’d test the whole workflow: configuring the interview, inviting candidates, reviewing their answers, and following up. That’s where having AI Interviewer built directly into Ashby makes a practical difference.
Quick answers
Will it replace recruiters?
No. Recruiters keep the relationship, the judgment, and ownership of the candidate experience. AI Interviewer adds a structured conversation wherever it fits in your interview plan, without adding another live call to someone's calendar. A person on your team still makes every hiring decision.
How is this different from AI-assisted application review?
Application review helps you assess the information candidates have already submitted. AI Interviewer gives them a conversation in which to explain their experience and demonstrate knowledge beyond the resume.
Which languages does it support?
English (US, UK, and India), German, French, Italian, Swedish, Latin American Spanish, and Brazilian Portuguese.
Can candidates retake an interview?
A recruiter can reset it for another attempt. That deletes the previous results, transcript, and review feedback. Candidates cannot reset it themselves.
What if a candidate declines or stops an AI interview?
Candidates can decline before starting from the welcome screen, or from their invitation if your team includes a decline link. Declining doesn’t automatically schedule a human interview; your team arranges the alternative.
If a candidate asks the AI to end the conversation after starting, the interview counts as completed and the partial transcript is evaluated. They can still contact your team to request an alternative format or ask for the interview to be excluded from consideration or deleted. Your team handles those requests.
What if a candidate has technical issues?
Your team can enable typed responses so candidates can answer in writing. If a connection drops, candidates can reopen the same link, on the same device or another one, and select Resume within 15 minutes of their last answer. After that, the interview is marked Abandoned, and a recruiter can reset it if another attempt is appropriate.
Put it to the test in your own hiring
Before inviting candidates, take the interview yourself and have a colleague who knows the role try it, too. Compare the evaluations with what each person said, and bring your legal or people partner through the candidate experience.
Then run a small pilot with a named reviewer and a clear alternative for candidates. Keep interviews to 15 to 20 minutes, with two to four competency topics and a 30 to 60 second welcome video. Aim to review completed interviews within one to two business days.
Track completion and advancement in the AI Screening Dashboard, and follow up with people who decline. If you offer AI and human paths, compare the results.
All-in-One plans include 200 AI Interviews at no cost. It’s a one-time allotment, and those interviews don’t expire. An organization admin can enable AI Interviewer under Admin > Opt-In Features.
If you're new to Ashby, request a demo and we'll walk through where it could fit in your hiring process. To see the product in action, watch the launch webinar recording or read Benji's post, Where AI Interviewing Fits, and Where It Doesn't.
Use what you learn to decide whether the workflow gives your team useful evidence, makes good use of recruiters’ time, and gives candidates a useful opportunity to show what they know.
As you try Ashby AI Interviewer, reach out with questions or feedback. Hearing what worked and what didn’t helps us decide what to improve next. Much of what’s in this post exists because design partners and early access customers told us where we got it wrong. Our next round of improvements will come directly from your experience hiring with this new capability.

