A Data-Driven Approach to Designing Candidate Experience Surveys

Candidate feedback can be an excellent source of intelligence for any talent team — that is, if the survey is set up to collect feedback you can take action on.

In this post, we’ll explore designing Candidate Experience Surveys that yield valuable insights through proprietary data findings and qualitative insights from talent leaders seeing positive results. This will include:

• Understanding what a healthy response rate looks like depending on how you design your survey
• Exploring why anonymous survey responses yield the highest response rate
• Candidate experience survey questions to consider trying

---

Designing Candidate Experience Surveys

Let’s start by understanding the core elements: timing, length, anonymity.

Timing

Within timing, there’s a few different considerations.

At what hiring stage should the survey be sent?

Various talent leaders adopt different approaches. Some companies prefer to hear feedback after every stage; others rely on key stages in the application process or close to real-time. There’s a balance here between hearing consistent feedback and overwhelming candidates with too many asks.

In a recent Talent Trends report, we found that response rates are lowest for candidates who don’t make it past the initial screen and then generally even out for other stages.

What day or time should you send the survey?

After an interview is complete, some recruiters set up their surveys to automatically reach the candidate. Cuyler Hirata, Head of Recruiting Operations & Analytics at Notion, says their team automates their surveys to send at a 3-day delay. This allows the recruiter enough time to communicate with the candidate before a survey potentially disclosing a rejection reaches them. Notion’s approach allows for the human aspect of outreach to remain intact while still hearing valuable feedback.

Jeremy Lyons, co-founder of the RecOps Collective, says recruiters overlook the time of day the survey is sent.

“From my experience, sending between 11:00 AM - 12:30 PM is often ideal. People are often distracted during that window and looking for something productive to do. If that is the case, they'll blitz through a survey you send.”

— Jeremy Lyons, Recruiting Operations Consultant

How long does it take to receive candidate experience responses?

While it can be tempting to jump into analysis mode right away, it’s important to allow candidates enough time to respond to your surveys. Our survey analysis found that 94% of responses come within nine days of receiving the survey. [Click to Tweet]

Length

One myth around candidate survey formats? “Shorter is better.”

Rather than taking this tip (often shared by products that provide a cap on the number of questions your survey can include) at face value, we wanted to explore what happens when you send longer surveys.

Turning once more to our Talent Trends report, we found a consistent 20.1% response rate when asking anywhere from 1-10 questions [Click to Tweet]. The response rate dipped to 18.2% once there were 11 or more questions.

From what we’re observing, a more comprehensive approach to your survey design can still yield meaningful results. This means it’s ultimately up to your team to decide what questions are important to ask and then include the appropriate number of options.

For longer experience surveys, one approach can include allowing candidates to skip certain questions and ensure the key feedback you want is marked as required questions.

Daniel King, Global Talent Acquisition Manager at M-KOPA, includes seven questions. Daniel believes scale and multiple choice questions increase the likelihood that candidates respond. However, his final question is a free text response, which he refers to as “a canvas for unbridled insights.” This last question is not required to submit the survey.

“It's not just about the survey—it's about the personal experience, the constructive critiques, and the raw, unfiltered voices that shape how we improve our processes.”

— Daniel King, Global Talent Acquisition Manager at M-KOPA

Anonymous Feedback

Another myth around candidate experience surveys? “Keep them anonymous.”

While anonymous feedback is the gold standard across surveys in most industries, recruiting is a unique scenario to consider. The human experiences of different groups, segments, and even individuals can be vital intel in improving a candidate's experience and ensuring honest feedback.

Daniel is among the top talent leaders who challenged the conventional advice around anonymity before our data was released. He believes the human experiences of diverse groups and individuals are key gems of intel for refining the candidate's journey. And without that context in your feedback process, he finds analyzing the received data becomes a puzzle.

What does the data have to say? Our analysis found that the highest response rates (20.6% average) came from surveys that didn’t allow anonymous responses [Click to Tweet].

Like all aspects of survey design, the ultimate decision depends on your recruitment process and strategy. Providing candidates with the option to share their identity or submit anonymously came in second at an average of 17.7% response rate.

P.S. Ashby helps you set various submission modes and automatically send surveys at the stage of the hiring process you deem best. Request a demo to learn more about our survey tool within the Ashby ATS.

---

Example Candidate Experience Survey Questions

Candidate experience surveys can be configured to have many of the standard question types (e.g. short answer, multiple-choice, scale, open-ended questions, etc.). Rather than a standardized survey template, here are some example questions to consider as you build out your candidate questionnaires. While there’s many here, they can be a starting point for inspiration as you commit to continuous improvement of your feedback surveys.

Communications Related

• [1-10 Scale] I didn’t have to wait long to hear back from my recruiter
• [Open Text] Did you feel that communication was prompt?

Professional Attitudes

• [1-10 Scale] Everyone I spoke to was professional and polite.
• [Multiple Choice] How would you describe the team members who interviewed you? (Professional, Rude, Excited, Knowledgeable, Disengaged, Withdrawn)

Sourcing

• [Multiple Choice] Please select where you heard about this job (Indeed, LinkedIn, Google, Other)
• [Short Response] Can you tell us where you first heard about this role?

Question Quality

• [1-10 Scale] I found the interview questions challenging.
• [True or False] I felt that the interviewers got a good sense of my skills.

Briefing

• [True or False] I knew exactly what to expect from each stage.
• [1-10 Scale] I wasn’t surprised by any part of the interviewing process.

Open Ended

• If I could change one thing about this hiring process, it would be...
• Is there more our team should know?
• What do you wish we’d done differently?

Interview/Recruiter Quality

Cuyler from Notion shared some example questions his team includes with Ashby. The questions Notion asks tie back to specific expectations their recruiting team has on how to operate.

• Were the recruiters clear about the roles we’re hiring for?
• Did the recruiters follow up in a timely manner?
• How much insight did they provide into Notion’s values?
• Did your interview start and end on time?
• Did your interviewer seem prepared?
• Was your interviewer actively engaged throughout the call?

---

Monitoring Candidate Experience Feedback

Naturally, continuously improving on experience surveys means monitoring performance to close your feedback loop. There are two important frames for looking at the data: individual results and all survey results.

Individual Candidate Results

We heard earlier from Daniel and how he doesn’t allow anonymous responses in the M-KOPA candidate experience surveys. Daniel also values looking at individual respondents on his feedback forms — especially when the survey includes negative feedback.

Clicking into the free text question reveals anecdotal data to inform their strategy or if there’s a situation that requires personal outreach to rectify. At Ashby, we’ve seen detractors become promoters when negative feedback is responded to appropriately and considerately.

All Survey Results

Mihaela Suilea, Recruiting Team Lead at Gorgias, routinely checks on the feedback her team is receiving; she pays particular attention to Candidate NPS, the score that reflects how positive a candidate’s experience is with your team.

As an Ashby customer, the Gorgias recruiting team has an automatic chart enabled showing Candidate Overall NPS Score.

Mihaela monitors this monthly and then drills into various filters, such as:

• Department, to understand how the Candidate NPS score may fluctuate function to function.
• Outcome, to explore differences between archived and hired groups.
• User aka Recruiter, to quickly notice if a recruiter is performing exceptionally well or if there are some obvious areas for improvement.

P.S. Ashby provides a prebuilt report template called Candidate NPS Score. It displays the NPS Score according to the standard formula: % of promoters - % of detractors, giving a score from -100 to 100. Request a demo to see how easy it is to track recruiting metrics with Ashby today.

Looking for more data on candidate experience surveys? We dig into even more findings in our complete Talent Trends report on Candidate Experience Surveys. Click here to read it for free (no email required).