Episode 61

Building Interviewer Scorecards to Raise Your Talent Bar

About this Episode

In this episode, Vanessa Paladini, Global Talent Acquisition Leader at Nubank, walks through the 18-month process of building interviewer scorecards at one of the world's fastest-growing digital banks. Vanessa explains how Nubank defines and measures interviewer quality, why a smaller well-calibrated pool outperforms a large inconsistent one, and how the data revealed just how uncalibrated their interviewers had been.She also gets into the challenge of building a clean data foundation before any meaningful measurement is possible, how Nubank structured cross-functional partnership to get buy-in, and what it actually looks like to move hiring accountability from sitting solely with TA to being owned across the whole team.

Topics

interviewing

Takeaway 1

Fewer Interviewers, Higher Standards 🎯

Every instinct says that scaling hiring means expanding your interviewer pool. More people, more availability, more throughput. Nubank did the complete opposite. They reduced the pool down to their best, most calibrated people, ran the process with that tight group, and only expanded once new additions had been properly trained. The quality of outcomes went up, not down.

Why It Matters:
Sometimes you have to slow down to speed up. If your interviewer pool is large and inconsistent, your hiring decisions start to look like a coin flip. Vanessa's team saw this in the data: two interviewers in the same debrief, assessing the same candidate, one a strong yes and the other a definitive no. A smaller, intentional pool removes that uncalibrated variance, and those top interviewers become the training mechanism for everyone who comes after them.

Quick Tips

  • Know your funnel before you cut. Reducing the pool only works if your remaining interviewers can still meet hiring demand without creating a bottleneck. Map your volume first so you know the floor before making any changes.
  • Identify the interviewer characteristics that become your selection criteria. At Nubank, assertiveness was the core signal, meaning whether an interviewer's decisions correlated with strong hiring outcomes over time. The right traits will depend on your organization, but define them explicitly so the selection is principled, not just based on who is available or enthusiastic.
  • Tie interviewers to a single profile and level. Vanessa found that interviewers who assessed one level consistently made significantly better decisions than those switching between junior, mid, and senior in the same week. The cognitive load of constant context switching degrades calibration, especially at high volume.

Takeaway 2

Bad Data Will Undermine Your Hiring Bar 📊

Before Nubank could measure interviewer quality, they had to reckon with the state of their existing data. What they found when they went to look was a mess. Interview notes were thin, scorecards were inconsistently filled out, and the raw material needed to connect interviewer decisions to hiring outcomes simply was not there yet. Cleaning up the data was the unglamorous and tedious first chapter of an 18-month program.

Why It Matters:
You cannot build a meaningful scorecard on top of bad inputs. Most organizations underestimate how much of their interview data is effectively unusable until they go looking. The outcome signals Nubank tracks now, including 90-day ramp, 6-month performance, 12-month retention, and promotions within 24 months, only became meaningful once the underlying data was worth trusting. Some of those are still forward-looking for Vanessa's team, but it is critical to design the system now so the data is there when you need it.

Quick Tips

  • Start with a data audit, not a scorecard design. Before building anything, go look at what you actually have registered in your interview notes and scorecards. Understanding the gaps tells you what needs to change in how interviewers are documenting their assessments before you can measure anything meaningful.
  • Define your outcome signals upfront and sequence them realistically. Decide which post-hire data points matter most for your organization — 90-day ramp, first performance cycle, retention at 12 months — and build toward collecting all of them. Some will take time to accrue, so start tracking early and be patient with what the longer-horizon signals can tell you.
  • Use AI notetaking to close the data gap at scale. Nubank adopted a notetaker with an embedded coaching system that gives interviewers feedback on how they are assessing skills in real time. It helped them scale calibration beyond what one-on-one coaching alone could ever cover.

Takeaway 3

Accountability for Hiring Outcomes Belongs to Everyone on the Field ⚽

For a long time at Nubank, hiring outcomes were tied almost entirely to the talent team. Vanessa described the shift the company had to make: recognizing that hiring is a team sport, and that the interviewers making decisions at each stage of the process share responsibility for the result. That mindset change is what makes the scorecard program land differently than just another internal initiative.

Why It Matters:
When interviewers have no connection to what happens after their decision, there is no real incentive to get better. The scorecard program is designed to create that feedback loop over time, connecting individual interviewer decisions to real hiring outcomes so patterns become visible and coaching becomes specific. Making that infrastructure visible within talent acquisition first, before broadening access, is what allows the accountability to deepen as the program matures.

Quick Tips

  • Frame the scorecard as development, not inspection. Vanessa's team introduced this as a way to help interviewers get better at an important job, not a tool to scrutinize them. That framing made the pilot feel natural, especially in data-fluent functions where the conversation around measurement was already comfortable.
  • Partner with each function to build the criteria together. Nubank worked directly with engineering, product, and business analysis teams to define what good interviewing looked like for their specific functions and levels. When the business co-creates the standards, the program does not feel like something being done to them.
  • Keep scorecard visibility within talent acquisition to start. Vanessa's team limits access to TA only, not hiring managers. This protects the developmental intent of the program while the data is still maturing. Once you start surfacing that data more broadly, you want to be confident in what it is saying.

What Hiring Excellence Means to Vanessa

For Vanessa, Hiring Excellence is a mutual learning experience. The employer learns about the candidate, but the candidate should walk away having gained something too: a real challenge, a new perspective, a genuine sense of what working there looks like. Excellence is a two-way path. It also starts with executing the basics so well that none of it looks effortful to the people going through it, because scheduling, feedback, and follow-through are still where most processes fall apart. And from a talent leadership standpoint, excellence means being brave enough to coach your stakeholders, drive the process with intention, and use data to back up the moments where you need to push back.

Watch the clip >>>

Vanessa's Recruiting Hot Take 🔥

Stop trying to block AI in your hiring process and start figuring out how to assess whether candidates actually know how to use it. Recruiting teams spent years trying to detect AI use and shut it down, and that window has closed. The more useful question now is whether a candidate is using AI with real judgment and genuine understanding, or just pasting outputs they cannot explain. Especially in technical roles, if someone is building a business case or writing code, AI use is expected. Design your process to probe the quality of that usage rather than the fact of it.

Watch the clip >>>

Timestamps

(00:00) Introduction

(01:03) Meet Vanessa Paladini

(01:40) Three pillars of Nubank's Hiring Talent bar

(03:15) Four workstreams to raise the talent bar

(06:15) Why companies should invest more on interviewer quality

(08:19) Building and measuring interviewer scorecards

(11:19) Defining the traits of a good interviewer

(14:41) Upskilling interviewers while avoiding disruption

(16:08) Who has access to the Scorecards?

(19:00) Additional learnings from the Scorecard program

(22:30) How the Scorecard data is expected to impact conversions

(25:20) Hiring Excellence: Mutual learning experience from hiring processes

(27:50) Hot Take: Candidates should be assessed on how good do they use AI tools

(31:18) Career advice: Stop optimizing for speed, start optimizing for learning

(35:06) Where to connect with Vanessa

Hosted By

Shannon Ogborn

RecOps Consultant & Community Lead @ Ashby

Shannon Ogborn is a Recruiting Ops expert with nearly ten years of experience at companies from Google to Hired Inc and more. She’s shining a spotlight onto what makes a recruiting strategy one of a kind.

Other Episodes

Simplifying Recruiting Metrics to Improve Team Performance

Simplifying Recruiting Metrics to Improve Team Performance

In this episode of Offer Accepted, Bryan Power, Chief People Officer at Nextdoor, shares how recruiting teams can simplify their process using the AAA framework: Attraction, Assessment, and Acquisition. Shannon and Bryan discuss how breaking the funnel into distinct stages helps teams identify where things are going wrong and take action with more clarity. Whether you're leading a small team or scaling operations, this episode offers practical ways to use data, structure, and empathy to improve hiring outcomes.

Listen to this episode
Building Internal Mobility to Improve Hiring and Retention

Building Internal Mobility to Improve Hiring and Retention

In this episode of Offer Accepted, Dr. Kalifa Oliver, Director of Employee Listening Strategy at Ford, shares how internal mobility connects hiring, retention, and equity. She explains why internal candidates often face more bias and less support than external ones, and how recruiters can take a more active role in building fair, effective internal hiring systems. Kalifa also breaks down the structural flaws that limit movement and shows how small changes can lead to better outcomes for candidates, teams, and the business.

Listen to this episode

Join the Hiring Excellence movement

New episodes every month - subscribe here so you never miss out.