Quality of Hire at Enterprise Scale

Laura Barnes, VP of Recruiting at Flock Safety:

What's up, everyone? (00:03)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

What's up? (00:04)

Laura Barnes, VP of Recruiting at Flock Safety:

This is actually my third time on stage. It's getting ridiculous at this point. But when Ashby reached out to me and was like, "Hey, do you want to come and speak again? It's in London," I was like, "Fuck yeah." That's a free vacation, guys. I'm originally from Atlanta, so I got to go to Scotland. (00:04)

Never been there. Took my time in London. It was fabulous. But today, we know we're here to do actual work. I'm putting my vacation on pause. So I'm excited to get to sit down with Tim. We've become close friends. He joined me in SF, and we're bringing it across the pond to y'all today. But Tim, I'm going to give you his bio.

I did not write this. Tim wrote this. Well ... we'll see. I think you're going to like it. Yeah. So Tim is a director of talent acquisition at Sony Sports, where he oversees hiring across five sports technology brands, spanning pretty much every country where they play live sports. His talk today on quality of hire is the result of two and a half years of data, predictive indicators, and intervention workflows that are either genuinely insightful or an elaborate way of avoiding his actual job.

Possibly both. We will let you decide that today. So Tim has 20 years in recruitment, which does explain the hair loss and the excessive crow's feet. We are working on getting him an eye cream and a moisturizer, guys. Don't worry. And when he is not re-architecting systems that were working fine, Tim plays Rocket League at a level that remains diplomatically unquantified.

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Gold now. (01:52)

I got gold last week.

Laura Barnes, VP of Recruiting at Flock Safety:

Okay. Well, let's get this show on the road. So when you stepped into Sony Sports businesses, you told me the board had flagged engineering hiring as a major business risk. Tell me about that. (01:55)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yeah. So I joined coming up to four years now. And all of recruitment needed an entire remodel in terms of processes, ways of working, how we interacted with hiring managers, everything that we did. (02:09)

And it all kind of sprung from the ATS we were using at the time just couldn't give us the answers we needed. It gave us data, but the data wasn't backed up by reality. I was looking at numbers, and none of them made sense when I looked at other numbers. So we got rid of that and came in with Ashby.

And we used Ashby as the integration to rebuild every single piece of the recruitment, every workflow from the ground up. And I think the bits about it that really helped were meeting the team for the first time. They all had the right ideas. They all knew what needed to be changed. They could see the problems, but they didn't have the data.

They didn't have the systems to be able to push back and say, "Look, we're right. This is not working. We need to redo it." So bringing something ... Not the flashcards.

Laura Barnes, VP of Recruiting at Flock Safety:

Not the flashcards. (03:13)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

They're all out of order. I'm going to make it up as I go along. So it was a whole new way of working for us. (03:15)

Laura Barnes, VP of Recruiting at Flock Safety:

I love that. (03:23)

So you definitely made some pretty significant changes to how engineering hiring worked. Structured interviews, tighter feedback loops, and you even removed technical assessments. Interesting. One thing you called out that I definitely want to sit with for a second is this idea that your scorecards came in with a lot of twos and threes.

Where people are tending to sit kind of in the middle. They're not strong no, strong yes. There's no conviction. And we definitely experienced that at Flock too, really early on, where you'd be in a debrief and you're sitting there and everyone's like, "I mean, yeah, we could hire them." No one's excited, no one's pounding the table, and you know that they're going to make that crappy hire.

Not if I'm in the room. I'm going to say absolutely not, but no one wants me on their panels these days. I'm too tough. But it is what it is. But most people, when you're faced with that very... hire, they usually make it. No one's ever brave enough to speak up and say, "Hey, let's not."

"The business is going to run without this person. We're going to be fine." So how did you start to use quality of hire data to coach people out of that middle ground and toward clearer decisions?

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yeah. Look, I think for us, scorecards were a bit of a mess then in terms of did they exist or did they not exist. (04:37)

And as you say, it was that kind of consensus-driven process or scoring that needed to move to conviction. And what we did was we stopped people sitting in the middle. Everyone in this room has probably heard of traffic light systems of scorecards, where it's green, amber, red. And most of the time, hiring managers, interviewers will sit there and go in the middle and go amber, and they won't make the hard decision.

They'll make it someone else's. And the person who shouts the loudest in the room will decide if that hire gets made, and that's not always the best voice to make the answer. So what we ultimately looked at was making sure that we could justify when someone gave a score. Was a three good? Why? If you give them a four, why?

We wanted hiring managers to own that. And where that really tied into quality of hire was we were measuring the outcomes. We were understanding if someone was a good hire. So if someone worked out that it wasn't a great hire, we could bring it back to the scorecards and look at who made the decisions.

How did you score them? Did that actually work out? Was the conviction really there, or did you just go, "Meh, let's get them through. They'll do."

Laura Barnes, VP of Recruiting at Flock Safety:

Oh, I'm constantly amazed at what quality of hire can unlock for people. And in your case, you really rebuilt how all of hiring works. So I think something really exciting is you now have so much data coming in, years of it. (05:52)

And you're really able to sit back and ask, what actually leads to a good hire? So first, I want you to walk us through how you're thinking about quality of hire for your core full-time roles. Your business breaks this into ops and non-ops hiring. So first, let's define what that is so the audience can follow along, and then we'll go into how the quality of hire works.

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yeah. Fabulous. So breaking the business down probably helps to understand. When we say ops and non-ops, ops is our operators on site. So we work in the sports world. If you follow any form of sport, probably the simplest one in the UK is football. I tried to mention that in the US, and it was a soccer thing. (06:31)

There was a joke. Over here, I don't have to do it, so it's good. But if you think of football, VAR, goal line technology, all those pieces are handled by our operators. They do not make the decisions. If you have a problem with VAR, feel free to take it up with someone else afterwards. We just do the tech.

It is the referees. Please don't send this to the referee association. But how that one works out is when we are looking at operators, the way they work in our business is very different from everyone else. So the non-ops track is your central business functions: TA, HR, marketing, sales, C-suite, all those pieces.

So we had to really look at it differently in terms of marking those two areas out.

Laura Barnes, VP of Recruiting at Flock Safety:

I love it. Love it. So apart from just hiring full-time employees, you've done some other stuff too. But before we get into that, break down the actual surveys. (07:32)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yeah. So our surveys for non-ops, fairly standard. (07:44)

We first went with the 30, 60, 90 days. So the whole goal of that one was to really understand the early integration, have a momentum check-in there after 60 days, at 90 days, check the consistency, followed it all the way through. Ops, as I mentioned, it's a very different job. They're aligned due to sport.

So we might have football operators that are two weeks into the business and they're on site at Man United working on medical systems, they're at the stadium. Or we might have a cricket operator that's gone to the West Indies for two months on their third day. So being able to gauge and measure how well they're doing on the same timelines as a non-ops hire just never works.

So we spread it out further so we could look at, okay, how are they doing in month three? That initial: are they filling the role? Have they been on site? What have they been doing well? Are they asking the right questions? Month six goes into CDP engagement, so it's actually their career development plan.

How are they following against that? Are they actioning it? Are they actually inquisitive around where do I need to be against my CDP? And then month 12 falls into promotion readiness, because most of our operators, when they come in after 12 months, will get promoted if they're successful. So it becomes a really quick segue into: are they successful?

Are we promoting? And that becomes the data for that promotion.

Laura Barnes, VP of Recruiting at Flock Safety:

Love that. And you're crazy. You know? You didn't just build two surveys, you built an egregious amount of surveys. (09:07)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Mm. (09:14)

Laura Barnes, VP of Recruiting at Flock Safety:

But I kind of like it. So let's also talk about how you're thinking about quality of hire across contractors, graduates, and university placements. (09:14)

We'll start with contractors first.

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yep. So this was someone on my team and some of the business actually pushing back, because we didn't think of this at the beginning. And what we've realized, and everyone in this room will understand this fairly simply, is that looking at our contractors in a non-ops way of 30, 60, 90 days, that contract might finish in 30 days or it might finish in 60 days. (09:27)

So we don't need to know how well they're doing if they're a permanent employee. We need to ask different questions. Within 21 days we want to understand: are they coming in? Are they ramping up quickly enough? Are they delivering? They're a specialist. They're here to do something.

They're not here to understand if we have beanie bags or if we have a PlayStation in the office. Are they here for the culture or this sort of thing? They're here to ship code. So it became a case of, day 21: are they doing what we want them to do? Are they delivering? Are they going to deliver this in the times of the contract?

If not, it's a fail fast and we exit and find the next contractor. Three months, mid-contract review, and then end of contract is understanding: will we rehire them? Are they going to come back into the business after this? Do we want to try and bring them in as a permanent, et cetera?

How does that work?

Laura Barnes, VP of Recruiting at Flock Safety:

I love it. So smart. Okay, let's hit graduates. (10:32)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yeah, grads. So we've been rolling this out recently and we're just about to do our graduate intake hiring for the year. This one is a two-year plan. Whenever we look at how our graduate plan works, we do the rotation system. So people will come in and they'll move every three, six months, and they'll move to the next rotation within the engineering or operations areas. (10:36)

So we wanted to look at it as a review period, where each hiring manager that runs that rotation is feeding back into an overall graduate engineering manager, and that feedback comes into them and we look at how have they done in this first rotation, how they've done the second rotation, where are they strongest.

Where are they adapting? Where are they showing the most promise? So when we get to the end of it, we can review all of the sum of the parts and understand where should this person land in the business. That's the key thing for us.

Laura Barnes, VP of Recruiting at Flock Safety:

I love it. And then the last one, we're thinking about university placements. (11:28)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yeah, this one's a little bit different. I think for us university placements, yes, we're looking at quality of hire, but it's also the quality of the source. If you use recruitment agencies or, whatever it may be, job boards, it's a source, and universities are the same thing for us. (11:33)

So which universities are actually delivering great candidates for us and people that go on and shine? So we take it as: yes, how are they doing in terms of themselves? How are they developing? How are they learning the material? How are they engaging and dealing with the business? But also, how well are those candidates comparing against other universities?

How is the data and the information they're getting from the universities about us? How is the pitch, the sell from them? And it becomes a kind of route for us to understand which universities we should continue our relationships with or which ones we should then move into new ones.

Laura Barnes, VP of Recruiting at Flock Safety:

I love that. I think that's honestly so smart, and also super useful for universities themselves. (12:20)

They exist to hopefully produce really good outcomes for their students, and if we're able to give them that feedback, they can actually go directly back and fix the programs they're creating, too. So this is super intricate. There's a ton of surveys going on, so I have to know, did you one-shot this?

Did you just sit down one day and you were like, "I'm a genius, boom," or was this multiple iterations? Multiple—

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

—iterations. At first it really was, what's quality of hire, and trying to understand that, and then it was, "Okay, everyone just gets 30, 60, 90 days." And it's like, that doesn't work for this area. (12:46)

Right, let's try this. Okay, the data doesn't work for that. Let's change some of the questions. Are we actually getting the outcomes that we want? Do we really get to the end of this and go, "This person was great. We believe they're going to be in the business for years," or, "We want to promote this person quickly or fast-track them"?

So it took time, and we're still iterating now. I've just finished the new graduate one. I think we're looking at a different one for now for AI, which we'll get to later, which... has a different name, but yeah. Quality of hire in AI becomes different because the signals are changing.

Laura Barnes, VP of Recruiting at Flock Safety:

Mm-hmm. (13:28)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

And I think that's the thing with the continuous integration. (13:28)

Everything changes in our business constantly, and what a great hire was a year ago is not necessarily a great hire in two years' time. Because the things they work with, the way they need to work, AI changes all of that. So we now need to change what we're looking at.

Laura Barnes, VP of Recruiting at Flock Safety:

Absolutely. And I think that's a really reassuring answer, because when I've sat in the audience before and I'm sometimes watching someone present, I'm like, "How the fuck am I going to go and implement this at my own company? (13:48)

How long did this actually take this human to build? Am I just behind the eight ball?" So it's kind of nice to hear that it takes time. No one's behind. You kind of have to push out that V1. Get the data back, iterate on it, and then just keep building. And even when you're finally done building, you're probably going to have to restart building because something's changed.

So Tim, you figured out the survey. Incredible. You have years of data, and now you're able to unlock something so much bigger than just measurement. You're starting to predict performance. So what does that look like practically speaking?

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yeah, look, this is a really scary bit. When I got through the quality of hire piece, I had a lot of data on it, I started trying to go, "Right, how many steps can I take this back?" (14:38)

How do I look at what a great hire is, and how can we understand that at interview? And at some point, how do we understand that much further back before that? But we're not there yet. So interview side, what we want to look at is instead of how did the hire go, it starts turning into, how is this hire going to go?

Like, during the interview, can we actually predict that this person's going to be a great hire? Can we predict things in terms of their quality of hire? So we centralized all of our quality of hire analysis. I built this massive spreadsheet that absolutely had nothing to do with AI, I promise. And we started looking at patterns.

What are the key takeaways? What are the key traits that we're seeing? And now what we've got to is we can turn around and say with certainty that there are three things that every candidate will do during an interview that will lead to them being successful.

Laura Barnes, VP of Recruiting at Flock Safety:

Love it. I think what you're talking about right now is some of the most exciting work that's coming out of quality of hire. (15:43)

And at Flock, we're seeing something similar but different. I'm taking a slightly different approach to it. And so what we've been doing is going back to our interviews and really trying to figure out, hey, does this interview actually produce high signal? Is this actually a really good use of our time?

So for instance, I'll use collaboration. So a lot of our hiring managers, when they're thinking about their interview plan and how they want to build it, they're like, "I want an interview dedicated to, can they collaborate?" And that felt like a really smart use of our time for a hot second. What we've found is that when you actually do the technical assessment, call it pair programming, whatever it is, you're actually demonstrating collaboration, communication inside of that interview.

And so what we found is that that collaboration interview, that use of 30 minutes, it's really low signal. We're actually already getting all the evidence we need in a different interview. So could we actually go and put a different attribute in that section where we can then go and have a better debrief that could then lead to better outcomes and quality of hire, and feel like we actually had a really strong interview process that was truly assessing a candidate on what actually mattered?

And that's been super fun, and something that quality of hire has really opened the door and allowed us to go back and analyze, where in the past we couldn't do it. So there's another point we got to weave in here. You already brought it up: AI. Oh my God. Everyone's talking about it these days. Woo-hoo.

So I think, obviously, in an AI native world, most people now have more access to knowledge and the ability to build than we have ever had before. And so what that means is obviously how you hire has to shift. And I think we both agree that this is hiring towards capability over the exact stack.

So do you think with AI — and you kind of already alluded to maybe a no here, but I'm still going to ask — do you think that these six categories you've identified are going to hold up as quality of hire signals, or how do you think they change?

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

No, probably not. There you go. Job's done. Saved 10 minutes. (17:58)

Yes and no. Look, I think a lot of these are soft skills. None of these are how good are you at AI, how good is your coding, all these sorts of things. These are soft skills. Curiosity is always going to lead to someone bringing and taking on new things very quickly, you'd hope. I think for us it shifted how we score and how we look at and how we need to predict things from now on.

We're evolving into two kind of dimensions of measurement, where we have the quality of hire and then it's the quality of adoption. So how well are people adopting to the AI world, and which of these... Sorry, you can't see the bit that I'm pointing at. That bit — how does that link into it?

So we're currently going through this rescoring mechanism now, because when we started looking at the data in terms of who was a great hire two years ago, when we're looking at, okay, how well have they adopted to the AI world in terms of engineering, it's actually surfaced that a lot of those people who were great hires then haven't actually necessarily adopted to the AI world as well.

So we now need to look at a different way of scoring, a new rubric in terms of what are we looking for, what traits, what things are AI native people bringing through in the interviews, and what are they doing in quality of hire towards the end of those months. So once again, we're in a rebuilding phase, which is hours of fun.

Laura Barnes, VP of Recruiting at Flock Safety:

It never ends. We're constantly building. But I'll echo off of that. Even at Flock with AI, one thing I've definitely seen is a rise in learning agility. That's something we're scoring candidates more heavily on these days. And the question we're really going and asking them is, "What's something new you've learned in the last six months?" (19:28)

They're going to answer, and then the follow-up question is, "Well, what are you learning today?" Curious people who are really good at learning are constantly learning and evolving and changing. You should take that question back. There are fabulous people who are going to be really quick at excelling in a world where we don't know what it's going to look like in six months.

It's kind of cool. The other thing we've implemented too on the AI front is in our recruiter screens, we actually have a gate around AI where we're figuring out how in-depth can this candidate go with it. And we have a really standard rating scale, nothing crazy: strong no, no, yes, strong yes. But if you do not score a yes or strong yes, we will not hire you today, because you're just already behind the eight ball.

Now, we do make exceptions in cases where someone might work in a regulated industry where you just don't have access to AI, but you have intense curiosity. You can tell you're learning in the ways that you're able to. That is absolutely a person we will still bring into the business. So we try to aim for 95% of our candidates being hired as fluent.

The other 5% are reserved for people who are curious but limited due to where they currently work. But—

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

I think the bit that we were talking about the other day was when we actually look at — how we measure if someone's good at AI, it tends to come down to activity. How many tokens have you used? (21:03)

How many prompts have you done? And even having conversations here today with people, that's kind of been a — we get through this many tokens a month. That's not a measurement in terms of quality. That's activity again. So the biggest thing for us that we understood was TA owns the data to measure if the AI adoption is actually quality, not just quantity.

So by just changing how we structure some of our quality-of-hire interviews, we can come out of the back end of it and understand within three months, is this person actually good at this, not how many tokens have they burnt through in the last nine months.

Laura Barnes, VP of Recruiting at Flock Safety:

Your CFO's going to love you. They're going to be your biggest fan — maybe (21:49)

— saving the company money.

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yeah. I'm going to ask her that next time I see her. (21:54)

Laura Barnes, VP of Recruiting at Flock Safety:

Hey, new bestie. All right. I think we could obviously nerd out on AI all day long — but we didn't come here to talk about that, so we got to keep moving. So let's start moving past interview signals. (21:57)

Let's talk about when someone's joined the team. So once a hire is in seat, what are the strongest early indicators that tell you someone's going to be successful? What are you tracking?

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yeah. So first one, team integration. So simple. Probably didn't need the data to tell me that. But it came up. (22:20)

So what we started seeing in the first 30 days for non-ops hires was how well someone actually integrated. Were they asking questions? Were they trying to learn about the business? Were they trying to find out how the team worked? Were they putting ideas through? It really was, are they getting to know the business and their team?

And if people scored an eight in that in the first month, they had an 88% chance of being a quality-of-hire score four or five after the 90 days. So it was our biggest metric in terms of easy wins.

Laura Barnes, VP of Recruiting at Flock Safety:

Love that. Something I'm going to go off of — Ashby just released an amazing talent trends report for quality of hire. (23:03)

Been waiting on this for a very long time. So inside of that, there's one metric I thought was really interesting that we should talk about. There's obviously several, I could only pick one, okay? So roughly 60% of hires land at the same quality-of-hire score they started with, and low first scores rarely improve.

How does this data land with you, Tim? Do you see a change in quality-of-hire scores between the first and second surveys?

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Do you know what? It's rare. We almost have a situation where you look at the quality of hire, and if someone is scoring low, they continue to score low. And what we've brought in is intervention windows to try and combat that. (23:41)

Because if we know in the first 30 days if someone is setting themself up to fail or it's not working, we need to figure out how we can fix that and turn that around. And at the moment, the data doesn't back that up. So we've started putting intervention windows in for all our areas, where if someone is starting to track low, HR or the hiring manager gets involved and tries to work with them to understand what can be changed.

How do we bring this person back up? I think the other aspect to this data that is somewhat problematic is you get that anchoring bias. If you have a hiring manager and you score someone in the first month, and they score them all ones, twos, low, next time they go to review them in the next month, they're going to look at the scores they gave them last time.

And even if they think they're doing great, they're probably going to sit there and go, "Yeah, but they haven't gone up to a six. They haven't gone up to a seven." They're going to give them a three. And that bias causes a huge problem in terms of trying to get someone higher up. So what we're looking at, at the moment, is how do we remove their ability to see the old scores?

Let's forget about that. Let's make it completely blind, so every time you go into it, you're having to think about, "Okay, what did they actually do? What have they delivered? What have they shipped?" Not, "I'll just look at the old scores and base it on that."

Laura Barnes, VP of Recruiting at Flock Safety:

Yeah. Completely agree. We've completed, I think, over 2,000 quality of hire surveys. (25:08)

It's an insane amount of them, and we're using all of that data to go back and look at onboarding, very similar to what Tim is doing, too. We just run our surveys at a different cadence. We do 90 days, six months, and 12 months for our employees. And what we're finding, too, when it comes to the first survey going out at 90 days, is that most habits are set by then.

Whatever you have taught your new hire as a manager, it is very, very hard to change after that point in time. So our scores don't really change from 90 days to six months. So what we have started doing is actually working with our managers to say, "Hey, you need to be giving feedback really quick to your new hires, at day 20, before day 30."

If you let it linger, you have done a massive disservice to yourself and that person, and they might not actually be a bad hire. You were just too scared to give feedback, and that sucks. That's expensive — now recruiting us to go re-recruit the role — and it feels like a huge loss to the business.

And so we're trying to do a better job where we're building them guides that essentially go out at 30, 60, and 90 days, with, "Hey, here's what green things look like. If your new hire's doing this, reward it. Reward it heavily. Here's red flags. If they're doing this, you need to have a conversation immediately and get it back on track."

And if you're seeing these same things be exhibited at 90 days, yes, we need to be having a conversation with HR, but hopefully by that point in time, you've actually had the conversation, and you're not just coming to HR with, "Hey, I want to fire this person," and you've done no work — because that's annoying.

But on that note — when the data flags that a new hire is trending the wrong way, what actually happens next? What has this allowed you to do differently?

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yeah, as I mentioned earlier on, a big thing for us was, A, understand the intervention window, so looking at the data across the different departments to understand, okay, when does that intervention need to happen? (27:05)

And then the other aspect was working with HR to understand, okay, how do you need to get involved? When do you step in? And the biggest, I suppose, win for us came out of, well, when does onboarding start then? When does TA hand over to HR? Because I think that can always be a really messy phase in terms of, we've made the hire.

Let's forget about that now. HR, pick it up. And that person's built a relationship with TA. They trust them. They're going to come to you. They're going to ask you questions. So what we've done with our team is TA now owns onboarding up to the intervention window. So everything through from the guides that we send out to them, the training in terms of quality of hire before a candidate starts, all of that's owned by us.

And then when we get to that intervention window, it's kind of a situation of, "Look, they're doing amazing. Carry on doing what you're doing. HR, here you go." Or it's a case of, "Okay, we need to review this. We need to sit down. HR, HM needs to sit down and try and understand, is there something fixable here?

Are we doing something wrong? Was the hiring manager on holiday for the first month, so this didn't work out? What needs to fix? What needs to be changed?" So that's kind of where we're at the point of now, bringing that into the business and seeing if we can turn some of these scores around with early intervention.

Laura Barnes, VP of Recruiting at Flock Safety:

I love it. And then let's sit here for a moment. Let's talk more about how you're coaching new hires on this data. So what does it actually look like in practice? (28:26)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yeah. When we looked at this data, and we took it back to the scorecards, it became really obvious in terms of, well, these are the three things that will make someone successful. (28:35)

Why don't we just tell them that? If you're doing an interview these days, you'll tell them what type of questions they're going to get, what the content of the interview's going to be like. You help coach them. You don't give them the answers, but you help coach them in terms of how to be successful in an interview.

All TA do it. So why don't we say, before they start, "Hey, these are the three things you do. When you come into the business, ask questions. Learn about the different departments. Go and sit on someone else's stand-up. These are some meetings that you can join over the first few weeks to really learn about other areas of the business.

Care about your career development plan. If your manager's not talking to you about this, ask them questions. What do you need to do to get to the next level? What do you need to do this? How do you get this training?" So we put together a document that is essentially there as part of onboarding of before you start — like, yeah, your laptop's going to be here.

Yeah, the bean bag's over there. Your desk is there. But these are the actual soft things you need to do when you come in. Ask the questions. Learn about the business. And that's what we've put into play. So we're waiting to see how that works. Does it change? I think you almost get that situation of, well, if we give them the answers, does that devalue the quality of hire scores, because we've now told them what they need to do, which then means we have to reiterate again, and I have to do this again.

I have new questions. Yeah. Looking forward to that.

Laura Barnes, VP of Recruiting at Flock Safety:

Always building. Always building. That's the theme today. But yeah, I love the new hire guide you did. I ripped it off. That's okay — this is why I love meeting good talent leaders. You get a good idea. I went home, and I basically built it myself. (30:08)

I was inspired. Absolutely. That's why I like these things. But we're doing the same stuff — we're sitting there, again, trying to feed to that new hire, as they're in that critical ramping period, "Here is what good looks like. These are those traits. Exhibit it. Here's how it should look at 30 days.

Here's how it should look at 60, 90. Here's how you're going to grow in it. This is why we reward that." We're trying really, really hard to almost give them those keys to success—

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yeah. (30:50)

Laura Barnes, VP of Recruiting at Flock Safety:

—which is what we're going to talk about here, and I think we've both been alluding to this a lot, is, you should give your new hire the blueprint to success. (30:51)

We talk a lot in recruiting where we're like, "This should feel like a test, not a pop quiz." Then why are we making onboarding feel like a pop quiz? Shouldn't it be a test? Because even if we tell you what it takes to be successful, you still have to do the work to be successful. So for you, what are the actual behaviors that, when you see them, you know someone's going to succeed?

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yeah. And I think I've covered some of them already, but really: communication, curiosity, business and team integration. Those three things are the simplest elements of what we need and what works in our business, and it's probably the same for most businesses. Be curious. Oh, that sounds like a Ted Lasso thing. (31:20)

I may have ripped that off. Sorry. The new season's fantastic, by the way. But they're the biggest things for us in terms of understanding: is this person going to be a strong hire, and they're the only things that kind of matter. It doesn't matter how good you are at Java engineering or anything along those lines.

Everything can be learned and everything can be figured out, and that never comes into it in terms of their ability to do the job. Can they learn to do the job — I think that's more relevant to us.

Laura Barnes, VP of Recruiting at Flock Safety:

Absolutely. I'm completely aligned here. I find soft skills have been a main differentiator in our top performers as well. (32:05)

In fact, if you go and look at a lot of our termination data at Flock, you would actually find that most people who do not succeed at this company fail because they don't have soft skills. They can all do the job. We rarely ever terminate someone because they are incompetent and can't do the work we hired them for.

We tend to have to part ways with them because they cannot do the work at the speed we need them to. They cannot work at the ambiguity that exists inside the company. It's these unique little soft skill factors that we've been really trying to hone in our recruiting process, to make sure we're finding people who are aligned to who we are as a culture, and that they can come in and be successful with us.

So you've obviously built something amazing. There's been a ton of people taking photos of your work, which is snaps to you. Love that. So how are you feeding... Snap.

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

I don't understand this. (33:03)

Laura Barnes, VP of Recruiting at Flock Safety:

Oh, it's okay. We got to get you on TikTok. I don't think that would help... (33:04)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yeah, I don't. (33:07)

Laura Barnes, VP of Recruiting at Flock Safety:

We can work on that after this. (33:09)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

I get really concerned when I see people doing that. I'm like, "Have I done something wrong? Is it like a bird dancing?" No, it's— (33:09)

Laura Barnes, VP of Recruiting at Flock Safety:

—like a, it's like a good thing. (33:13)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Oh, is it? (33:15)

Laura Barnes, VP of Recruiting at Flock Safety:

Snaps. (33:15)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Okay, cool. All right, cool. I don't know. (33:15)

Laura Barnes, VP of Recruiting at Flock Safety:

He's not cultured, guys. I'm sorry. We're working on it. That's— (33:17)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

—it. I've— (33:19)

Laura Barnes, VP of Recruiting at Flock Safety:

—been trying hard the last few days. (33:20)

How are you feeding this back into how you hire the next person?

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yeah, look, quality of hire is the strongest data you are going to have when you're looking to make a new hire. Because if you're looking at a job description or looking at a new job and going, "Okay, how we hired that two years ago is the same as how we're going to hire it now," then you're wasting the best data you've got. (33:28)

Every conversation about a new job should be different, because a new person may have come into that team and completely changed the way they work. They might have come in and said, "We need to use this tool. We should be working this way, or this process should be changed." It's not static. Oh, continuous improvement again.

There we are. So what I say is that review your last hire's performance. When you're going to an intake meeting, look at the last person that was hired, and look at what did they do well, what did they not do so well, what did they display. Do they want that again? Has the role now changed that person who's doing those things?

How has the role evolved since that person came in, and how do you want the role to evolve after this? So that's the biggest bit for me. Never treat a job as the same as the last one you hired. Treat everyone differently, but always bring it back to the last person you hired, or the last few people you hired.

Laura Barnes, VP of Recruiting at Flock Safety:

I love that. I'm a huge believer in it, too — where we're feeding our quality of hire data back into all of the kickoffs for new roles. And something we've adopted inside of onboarding that my team has enjoyed is we actually have recruiters at 90 days meet with all of their hires and essentially do a stay interview. (34:37)

And it's fun, because it's like — that's your friend. That's the person you probably have one of the closest relationships in the company, and you get to sit there and be like, "Hey, this thing I sold you, did it come true?" What's the job like? Were there good surprises? Were there bad surprises?

And we're capturing all that data with Granola, and then feeding that also into the kickoff process as well, so we can go and correct our job descriptions with actual feedback from the humans who are doing the job. And that's been super helpful in how we just go and pitch that role to the next candidate, and how we can avoid, again, unpleasant surprises down the road when we're bringing someone in.

Is there this weird caveat we didn't actually know existed and now we've unfortunately found out? That's kind of the nature of a startup. Roles change so much. But one thing I want to end on that I think is kind of a fun, sentimental thing — do you find quality of hire has unlocked more happiness for your team in their role?

It sounds like you've actually been able to transition to being true business partners, which I think is what every single recruiter wants. We just want that seat at the table.

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yeah. I have to answer this question differently now, because I told my team not to come to this. (35:56)

Laura Barnes, VP of Recruiting at Flock Safety:

Because— Oh, and then they came— (36:01)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

—I did, yeah, they turned up. I felt like — I thought there was only one, and then I saw the rest at the back and was like, "Bugger." So I need to change my answer to be polite. (36:02)

Laura Barnes, VP of Recruiting at Flock Safety:

Should we ask them if they're happy? (36:10)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yeah, no, no. I think the thing with this is a sense of pride. The modern metrics that recruitment is always gauged by — time to hire, time to fill, all that crap. (36:12)

It tells you when the race is over, and what quality of hire tells you is, did you back the right horse? And I think from a team perspective, being able to not just disconnect and move on to the next job — it's actually taking an interest in how that person is doing, how are they onboarding, are they succeeding in the business?

And being able to readily access that in Ashby allows them to realize they're doing an amazing job, and I think that gives a huge sense of pride, more than, "Oh, I filled the job in 20 days. That's an awesome metric." It's like, "No, I've hired someone that's built this amazing thing that's changing the way that we work."

One hire can completely change a business. So I completely believe that if the team are checking their quality of hires on a regular basis, like I tell them to, then they will of course be very prideful. Fuckers. Sorry, carry on. We're—

Laura Barnes, VP of Recruiting at Flock Safety:

—going to assume they are. (37:14)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yeah, yeah. (37:15)

Laura Barnes, VP of Recruiting at Flock Safety:

But— Oh— (37:15)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

—they're all great— (37:16)

—they're great.

Laura Barnes, VP of Recruiting at Flock Safety:

They're great. We love your team. We— (37:17)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

—love them. (37:18)

Laura Barnes, VP of Recruiting at Flock Safety:

But I would definitely agree with you there. Even when I look back on my career — I joined Flock as a recruiter, so some of my hires still work in the business — but the ones I'm most proud of are the ones who came in as an engineer and now they're leading teams today, or they came in as a director, now they're VP, or they're a principal. (37:18)

And you're just like, "Oh my God, we have been on this five-plus year journey together. We have been through thick and thin." That's such a badass hire. Those are the ones I want to pat myself on the back and be like, "Who's a good recruiter? You are." And it's those moments in your career that you feel really good, because that's the thing you're trying to do — you want to match people to the right opportunity.

We want to give them that amazing thing that's going to go and challenge them and make them feel proud and tell their partner about their work or their family. That's what this job's ultimately about. But we're wrapping up here. I'm in the yellow minute time. Elizabeth, I can keep going, but I will stop myself.

Tim—

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yep ... (38:17)

Laura Barnes, VP of Recruiting at Flock Safety:

—thank you for sharing the stage with me again. I appreciate it. I didn't scare you off in San Francisco, so that's huge. (38:17)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yep. (38:23)

Laura Barnes, VP of Recruiting at Flock Safety:

Yeah, you did it. (38:23)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yeah. When's San— We're doing Ashby One Maldives, Ashby One China— (38:25)

Laura Barnes, VP of Recruiting at Flock Safety:

—I know. My husband's asking for Ashby One Tokyo. (38:29)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yeah? Oh. (38:32)

Laura Barnes, VP of Recruiting at Flock Safety:

Thoughts? Anyone? (38:33)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Elizabeth's nodding. This is on camera. (38:34)

Laura Barnes, VP of Recruiting at Flock Safety:

Yes. All right. Benji, I want to go to Tokyo. (38:37)

Tim Chattaway, Director of Talent Acquisition at Sony Sports Technology:

Yep. (38:39)

Laura Barnes, VP of Recruiting at Flock Safety:

But aside from my travel plans, the story you shared is incredible. I love it. I've obviously been learning from it. I'm copying your work. I hope many of you copy his work. Put your own flavor to it. (38:40)

That's very important. But you were able to take something that I think most people think is daunting, which is sometimes quality of hire, and turn it into something that's intricate, but also show that it takes time. It's not perfect. We're all going to be building. We'll probably — if you see us again in six months, our systems will probably look entirely different.

But needless to say, I'm inspired. I hope all of you are inspired. Thank you for having us.