Jul 29, 2014

    Crunching Big Data to manage people

    LASZLO Bock, senior vice-president of people operations at Google, shares how the tech giant uses Big Data to improve hiring and the performance of managers, and why academic results often have no relation to how an employee performs at work.

    How is Big Data being used more in the leadership and management field?

    I think there has been a fairly recent confluence of the ability to crunch lots of data at fairly low cost, venture capital investments that support new businesses in this field, and changes in what people expect.

    Leadership is a perennially difficult, immeasurable problem, so suddenly people are saying: "Maybe I can measure some piece of it."

    Part of the challenge with leadership is that it is very driven by gut instinct in most cases - and, even worse, everyone thinks they are really good at it. The reality is that very few people are.

    Years ago, we did a study to determine whether anyone at Google is particularly good at hiring. We looked at tens of thousands of interviews, and everyone who had done the interviews and what they scored the candidate, and how that person ultimately performed in their job. We found zero relationship.

    It is a complete random mess, except for one guy who was highly predictive because he interviewed only people for a very specialised area, where he happened to be the world's leading expert.

    What else has Google done in this field?

    I have to preface the answer by saying that when we look at any data related to our people, we treat the data with great respect. Typically, we give people an option to participate in anything either confidentially or anonymously. The lesson for anyone looking at this space is that you need to construct this really powerful tent of trust in the people gathering the data and how they use it.

    We have done some interesting things to figure out how many job candidates we should be interviewing for each position, who are better interviewers than others and what kind of attributes tend to predict success at Google. On the leadership side, we are looking at what makes people successful leaders and how can we cultivate that.

    We are also observing people working together in different groups and have found that the average team size of any group at Google is about six people. So we are trying to figure out which teams perform well and which do not. Is it because of the type of people? Is it because of the number of people? Is it because of how they work together? Is there something in the dynamic? We do not know what we are going to discover.

    Other insights from the studies you have done?

    On the hiring side, we found that brain-teasers are a complete waste of time. How many golf balls can you fit into an airplane? How many petrol stations in Manhattan? A complete waste of time. They do not predict anything. They serve primarily to make the interviewer feel smart.

    Instead, what works well are structured behavioural interviews, where you have a consistent rubric for how you assess people, rather than having each interviewer just make stuff up.

    Behavioural interviewing also works - where you are not giving someone a hypothetical situation, but you are starting with a question like, "Give me an example of a time when you solved an analytically difficult problem".

    The interesting thing about the behavioural interview is that when you ask somebody to speak about their own experiences, you get two kinds of information. One is you get to see how they actually interacted in a real-world situation, and the valuable "meta" information you get about the candidate is a sense of what they consider to be difficult.

    On the leadership side, we have found that leadership is a more ambiguous and amorphous set of characteristics than the work we did on the attributes of good management, which are more of a checklist and actionable.

    We found that, for leaders, it is important that people know you are consistent and fair in how you think about making decisions and that there is an element of predictability. If a leader is consistent, people on their teams experience tremendous freedom, because then they know that within certain parameters, they can do whatever they want. If your manager is all over the place, you are never going to know what you can do, and you are going to experience it as very restrictive.

    Other examples?

    Twice a year, anybody who has a manager is surveyed on the manager's qualities. We call it an upward feedback survey. We collect data for everyone in the company who is a manager on how well they are doing on anywhere between 12 and 18 different factors. We then share that with the manager, and we track improvement across the whole company.

    Over the last three years, we have significantly improved the quality of people management at Google, measured by how happy people are with their managers.

    We have actually made it harder to be a bad manager. If you go back to somebody and say: "Look, you're an eighth-percentile people manager at Google. This is what people say." They might say: "Well, you know, I'm actually better than that." And then I'll say: "That's how you feel. But these are the facts that people are reporting about how they experience you."

    You do not actually have to do that much more. Because for most people, just knowing that information causes them to change their conduct. One of the applications of Big Data is giving people the facts, and getting them to understand that their own decision-making is not perfect. And that in itself causes them to change their behaviour.

    Other insights from the data you have gathered about Google employees?

    One of the things we have seen from all our data crunching is that GPAs are worthless as a criterion for hiring, and test scores are worthless - no correlation at all except for brand-new college grads, where there is a slight correlation. Google famously used to ask everyone for a transcript and GPAs and test scores, but we do not anymore, unless you are just a few years out of school. We found that they do not predict anything.

    What is interesting is the proportion of people without any college education at Google has increased over time as well. So we have teams where you have 14 per cent of the team made up of people who have never gone to college.

    Can you elaborate a bit more on the lack of correlation?

    After two or three years, your ability to perform at Google is completely unrelated to how you performed when you were in school, because the skills you required in college are very different. You're also fundamentally a different person. You learn and grow, you think about things differently.

    One of my own frustrations when I was in college and grad school is that you knew the professor was looking for a specific answer. You could figure that out, but it is much more interesting to solve problems where there is not an obvious answer. You want people who like figuring out stuff where there is no obvious answer.