When we look at these machine learning models, a lot of the clever stuff, it's actually not the models themselves. It's figuring out what data to put into them. I think it changes in the way that we work and particularly changes in the way that so much of what we do is online, this is now more data that goes into these that can enable us to get better estimates of who's going to leave. What are some of these factors that matter? If we're going to try and build a machine learning model to predict who's going to stay and who's going to leave, what are some of the pieces of information that we might put into it? Obviously, as I mentioned, there's some information around demographics, the job role that people are in that is very useful for understanding who's likely to leave and who's likely to stay. The other thing that I think can be very useful is transaction level data. Data on specific events that have happened to people that also shape their probability of leaving. The reason this transaction data on events is useful, is that turnover is really about change. If you think about why people leave organizations is in a sense, the answer is obvious. We usually leave because we think the job we're in is worse than the alternatives for us. Fair enough, so maybe we just have to figure out how bad the job is. The thing that's interesting about it though is, we leave because the job's worse than the alternative. But we also took that job. The fact that we previously took that job in that point it must have been better than the alternatives. We want to ask ourselves, what changed? What has changed for somebody? Now they think their job's worse than the alternatives, but previously they thought it was better. We can use a lot of HR data on how people are being managed to give us insight into that. What might have changed? One thing that can change for people that might make them more likely to leave, it's receiving a signal that they're not really being valued as highly within the organization that they thought they were and then therefore their prospects in future are not so strong. What might some of those signals look like? They might have changes in their performance evaluation. It might be in my performance evaluations are doing okay. Now I'm getting low evaluations. If the evaluations go down, people see they're not a good fit. They're more likely to leave. If I'm not getting raises, not getting bonuses or my bonuses and raises a low. That again is a pretty bad sign for me. I'm doing less well at this organization that I might have expected and so probably I should look to leave. Not getting raises is one thing that's often a red flag for people being expected to leave. Another big one, is being turned own for promotion. People go for promotion and they don't get it, they're much more likely to leave. Indeed, we find more broadly that when people apply to other roles within the organization, often I think you see in a lot of organizations you can apply to jobs internally. If people apply to those jobs and get rejected, again, they're more likely to leave. It could be there is for that maybe that's partly explicitly because it's a rejection. Sometimes it could be because the fact they're applying to jobs inside the organization is a good sign. They're ready to leave their job and if they can't move to another job inside the organization, they're going to move to one outside. Either way, if we can incorporate that information into our models, whether or not they're applying to other jobs. Again, we have better evidence of their flight risk. Another piece of information about change that can predict turnover, is how their environment is changing. We know people are more likely to leave if their colleagues are leaving, often described as turnover contagion. They have all the trouble of learning to work with new colleagues. Maybe those colleagues were friends. It's not as much fun coming into work anymore. May also just be that when those people leave. It gives them a sense of, there are possibilities outside this organization. Maybe I should be doing more with my life. When we see their colleagues leaving, they're at high-risk, certainly when they're manager leaves, particularly if it was a good manager, their manager leaves that can be a big disruption, can lead to turn over. More generally, reorganizations, other organizations so far. When employees are subject to multiple reorganizations that also raises their propensity to leave. We can look through all of the data that we're acquiring in our HR system and look at all of these changes, all of these different flags. To start getting input to our attrition models to help us do a better prediction of when people might leave. A second set of data that we can use to understand whether people might leave is behavioral data. What are they actually doing day-to-day? Again, as more of our lives and our work is online, there's more data we can put into these models. A lot of people, once they start thinking about leaving, they start to withdraw more generally, their performance goes down. Sudden declines and performance productivity, that might suggest that they're at flight risk even more so we can use information about how people are interacting with others to get a sense of whether or not they're likely to stay. We generally expect people who have more ties in the organization are spending more time interacting with others, well they're going to get more support from those others so that makes this an attractive place to be. When we tend to enjoy our relationships. The more socially embedded we are in the organization, the more likely we are to stay. Generally we find in studies that when we look at interactions with the organization, those people who have more contacts and those people are communicating more with them, are more likely to stay. Those people who are more peripheral in organizational networks, those are the people who are more likely to leave. We even see that the quality of contacts matter. For example, the people who stay are not only the people who have more contact, but also connected to those people who themselves have more contact and a high status. There's a lot of interesting evidence that tracking the communication networks that people are part of can give us further information we can put into these attrition models to get a sense of who's likely to stay and who's likely to leave. Understanding the role of interactions in who stays can even go further. There's a nice study by a group of researchers at Stanford and Berkeley that looked at what people said in the email they sent to one another. Their theory was that the extent to which all language in an email with somebody else mirrors their language is a good indicator of the extent to which we're making an effort to adapt to each other, making effort more broadly to adapt to the culture. When people use similar language to another, it suggests they're really trying to fit in. When people don't use similar language it may suggest that they're withdrawing, and they found evidence for this. When they track the similarity, the language people are using in their emails with the people there corresponding to, when that similarity started to go down, when people diverging from the language other people are using, that was also a predictor that those people were leaving. It's interesting as possible, not just looking at who people talk to, but even looking at the language they use can also potentially be an input to a model to try and build an even more accurate understanding of who stays and who leaves. I'm not even done yet. They're other things that we can look at. Some of the most interesting comes from social media. There have be some startups such as hiQ Labs and Chloro analytics that for a number of years have been trying to build businesses around the idea that people social media profiles can tell you a lot about whether they're going to leave and inform this flight risk models. I should think there's a nice logic behind these ideas. All of the things that I've been talking about before have really been different indicators of how much people are attached to the organization and their job. I come to think about these as push factors. How much are they being pushed out by the organization? We also know that when it comes to turnover, pull factors matter as well. How much am I likely to get a good job elsewhere? Those jobs matter whether or not I leave. We also know that most people tend to find jobs through their networks. We find jobs through people we know and online networking has made those searches visible. By looking at people's social media activity, one thing we can really track is, how much are they actually searching for new jobs? If they're searching for new jobs, clearly they're more likely to leave. In a very crude way one thing we can think about it, if people are updating their profile on LinkedIn and trying to connect to a lot of people, that's a pretty good indicator of job search activity and it does suggest this person's flight risk should be that much higher in our model.