Here, it's helpful to compare the problems where we're using machine learning like who's going to leave with the problems where machine learning actually does much better. While the excitement's there about machine learning, there have been a bunch of applications when this is really revolutionized performance. I think probably the best example is speech recognition. Actually, when I was preparing to record these videos, I wrote up a bunch of my notes actually using the speech recognition on my computer. I know what you're thinking, you're thinking, he had notes? But anyway, when I was doing this, I was dictating into the computer and it was frankly almost flawless. It was pretty much as good as if I've been dictating it to a human. I've been playing around with speech recognition software for over 25 years. It never used to be anywhere near that good. It used to make a mistake every 10 words, now it makes a mistake every 100 words. This really is 99 percent plus accuracy. Same machine learning algorithms that Sony was talking about. Why is this so different to an attrition prediction? Well, I think part of it is that, in one case, speech recognition, you're taking something that's already happened. I said a word and then you're trying to classify into categories, which word was it? It's somewhat different to predicting the future. Partly because when you're doing that classification task, you have all the information necessary. When you're doing speech recognition, I say a word and then the computer tries to classify it. Let's say turtle. I don't know why. It sounds like a nice word, say turtle, small amphibious creatures like pizza according to some accounts. When I say turtle, what do we need to classify it? We need the sound file, we need the data of the sound waves I made with my voice. We need also to know which language I'm speaking. In some sense, with longer sentences, often it's useful to have the context for each of the words around it, but actually in this case, it's just one word, turtle. Usually, the computer does a pretty good job of recognizing it. It really has all the data necessary to do so. Now let's think about an attrition prediction. What do we need to make a really accurate attrition prediction? We've discussed a lot of data that would be helpful here. Demographics, knowing how long they've been in the job, knowing what the job is, knowing their performance evaluations, what job applications they're making, posts on social media, those sources. We can put all that in. Do we think that's everything that we need to know to make a fully accurate prediction of whether they'll stay or leave? No, we could think of all sorts of other information that also might influence whether they stay or leave. What else might we want to know; are they going to have a fight with their manager in the next month? Do they have a plan to return to school that they haven't told us about? Will they be offered a better job? We don't have that in our model. Will the better job be better paid, better commute, better for their careers? That would explain it. Even holding all those things constant, other things can matter. For example, might they have to leave for reasons of ill health? We don't have that in our model. Might they decide to leave work to look after a family member? Or maybe relocate because they want to follow their partner or be nearer to the family? What else could lead them to leave? Well, maybe they've always wanted to set up their own company and it turns out that now is the right time to do it. Maybe a friend always wanted to setup their company, they've done it and now this person is going to join them. There are some personality factors there. Even holding all those sorts of things constant, we could imagine that somebody who's easily bored and likes change in their lives is going to be more likely to leave. Somebody who's really risk averse is going to be less likely to leave. Maybe even holding all these things constant. We don't know everything. Maybe they're secretly in love with one of their colleagues. They're unable to consider working anywhere else without them and so they're not going to leave. There are so many different things. Human behavior is complex. On the personality level, there are so many different aspects of their personality that might affect decisions. We're unlikely to have mentioned out all of them. But even beyond that, there's so many things outside the individual that shape their behavior, what they do, what they behaved. You might describe it, it's an open system. Even if I just perfectly understand you, that's not going to be enough. Because I also need to understand your interactions with everybody else in the organization and then I also need to understand your interactions with all of the opportunities outside the organization. There's just so many things that go into whether or not you leave. So many of them, we can't measure and we're never really going to measure. It's not just whether you leave. When we look at some of the other things, so again, hiring, we're trying to predict performance. But all of those external factors also shape your performance. So many things you'd like to know which you never can. Without that data, with all of these different complex factors and interactions that shape human behavior, it becomes very hard to make very strong predictions.
