But that secret sauce isn't a code. It's not just an algorithm. It's actually this organizational know-how that we've acquired over years of managing probably more value generating ML systems than any other company in the world. If we're going to share this organizational know-how, why start with technical ML skills? Well, we want you to become great ML strategists and to do that, we believe that you need to get your hands dirty. You actually need to go up and you need to build some of these systems and learn about them. The good news is that these technical ML skills that you're looking for here are mostly software and data handling skills. Skills you may already be very comfortable with. Talking about these technical skills will also give us an opportunity to leverage Google's experience to help you avoid some of the most common pitfalls. What are some of these common pitfalls? I'm glad you asked. Here is our clickbait hyphen, top 10 pitfalls organizations hit when they first try ML. Here's a list very informally. We've aggregated after several years of talking with new ML practitioners that come to us and they say, we're so excited about this great new thing, it's going to be awesome. Then they might fall into some common pitfalls. We've seen it at Google and we've seen it with our partners as well. The first one, perhaps one of the most common. You thought training your own ML algorithm would be faster than writing the software. Usually, this isn't the case. The reason is that to make a great ML system beyond just the algorithm, you're going to need lots of things around the algorithm, a whole software stack to serve to make sure that it's robust and it's scalable and has great uptime. All of this you're going to have to do for software anyway. But then if you try to use an algorithm, you put in additional complexities around data collection, training and all of that just gets a little bit more complicated. Usually, we really push people to start with something simpler in software only. Next one, one of our favorites. You want to do ML, but you haven't collected the data yet. Full stop, you need the data. There's really no use talking about doing great ML. If you haven't collected great data, we don't have access to great data. Let's say you do have the data you've been logging in for years. It's written on some system that someone in another department controls. But you haven't looked at it, willing to bet that if you haven't looked, that data isn't really ready to use. It goes even beyond that. If there's not someone in your organization who's regularly reviewing that data or generating reports or new insights. If that data isn't generating value already, likely the effort to maintain it isn't being put in and data has this magical way of going stale. Of all the clients we've ever talked to, we've never met one who overestimated the amount of effort it would take to collect clean data. No one has ever said that it was easier than they expected. Expect there to be a lot of pain and friction here. What's the next one? You forgot to put an keep humans in the loop. When we get into these ML systems that start to perform core tasks or core business processes in our organizations, they become really important. Appropriately, organizations become risk averse around these systems because they are the breadwinners of the organization. Then it becomes very important to mitigate this risk. One of the myriad of ways we do that is we keep humans inside the loop. They're reviewing the data, handling cases the ML didn't handle very well and curating its training inputs. We're going to talk about this more later. But this is a feature of every production ML system we know in Google, is that it has humans in the loop. But what about this one? You launched a product whose initial value prop, was it's ML algorithm instead of some other feature. This is a problem because your users probably don't care if what you're giving them is the ML. They just care if it's got that new cool feature or if its recommendations are really good. If you launch something whose initial value prop, is just ML. It also needs new data to operate on. It needs a lot of users to generate that data. It may learn how to interact better. What about you made a great end ML system? It just happens to optimize for the wrong thing. Imagine if Google search was optimizing for, let's say, a user engagement as measured by how often someone clicked on search results. It sounds good. We want our users to like our product. We want our users to stay engaged. But if we optimize for how often they click, maybe then the ML algorithm will learn to serve bad content because it forces users to come back, keep clicking. We always want to be careful about optimizing for something that's pretty good, need not be perfect. But we will always want to look out for perverse incentives. What happens if you forget to measure if your ML algorithm is actually improving things in the real-world? You put it out there, you turn this on. It serves users, but you can't tell how much better it is. You can't tell if there's any uplifting customer engagement or lifetime value. That's always really worrisome. Because then how are you going to go back to your boss or your boss's boss and say, Hey, I want to do this for another product. If you cannot show the impact of the success. Then we've seen a couple of customers with these next ones, you confuse the ease of use and the value out of somebody else's pre-trained ML algorithm with building your own. Google Cloud has a couple of what we call ML APIs. For instance, with vision, you can send it's image and it will perform image classification on some predefined labels. Well, that's great. It's super easy to use. You don't have to worry about any infrastructure or any training data or any data collection, really easy to use. It's a very different ball game than if you wanted to start to build your own. Especially if you want to do your own ML algorithm, but doesn't come pre-canned. It's a lot more effort. We thought after we researched that production ML algorithms were trained only ones. You're like, hey, it's on my laptop. It's doing great on that dataset. I'm basically done. Know, you're probably about 10 percent of the way through. It turns out that if you're going to have an ML algorithm that's going to be part of your core business processes is going to be retrained many, many times and you're going to want to invest the effort to make that process very easy and seamless. The final one is actually the only one of these we have that addresses a confusion about the challenge involved in optimizing the ML algorithm and that's you want to design your own in-house perception, IE, image or speech, or NLP classification, that's natural language processing. These are a peculiar pitfall in the sense that they seem they're much easier than they really are. In fact, all the algorithms we have to address these are very highly tuned from decades of academic research. You should almost always take one off the shelf, already made or already defined. Instead of trying to do your own research is that it's very expensive. There's a lot of pitfalls, a lot of problems. But what's the good news? The good news is, most of the value comes along the way. As you march towards ML, you may not get there and you're still greatly improve everything you're working on. If you do get there ML improves almost everything it touches once you're ready and think about this. If the process to build and use ML is hard for your company, it's likely hard for the other members of your industry. Once you have that ML enabled product or internal process, it's going to provide the users or the consumers of that process great experiences that become very hard to duplicate or catch up to. Because of this beautiful feedback loop, where it's collecting more data and learning all the time. Let's double-click into this idea that value comes along the way. I know it's tempting to try to jump to a fully machine-learning, automated, end-to-end auto magic everything solution. We want to make this leap, but it usually doesn't lead to great products or organizational outcomes. We've seen that in Google, and we've seen that in our partner organizations as well. Let's review a more realistic path and all the great things that come along the way.
