In this lecture we're going to talk about the secret sauce. So Google is going to share the secret sauce with you, but that secret sauce is not code, it's not just an algorithm. It's actually this organizational know-how that we've acquired over years of managing probably more value generating ML systems than any other company in the world. So if we're going to share this organizational know-how, why start with technical ML skills? Well, we want you to become great ML strategists, and to do that, we believe that you need to get your hands dirty. You actually need to go out and you need to build some of these systems and learn about them. And the good news about that, is that these technical ML skills that you're here looking for Coursera, well they're mostly software and data handling skills anyway, they're things you may already be very comfortable with. And as we talk about these technical skills, it also gives us an opportunity to leverage Google's experience, to help you avoid some of these common pitfalls. What are some of these common pitfalls? I'm glad you asked, so here's our kind of clickbaity fun top 10 pitfalls, organizations hit when they first try ML. And here's a list, very informally I've aggregated after several years of talking with new ML practitioners that come to us. And they say we're so excited to this great new thing, it's going to be awesome, and then they might fall into some common pitfalls. I've seen it at Google, and I've seen it with our partners as well. First one, perhaps one of the most common, you thought training your own ML algorithm would be faster than writing the software. Usually this is not the case, and the reason is that to make it great ML system beyond just the algorithm, you're going to need lots of things around the algorithm. Like a whole software stack, to serve, to make sure that it's robust and scalable, and state has great uptime and all of this you're going to have to do for software anyway. But then if you try to use an ML algorithm, you put in additional complexities around data collection, training, all of that just gets a little bit more complicated. So usually we really push people to start with something simpler in software only. Next one one of my favorites, you want to ML, but you haven't collected the data yet, full stop, you need the data. There's really no use talking about doing great ML, if you have not collected great data or you do not have access to great data. And let's say you do have that data, you've been logging it for years, that's written on some system that someone in another department controls, but you haven't looked at it. I'm willing to bet that if you have a look at that data is not really ready to use, and it goes even beyond that. If there's not someone in your organization whose regularly reviewing that data or generating reports or new insights, if that data is not generating value already likely, it's not the effort to maintain, is not being put in. And data has this kind of magical way of going stale. Of all the clients I've ever talked to, I've never met one who overestimated the amount of effort it would take to collect and clean data. No one is ever said that was easier than I expected, you expect there to be a lot of pain and friction here. What's the next one? You forgot to put and keep humans in the loop. So when we get into these ML systems that start to perform core tasks or core business processes in our organizations, they become really important. And appropriately, organizations become risk averse around these systems because they're the breadwinners of the organization. And then it becomes very important to mitigate this risk, and one of the myriad of ways we do that, is we keep humans inside the loop so that they're reviewing the data. Handling cases the ML did not handle very well, and curating its training inputs. And we're going to talk about this more later but this is a feature of every production ML system I know in Google, is that it has humans in the loop. What about this one? You launched a product whose initial value prop, was it's ML algorithm, instead of some other feature. So this is a problem because A, your users probably don't care if what are giving them is ML, they just care if it's got that new cool feature or if its recommendations are really good. And, if you launch something whose initial value prop is just ML, it has no data to operate on, it needs lots of users to generate that data so it may learn how to interact better. What about, you made a great end ML system, it just happens to optimize for the wrong thing. So imagine if Google search was optimizing for, let's say user engagement as measured by how often someone clicked on search results, it sounds good, right? We want our users to like our product, we want our users to stay engaged, but if we optimize for how often they click maybe then the ML algorithm will learn to kind of serve bad content because it forces users to come back, Keep clicking. So we always want to be careful about optimizing for something that's pretty good, need not be perfect, but we will always want to lookout for perverse incentives. So what happens if you forget to measure, if your ML algorithm is actually improving things in the real world? You put it out there, you turned it on, it serves users, but you can't tell how much better it is. You can't tell if there's any uplifting customer engagement or lifetime value. So that's always really worrisome, because then how are you going to go back to your boss or your boss' boss and say, hey I want to do this for another product, if you cannot show the impact or the success. And then I've seen a couple customers to this next one, you confuse the ease of use and the value add, of somebody else's pre-trained ML algorithm, with building your own. So Google Cloud has a couple of what we call ML APIs. For instance with Vision, you can send it an image, and it will perform image classification on it, on some predefined labels. Well, that's great, it's super easy to use, you don't have to worry about any infrastructure, or any training data or any data collection. Very easy to use it as a very different ballgame, than if you want to start to build your own, especially if you want to do your own ML algorithm, that does not kind of come pre-canned, it's a lot more effort. You thought, after research that production ML algorithms, were trained only once you're hey, it's on my laptop, it's doing great on that dataset, I'm basically done. No, you're probably about 10% of the way through, it turns out, that if you're going to have them algorithm, that's going to be part of your core business processes. It's going to be retrained many many times, and you're going to want to invest the effort, to make that process very easy and seamless. And the final one is actually the only one of these I have that addresses a confusion about the challenge involved on optimizing the ML algorithm. And that's you want to design your own in-house perception i.e image or speech, or NLP classification, or that's natural language processing. So these are kind of peculiar pitfall in the sense that they seem much easier than they really are. And in fact all the algorithms we have to address these, are very highly tuned from decades of academic research, and you should almost always take one off the shelf. Already made or already kind of defined, instead of trying to do your own research, it's very expensive, so that's a lot of bad news, that's a lot of pitfalls, that's a lot of problems what's the good news? So the good news is, most of the value comes along the way. As you match towards ML, you may not get there, and you will still greatly improve everything you're working on. And if you do get there, ML improves almost everything it touches, once you're ready. And think about this, if the process to build and use ML is hard for your company, it's likely hard for the other members of your industry, right? And once you have that ML enabled product or internal process, it's going to provide users or the consumers of that process great experiences, that become very hard to duplicate or catch up to. Because of this beautiful feedback loop, where it's collecting more data and learning all the time. So, I would like to kind of double click into this idea that value comes along the way. I know it's tempting to try to jump to a fully machine learned, automated end-to-end auto magic everything solution, we all want to make this leap, but it usually doesn't lead to great product's organizational outcomes. I've seen that in Google, and I've seen that in our partner organizations as well. So what I want to do now, is review a more realistic path, and all the great things that come along.
