I'm going to be talking about a more sophisticated technique for creating mathematical models in this screencast. I'm going to be going over nonlinear regression. It's a bit more sophisticated than the linear regression techniques that I've already talked about. You've already learned in some previous screencasts about general linear regression here. We have a single regressor variable, the output y is some function of x and we have the linear combination of different terms. An example of this was a polynomial regression where we had these different functions of the single regressor variable x. So this is an example of a general linear regression model. We then talked about multilinear regression. Instead of having a single regressor variable, we can now have multiple regressor variables. An example of this, we might have to regressor variables, x1 and x2 and we might hypothesize or propose this following model and then we can fit our data to come up with beta naught beta 1, beta 2 and beta 3. So that was all covered in previous screencasts. Now, we're going to be talking about nonlinear regression. And first, I'm going to be talking about this for when we have one regressor variable. Nonlinear regression occurs when our model equation is not linear in the coefficients. Don't be confused, we're not saying that the functions of our independent variable are nonlinear. We're saying that it's nonlinear in the coefficients. In this example, y is equal to beta naught, that term is linear. We add to it beta 1 times the exponent of negative beta 2 times x. So this term is actually nonlinear, because y is not linear in beta 2. Another example that we're going to talk about in this screencast, this is known as a logistic regression model. It's used a lot in finance and machine learning. In this equation theta 1 and theta 2 are fitting parameters. Those are the coefficients similar to beta naught, beta 1 and those are the fitting parameters that we are trying to find. So this is nonlinear. Probability is not linear in theta 1 and theta 2. And finally, for control theory. If you're an engineer, you might learn about control theory at some point. The output might be some sort of gain, which is K here times 1 minus the exponential of negative quantity t plus theta divided by tau. Like beta naught beta 1 and beta 2 for this first example, K, theta and tau are all fitting parameters. So these are examples where we might implement nonlinear regression. The example that I'm going to go through in this screencast is called logistic regression. It's used when the output is a probability of success or failure. Probability can go between zero and one and that probability depends upon a continuous input variable. I've just got a simple example here where we look at the probability that somebody wears a jacket and that's either a one or a zero. One is 100% probability. Zero is they don't wear the jacket. So essentially we go over here and collect a bunch of data. Maybe we have 25 different days. We have the morning temperature in degrees Fahrenheit and we just record or we measure whether or not somebody wears a jacket. And that would be yes, they are wearing a jacket. Zero means no, they do not wear a jacket. And what we're trying to do is we're trying to create a probabilistic model. So a model of equation that's going to allow us to predict with a certain probability whether or not a person is going to wear a jacket, depending upon the temperature for that day. So our goal is to create a mathematical model that's going to predict the probability. I've got a plot here, if we plotted that data up. It looks something like this. The blue dots here represent our experimental data, either somebody wears a jacket or they don't. So our experimental data is either going to be zero or a one. We see that there's this region here roughly around upper 30s to mid-40s where it could go either way. It's not clear cut. And so if we were to put in a model equation, it might look something like this. It's known as a sigmoid curve. This is going to essentially be were creating. We're trying to come up with an equation for the probability as a function of the temperature. If we're really cold for below about 35 degrees Fahrenheit and then we see that we have approximately a 100% probability. If we're above let's say upper 40s, so maybe 50 degrees. We are at a 0% probability that that person will wear a jacket. But in between, we're going to use this logistic regression, the sigmoid curve to allow us to predict the probability. So maybe at 41 degrees, we might get our model to predict that there's a 40% chance of this person wearing a jacket. If it's 38 degrees, we might predict. There's a 65% chance that this person is going to wear a jacket. I mentioned this earlier, but this is you a lot in finance and advertising. It's used in machine learning. So you've got maybe all sorts of input variables going into it and you want to determine the probability that someone will buy a certain product based upon a lot of input regressor variables. So this can be extended to multiple variables and I'll touch on that the very end of this screencast. The logistic regression model I showed earlier, but here it is. It's the inverse. So it's 1 over 1 plus the exponential of this term beta 0 plus beta 1T. Now, this is a general form. The equation that you put in the parenthesis here can really be any linear model, so I could put a quadratic model in there. If I wanted to, I can create any sort of model that I want. The model developer has the liberty to choose whatever model they think will work in there and this is usually a guess and checks. You hypothesize a model. You test it to see how it performs and then you can tweak it such that you get a better model equation. But again, this is going to in the end predict the probability based upon the temperature. Let me show you how we can make nonlinear regression models in Excel. I've got this starter file for you called logistic regression. We have our data over here, the 25 different days temperature and whether or not they wore a jacket. It's important that you guys realize that this is experimental data and we are trying to create the model. The first step in this process is to create guesses for our parameters. The parameters in this logistic model or beta 0 and beta 1, we always start with just some guess. And I've learned from experience that if we go from as we increase the x value and we go from 1 to 0, then that beta 1 term is just going to be a negative parameter. So I've guessed those. I'm now going to go up here to formulas. Create from selection just so I can have those variables, beta naught and beta 1 named. Now what I'm going to do is based upon our independent variable here which is temperature, we're going to predict what our probability is based upon our guesses or estimates of beta naught and beta 1. So I'm going to add in a couple of columns here. Let's add in two columns. Let me move this plot back over here, because we're going to want to see that in a moment and I'm just going to put here predicted. The predicted is just going to be this equation over here. But where we have T, I'm just going to plug in the value in cell C4. So I'm just going to plug that in. So I plugged in that equation there. That's our hypothesize regression model. I can press Enter and then we have a predicted value, and I'm just going to double-click this, and we go down. You see that right now, all of our predicted values are very low. This is where you have to fine-tune your parameters here. And one thing I've learned over time is if we have this just this simple linear model beta naught plus beta 1T, the inflection point of that sigmoid curve basically where kind of the halfway point where the S curve goes from being concave down to concave up. That usually occurs when we put in a beta naught value of approximately the middle in here. So if I wanted to put the inflection point at 40, then I can change beta to 40. And essentially what we're doing there mathematically is in this parenthesis 40 plus negative 1 times a temperature 40, that's going to be 0. E to the 0 is 1 and we have 1 divided by 1 plus one-half. And now you see over here that at 40 degrees, our probabilities is exactly one-half. So this is going to be our predicted. And again, right now we're just basing that on our estimates here. I'm going to show you how we're going to fine-tune this and we're going to use actually the solver tool to allow us to solve this problem. But for now, let's go ahead and we're going to do Ctrl+shift down and I'm going to do Control copy and I'm just going to paste that, Ctrl+V. And you see that, that's our sigmoidal curve and that's our guesstimate for now. So it's really important when you put your predicted on here that you can see it well that it's matching the experimental data fairly. Well, if that doesn't occur, you just need to tweak your beta naught and beta 1. Those are your guesses what I'm going to do now. I want to plot this before we can add a nice smooth line onto here, I need to sort my predicted. So I'm just going to select this entire block of cells, Ctrl+shift down. And real quick, I'm going to go over here to sort and filter, custom sort and I'm going to sort by column E smallest to largest. And now, what that allows me to do is since they're in ascending order. I can go format data series and I can add in a solid line. I'm just going to make this a red line. I'm going to smooth that line. I'm going to turn my markers to none, because I just want to line, no markers. And now this is kind of showing our model equation based upon our experimental data. Now, we have to fine-tune this. We have to fine-tune beta 0 and beta 1. The way we do that is we setup a residuals column. The residual is just the difference between the actual experimental data. So that's our experimentally collected data point is either 0 or 1 minus what the model is predicting and I can do that, and I can just double-click on this to get it down. Now, what we really want to do is we want to minimize the residuals. My guest here for beta naught to something like 50, our residuals change and you see our model over here is going to be a lot less accurate to determine the total error. So I'm just going to put in a cell here called error. What we want to do is you want to square each of our residuals and then sum, this is known as some square root of the error. I can use the sum squared function. I want to square the residuals, because we don't want the negative and positive residuals to cancel each other. You could use the absolute value here. It's more typical to just square the residuals and then some them. So I'm going to select this entire column of residuals. When I press Enter, it's squaring each of the residuals and summing them. So this kind of gives me an overall picture or snapshot of the error. If I change beta naught back to something like 35, you see that our total error goes down to 6.9. You could manually play around with the error here. If I increase it back to 40, I'm decreasing that even more or you can simply go up to the data tab and you can use solver. What we're trying to do is we're trying to minimize the error. We're trying to minimize that by changing the cells that I'm trying to change our beta naught and beta 1, so I can select those two cells. It's really important that you deselect this box make unconstrained variables non-negative. Otherwise, it's not going to work here, because it's going to force those two variables beta naught and beta 1 to be positive which we don't want in this case. And then I'm going to click solve, it's going to churn through and it finds a nice solution to this which minimizes the error there. So this actually is our logistic model and it's going to predict then the probability that this person wears a jacket as a function of the temperature. We can just finish this off by creating a nice spreadsheet here. We have the temperature, maybe we put in 30 degrees and we're going to predict the probability of wearing a jacket and I'm just going to plug in the equation up here. I've plugged in my equation here in terms of beta naught and beta 1. This means that at a temperature of 30 degrees, the probability that this person will wear a jacket is about 94.2% and I can change this to 45 degrees. The probability that this person will wear a jacket at 45 degrees is about 26.7 and we can now use this in a nice predictive model. Let me just wrap up this screencast by showing you how we can extend this to multinonlinear regression. So we have multiple inputs in a logistic model. We might have a mathematical model here. And instead of just having a single predictor or regressor variable we might have for, you might have dozens or even hundreds of these inputs that go into your logistic model. The output is still the probability. So in this example, we might have the probability of defaulting on a loan. Obviously, banks want to minimize that and you might have income, debt to income ratio, recent payment history, credit score and many others that might go into this logistic model just a multilinear model here in the exponent in our logistic model. We could solve this and you could create a nice logistic model that would predict the probability based upon the inputs. And then going forward when somebody applied for a loan, you could kind of put those different inputs into your model and you can predict the probability. So hopefully, you enjoyed learning about nonlinear regression in this screencast. Thanks for watching.
