Now I'd like to talk to you about Testing Principles and The Quality Process. So, how it all relates. We start whenever we approach something academically from a set of principles about how we're going to approach a topic. So in this case, we're approaching testing. So what are the principles, the foundations if you want to think of it that way, that we're going to use to do effective testing? On top of these Principles, we can build Methods and Techniques. And on top of those Methods and Techniques, we can build Methodologies, so applying the methods effectively. And finally, we can build Tools to automate the process and make it more efficient. So what are the principles of testing analysis that we're going to focus on? First, What, what are we trying to accomplish with testing? This is something called the Quality Process. We have certain quality goals that we want for our software. And we want to be able to achieve those quality goals as cost effectively and as rigorously as we can. Where, is another principle. So what we want to do is we want to focus on those areas of the code where the programmers are likely to make mistakes. It turns out that certain parts of the program and certain modules within a larger system are the ones most likely to contain bugs. And those are the ones where it makes the most sense to spend your effort testing it. When, we want to talk about testing process, when should we test? And how often should we test? And how should we schedule our testing in order to make the most effective use of our testing resources both in terms of time and money? Who, who should perform the testing? Should it be the same people who wrote the code? Should there be separate groups within an organization that perform testing, as opposed to the groups that perform development? How you structure your organization will have a big effect on how effective your testing process will be. And finally, How. When an individual tester is sitting down to write tests for a particular segment of code, what are effective strategies for focusing their attention? So when we look across these five questions, that's a lot to cover. And what we're going to do is we're going to cover some of them in more depth in the testing processes section, so we won't cover them further here. But what I'm going to focus on in this lecture is What, the quality process. And this is again a topic that you can spend an entire course on is figuring out what your quality goals are and mapping them to different processes. So we're just going to give you the briefest of overviews here, just to provide motivation for the testing process. What we want is a set of activities and responsibilities that ensure that our software is acceptably dependable. So depending on what we're building, this acceptably dependable idea may be as simple as it doesn't crash most of the time, to it has to work every single time we use it. And depending on what we're building, we're going to have entirely different constraints for what we consider an acceptable solution. So what it focuses on is given those constraints, how do we figure out a schedule by which we can meet our product deadlines and how can we make our product usable. So this quality process provides a framework for selecting and arranging activities within the development cycle. Testing being one of them, other things being inspections and code reviews, and development time in and etc., etc. And what we want to do is we want to consider the interactions and trade offs between different quality goals. Sometimes different quality goals can be in conflict with one another. For example making a very secure system may make it difficult to make a very usable system, those goals are often in conflict. So what are these Quality Goals? We can break them down along a couple of different axes. The first one is goals about the process. So when we develop software, we want to have certain goals when we develop system after system. First, we want it to be repeatable. We don't want to get lucky once and develop software on time and on budget and then the next time completely fail. So we want to have a process that allows us to repeatedly develop our software. Second, we want one that's timely, that is low cost, that yields low defects. And over time, what we want is continuous improvement. So not only can we develop software as well as we did last time but we get better every time we do it. So we want some level of feedback to the process. Another thing we can look at is the Product Qualities, which are separate from the Process Qualities. So, in terms of Product Qualities, there are two different levels we can describe there as well. We have internal qualities of the product. These are things that we are concerned about as developers for the future maintenance of the system. So, we can talk about, how much reuse can we get from different pieces that we used in the current product? How easy is it to manage the software that we constructed? How maintainable is it? How easy is it to extend or modify? These are things that we're concerned about as developers that are in charge of maintaining this system. But often for users what they're concerned about are external qualities, and those can be grouped into dependability. So basically how often is the software going to fail? And we can break this down into lots of different kinds of dependability measures. Availability, how likely is it to be running. Correctness, how likely is the result that we compute likely to be the correct one. Reliability, Safety and Robustness. We'll talk about these again in another slide pack.