Welcome back. Our first topic as we move into this course on evaluation is evaluation of user interfaces without users. This video lecture will serve as an introduction to the topic. So why are we even talking about evaluation without users. When we talked studying users for research, we talked about how important real people are, how important it is to see them in the context of usage. And that's all true, and in fact, as you'll see, much of this course focuses on how do you study users either by observation or in a lab or through field work and watching their behavior with your system. But our goal of evaluation Is about improving the interface. And if we can find some problems particularly big problems early. Cheaply, without waiting until we have enough of the system or prototype to put users in proud of. We might be able to saves some of our resources for the more expensive evaluation later. So testing on users tends to be expensive. You need infrastructure and planning and to get enough of a prototype that users can give you meaningful feedback. And given that expensive, we really don't want to waste that kind of test, on circumstances where we could find the problem some other way. This is going to be a delicate balance as we'll see. We're going to go through many techniques and one of the things that we're going to see that these techniques focus on tends to be more mid and micro level usability issues. We're going to look at, how hard is it to accomplish something? Where might somebody go wrong? But nothing about the evaluation without users is going to catch us if we made a huge mistake. What if the overall design is just wrong? We came out with the idea that what people need is an app for tracking their dog. And it turns out our assumption about how people interact with their dog are just wrong. Figuring out whether it's a menu structure or a touch button or something else isn't going to solve that problem. And so, we do want to be really careful, about making sure that we have really user feedbacks, and enough stages on the process, including the early ideas that were not spending on up to time over optimizing things, that were going to lead after throughout. So in the end were going to benefit for having, obviously evaluation with users. But also, economical and efficient without user evaluation, and that's what this module's going to focus on. So there are many forms of evaluation. And I'm dividing them only loosely into quantitative and qualitative. The type of quantitative evaluation we're going to talk about, primarily are forms of action and analysis. The next lecture, we'll talk in much more detail about action analysis, but the basic idea of action analysis is that we break down the steps involved in carrying out some tasks with an interface. And look at that break down through some analytic process to learn something about what's good, bad or improvable about that interface. There are a number of different techniques here. We'll hear about a keystroke level model, GOMS which stands for goals, operations, methods and selection rules. There are other formal methods of analysis, and we'll also look at informal analysis, which is the method we'll have you use in this class after we see that it would be unrealistic to try some of the formal methods without a lot more training. We also are going to talk about techniques that are more qualitative. In this lecture, I'll talk briefly about expert evaluation, but then we will have separate lectures and activities on a cognitive walkthrough as one of a form of walkthrough evaluations and the heuristic evaluation, which is a form of, in this case, really checklist driven. Evaluation, the walkthroughs are all models of evaluation where you step through the process of using interface as a mechanism for identifying where the problems might be in that process. Unlike action analysis where you break it down and sort of tear it up what the steps are. In a walkthrough you're using this process of walking through to open your eyes to what's going on and say, in every step along the way, how do things look here, are people thinking the way they should be thinking? Are we presenting what we should be presenting? Are we confusing, what's going on here? Heuristic evaluation comes from a long history that I'll go into a bit later, but starts with the idea that there are good heuristics, rules of thumb. Simple guidelines that in general, work more often than they don't and combined with some judgement in how to apply them, can help you find the kind of problems that might exist in the interface and improve them before you need to go off and do your testing, so we'll look at all of this. So in this module, you're going to learn to conduct three different types of evaluation. But first, I want to say a little bit about the things you're not doing in this module. Starting with expert evaluation. It seems, I guess in some ways, obvious, but there are people who are good, talented and use an interface design. And also people who are talent and use a inner face design critic. Like in any design discipline, critic is a separate skill. The ability to come in, learn enough about a problem to evaluate an interface. And offer suggestions for where that interface might cause problems, where it might be better. I've actually been privileged to work in a number of cases with companies that needed a quick and relatively inexpensive way to get feedback on an interface before they went to much more elaborate development stages. I worked once with a company that did online learning software, or online learning support software. These were online systems to support people in physical classrooms. And we actually spent about four hours one day. With them having me use their system and then sit down with developers, managers and one case the company CEO and say, well, what are our real challenges? Why would you or wouldn't you use this where are you going to run into trouble, and because I had a combination of experience using these types of systems, and experience in UI design and critique. I was able to in half a day point out of four or five fundamental problems with their model for how material was entered into the system, what the instructor view was, what the student view was. And where I thought they could make things better. I've had similar opportunities and seen others do similar things in much more widespread public interfaces. Where I worked with a company that was setting up a sports information website. They did extensive user testing with target users from the right demographics but before we did that they sat down with my and I actually brought in a bunch of students in the UI design course. They said, let us just take a look at this thing, and see if we can point out some things you should be aware of. In this case, they didn't fix what we pointed out, but they made a note to look at that when they did the testing on target users to see if these were real issues. So expert evaluation turns out to be productive, if you do it right. The challenge is finding an expert, who actually has expertise and also knows enough about the domain of your application that you can come in and say, hey, I don't have to start by explaining to you what do the people who walk dogs do with their lives. You are a life long dog walker and a UI design expert, well, you'd be perfect to evaluate my dog information app. There are a number of companies out there around the world that do consultation around expert evaluation. There's also lots of individual designers and people who teach you IDesign, who do this and if you're ever in that situation you will be able to find people who can help you through expert evaluation. The second bit of background and the last thing in this first lecture, it's the idea of early checklists in the pre-Nielson era. Nielson and Molich came up with a form of heuristic evaluation that's become very popular. Before that, there were checklists for decades. Many of these were tied in with contracts and government procurement and standards and the checklist were often of questionably utility they defined mush more did you meet our standard than is the thing usable and only worked to the extent that the checklist actually reflected true usability. In the U.S. for primarily defense systems, there was a book of guidelines known as the known as the minor guidelines after a research center that was involved in their preparation that included an amazing level of detail as to how should error messages be displayed. They should be displayed in an emphasized way on a screen. And these guidelines dated back to the time of mainframe computers, where you couldn't bold, but you could make something bright, or you might be able to make it reverse video. They had all sorts of instructions as to when you do use all caps and when do you use mixed case. All of these were grounded in something useful, but often didn't rise together to the level of understanding very much high level usability. They were much like a style guide would be today and saying, gee, here's how things are supposed to be, if you violate this, it's probably not a good idea because everybody expects it to be this way. But they did not get into the question of, well, what's the user trying to achieve? How easy is it to achieve that? Which would be the next step that we really need and is the first place that we get as we move up to the heuristic evaluations from Jacob Nielsen and from Molich. So this has been an introduction to evaluation without users. In our next lecture, we're going to move into action analysis. See you then.
