[MUSIC] Welcome everyone. This is the first episode of our course on assessment. My name's Gavin Brown. I work at the University of Auckland and my specialty is in educational testing and assessment, and we'd like to share in this course what we've learned about good assessment practices that can be useful in school situations. Our first week is to introduce you to assessment in general, and our first lecture is to look at how societies use assessments and what you believe as a teacher, and as an ex-student, what the purposes of assessment are. In my research, we've proposed that assessment has four fundamental purposes and this diagram illustrates that structure of those relationships. And then, instead of defining assessment by the type of form it takes, like a test or an exam, or the kind of questions that it uses, like multiple choice or open answers, what we do is we look at what's it for - what are people doing with assessment? And the first question to ask about assessment is: is it useful? And if the answer is yes. Then a number of alternative questions arise. In what way is it useful? And my research has suggested that at least here in New Zealand, teachers think that assessment can be useful for summative purposes - that is evaluating students; giving them certificates, giving them grades, writing report cards home, or even summatively to evaluate schools and teachers. We might not like this, but parents make up their minds about is this a good school, are you a good teacher? And they use assessment data to help inform that decision. And much more importantly for educationalists, assessment has a formative role, in that it serves improvement. And that improvement is giving feedback to both teachers: here's what you need to teach better or differently or here's where you've already successfully taught, move on. And it gives feedback to learners: here's what you've done well at, and this is what you need work on. And it can be formative, because it describes accurately and validly what it is that students know and can do. It is possible to some fundamentally say, there is no purpose in assessment. We don't need it. It's irrelevant. We already know what kids need to know, so we don't need an assessment tool, a test, or some formal process because we interact with children on a regular basis. Other times, we don't have much power as classroom teachers and we simply administer tests, because somebody told us to do it, and we put the remarks in a book, and we kind of ignore it. So, there's a sense in which assessment is seen as something bad for students, because you label them as high or low achievers. And it's always got error in it. So, there's a lot of reasons to have doubts about the validity and relevance of assessment. And interestingly, in my research what we've found is that teachers have at different due times and in different circumstances, different levels of agreement with each of these fundamental purposes. Now, these purposes create three logically different categories. Improvement, it helps teachers improve, and it helps students improve. This is the number one reason in school and in education to use assessment. From an external point of view accountability says, "Are we evaluating students? Are we giving the right students the certificates or the excellences or the scholarships that they deserve? And are schools delivering what society wants?" We're fortunate in New Zealand where this is not a high practice, but there are other countries, like the United Kingdom and the United States, where evaluating schools is done regularly with testing. And thirdly, there's the irrelevance: we reject assessment because we don't need it and because it's bad. The problem with teachers and the problem with assessment, and the problem for teachers, is that assessment has legitimacy as an evaluative accountability tool, and as an improvement tool. And this creates real tensions for how you and I carry out assessment. The other thing to note is that, although I've proposed these four fundamental purposes for assessment, our research has shown that not every belief about assessment is universal. Education practices take place in cultures of families and societies, and policy settings that determine how teachers teach, how students learn, and these factors shape the beliefs that people have in any society. And so what we're saying is, how you think about assessment as a teacher matters to how you carry out assessments and your context matters too because you're also part of a system and not just independent of it. To help you see the difference these two diagrams show the results of surveys; one carried out on New Zealand on the left of the screen under number one New Zealand teachers; and in Hong Kong, number two. And each of these four big ideas that I've talked about are plotted in this bulls-eye target. And you can see that in both targets, improvement is well in the yellow, meaning they agree with it. School accountability is in the pink, meaning they kind of disagree with it. Student accountability's on the edge of the green and the pink. Sorry, in the yellow and the pink, meaning we kind of believe in it and irrelevance is definitely in the pink, meaning we disagree with it. But where it's really different between these two diagrams is not where the scores lie but the arrows between them, how thick they are - the thicker the arrow the stronger the agreement. And the biggest diagram difference is the arrow between improvement and student accountability. For New Zealand teachers this is very low, around point two, point two-five correlation. It's a very weak association. But in Hong Kong we've found that this correlation was point nine. Point nine means they're interchangeable, they're almost identical, for Hong Kong teachers that one of the best ways to improve student learning is to test them. Whereas New Zealand teachers only have a small sense that that might be true. So, these two different societies have quite different understandings of how these purposes relate to each other. We went on and we did further research with Hong Kong and China teachers, and what we found is that their beliefs fell into slightly different patterns. And, again, in this large survey of over nearly 2,000 teachers in Hong Kong and Southern China, the correlation between accountability, which means examinations, which meant controlling schools and teachers, was positively correlated with improvement, helping students develop and learn. And yet, improvement was negatively correlated with irrelevance, meaning improvement we believed in, because the opposite of irrelevance is relevance. Whereas accountability they kind of disagreed with, they don't want accountability. So, even though they have a tradition of-- a long standing tradition of exams, they didn't really feel comfortable with using exams to hold schools under control and to determine what schools and teachers do. So, there's some interesting tensions in societies. We've done a number of survey studies using this instrument, and this table shows different countries and groups within each country. In New Zealand, primary and secondary. Hong Kong was a primary and secondary group. Queensland, we had a primary and secondary group. And a group in Cyprus made up of primary and secondary. The important message here is look at the green values in every group except for the Queensland secondary, improvement was their number one belief. Bottom line, teachers believe that assessment should help improve their teaching and student learning, and that's the good news story for policy makers, for school leaders and for school teachers. At the same time, the teachers, practicing teachers doubt the validity and usefulness of assessment to judge school quality. Can you tell if its a good school by testing kids? Well, probably not but it's a really simple idea that a lot of people have. Not every school starts with kids as equally good, not every school makes kids go up as much. So clearly, judging a school by how well the students do on a test is highly problematic to teachers, and technically very difficult in any case. In a study we did with New Zealand teachers, we asked them, what do you think about assessment? And we've got their four scores on the righthand side of the screen. And we asked them what kind of tests or assessments do you use? And we look at the relationship of their beliefs about the purpose of assessment to the form or type of assessment that they said they were using. And you'll notice that two arrows go to the top group: informal, interactive assessment practices. And those two beliefs were assessments for improvement, so they'd like to use informal, interactive processes, like group conversations and group work and student projects. At the same time, if they thought assessment was irrelevant, they also believed in using this same style of assessment. And if you gave them a school accountability test, the third bubble on the right down, that predicted, very small but positive prediction, to deep learning, suggesting if we want to find out if we're a good school look at students deep learning. And deep learning has to do with transformational understanding. Not just remembering and recalling and regurgitating, but actually being able to do analysis and synthesis and criticism. And that's whats teachers say: if you want to hold me accountable, make sure your tests pick up on those deep aspects of learning. And finally, if you're going to hold students accountable, they associated that with formal external things like tests, and quizzes and exams, but they also thought that those things were surface skills - remembering, recalling, gaining facts. So, this is one of the challenges for the testing industry, and for policy makers. Can we make tests that we're going to helps students with? That we're going to help schools with? And can those tests actually test deep learning? Or are they always only going to be memory tests? So, when we think of assessment, one of the things we want you to do at the end of this video is to look at this checklist, and say, when I think of assessment which of these activities do I associate with assessment? And what you might find is that your beliefs have certain similar patterns to what we found in New Zealand. That they form groups of assessments, and you believe in these ones, and these ones you don't. And of course there may be things on our list that happen in your society that we didn't even have. So, use your time to think about, what actually do I mean by assessment? And then you might want to ask yourself, what does my school leader think assessment is? And what does my minister of education think assessment is? What do the school inspectors think assessment is, are we even on the same page? And then finally, we give you on the webpage an opportunity to complete the same questionnaire that we've been showing you results. And there's a self-score answer guide as well, so that you can look at this assessment, read each statement and answer for yourself how much do I believe what this statement is saying. And, we've asked you to score it on an agreement, and it starts from left, being strongly disagree to right, strongly agree. So, the first options are the negative ones, and then it gets increasingly more positive. And this is the score sheet. So, you'll need to stop the video, look at the resources, answer the questionnaire and give yourself a score, but then go back to the video and look at how you did compared to those other groups. Are you strong on improvement? Are you doubtful about school accountability? Does assessment matter for your contexts? Having your own understanding and a clear understanding of where you're at, will help you in moving forward in understanding this course on assessment. Thank you for your time. [MUSIC]
