[BLANK_AUDIO] So you have heard a lot about designing different questions so far. But what we haven't talked about yet, is how you actually put everything together into a questionnaire. So this is what this unit is all about. Where to place things, when to place them, and how to create a good layout that helps respondents and interviewers to administer the question correctly. The steps, you know, after a little bit of introduction, the steps we're talking about here are screening modules, the main questionnaire, issues with demographic questions, and how you should structure the end of the questionnaire. And then that will be followed by segments on mode and mode selection. Another consideration that you should keep in mind when deciding how to administer your questionnaire. So, the first piece, is, of course, a good introduction. How can you get the respondent started easing into the questionnaire? You can't start your survey with, "Have you ever used cocaine in your life?", right? Obviously that wouldn't work. Respondents, while this is very topical and starting with something topical is good, they need to be eased into the data collection. So, the first thing that you should keep in mind is your introduction should include information on who it is that conducts the research, what the research is about, how long, roughly, the questioning will be, and any issues that will be confidential. IRB boards usually will require that, those are Institutional Review Boards, that respondents know what is coming in a questionnaire. It's also important to tell the respondent that the participation is voluntary. And if you have such thing, it is good to give names of local, national Institutional Review Board representatives. The content of the questionnaire of course will vary by mode of administration. The introduction for an interviewer administered survey has informations that will be provided as part of the doorstep introduction, which means when the interviewer is approaching the house and talking to the respondent, they might explain what the survey is about. Often, those are less scripted introductions because the interviewers are trained to react to the respondents needs as they try to convince them to participate in the survey. There will however be information provided to the interviewer what they should say, in order to address respondents concern. In self-administered surveys all of this needs to be spelled out or be available either in an advanced letter or through a Help-link in a web survey. The content of the introduction also varies by the nature of the survey. In panel surveys only some information is needed in subsequent wave because a lot of that has been given already in the first wave of recruitment. In cross-sectional surveys you usually need to tell the respondent much more than that. So here are the examples of introductions. This is from the National Health Interview Survey in 2012. It's an interviewer administered, cross-sectional survey, and you can see what default text is that's read by the interviewer. "I am..." and then usually the name of the interviewer appear here, "...from the U.S. Census Bureau." This is a face to face survey, so the interviewer can show their identification card and say who is the sponsor of the survey. Often, face to face service are paired up with advance letters, in which case the interviewer can refer to that advance letter and the information that was given there. Here is another introduction from the Panel Study of Income Dynamics, that is a study run here out of the University of Michigan. And this is a telephone survey, at least this segment here. And the respondent is asked if they agree for this interview to be recorded. More and more that is been done, because it allows much better quality control. Many of the issues we talked about before, you know, problems that you can't pick up in cognitive interviews or any kind of pretesting, you can still detect if you monitor or listen to the recordings afterwards. Likewise, you want to mention that the survey is voluntary and confidential. The respondent should always know that they have the chance to either decline to participate in the survey or at least decline to answer certain questions. Here's a third example from a Web survey, a survey on recreation and national parks. And you see here, the same is true. Confidentiality is mentioned, the fact that they don't have to answer if they don't want to, and not uncommon in Web survey, the need for respondents to click "Yes," that they have read that particular statement. Here the first question launches into the survey, but it's still on the same page hoping that the respondent really have read this particular piece of introduction. After the introduction many surveys have screening questions. These screening questions are necessary if you have targeted population surveys. So, let me start with the second bullet point. In targeted population surveys you might not be interested in anybody that you reach in the household but only in, for example, in households with at least one child or of a certain age group, for example, adults aged 35 to 55, if you're planning to study how people save up for retirement before they actually hit possible retirement ages. But even in general population surveys you might have some form of a screening ahead of time because you only want to select one person from the household and not everybody in the household. And that probability of selection is a process that need to be scripted and is part of your questionnaire. We haven't covered it in the questionnaire design piece so far, so let me use this opportunity here, to tell you a little bit more about the difficulties of administering screening questions. This within-household selection is usually done as follows. We have a couple of probability methods that require a full household roster. Most known is the Kish method, another researcher here from the University of Maryland, former researcher, I guess. And it, you know, is usually a sequence of questions. First, the respondent will be asked to name all names and ages of adults in the household. And, you know, sometimes this is done separately for males and females. The individual units are listed, and then Kish' table is used to randomly choose one adult. That Kish table is like a random number table, goes a little bit too far to explain that in detail here, but any good survey methodology book will show you those data collection details. The weakness of such procedure is it's very time consuming, in particular, if this household has many household members, and it's burdensome for both respondent and interviewer. It's also seen or people feel that it's seen as intrusive by the respondent. However, if done correctly, this will give you a known selection probability for the target person. The variants of this that we call quasi-random probability methods. There no household roster is needed. Most popular and done often on telephone surveys is the next or the last birthday method. There the interviewer would usually start the interview by saying, "In order to determine whom to interview, could you tell me, of the people who currently live in your household who are 18 years of age or older, and had the last birthday." So, in particular, a person is selected assuming that you know, the birth order has nothing to do with when the survey takes place and what the respondent would say, and therefore being a less burdensome procedure to the respondent and the interviewer and much less intrusive. But still, given the assumption that birth is random, it will allow a quasi-probability sample. Now, we do know birth is not random and decreasingly less so, but often this assumption is made nevertheless. And then there are some non-probability methods where no household listing is needed. They are mostly tailored to, actually to get cases according to demographic characteristics, sacrificing randomness but making sure that, for example, young males are selected at a decent rate. They're often underrepresented in surveys, and these methods, they do help mitigating that effect. There're also mixed selection methods that are based on household size, and Rizzo's method is one of those. Now screening for household characteristics, not for the selection of a respondent with a certain characteristic in order to get a random person into your survey, but actually because you're only interested in that subset has also been done a lot, or is done a lot. Here's an example from the Longitudinal Survey of Youth in 1997. There's an interesting paper by Horrigan and colleagues from 1999, and you can see the table here, but this graph also shows you these results quite nicely. You can see that of all the age groups down here on the x-axis, the expected number of persons in the survey and the type of population are compared here on the y-axis. This is an old paper, not a great graph, but what you see here in the background are the expected number according to the Current Population Survey. And in the front, in black, are the numbers from the National Longitudinal Survey of Youth. So, let me go back and report a little bit more on the findings here. What is curious, that dip that you see at that age group 12 to 23, is the target age group. And in the advance letter to the survey, it was mentioned that that's the target age group. And, you know, oddly enough, we don't find these people when we look for them. Now you could say, okay, it's always hard to find the young people, and when you screen for them, they don't show up. But, turns out that the Health and Retirement Survey that the University of Michigan runs here has the same problem. So this is not unique to a survey of young people. This can happen in a survey of old people, in a survey with, you know, infants. Whatever you look for seems to be not there. And so, for that reason, I'm going to show you a little research project in the next segment that will show some manipulation to see if we can tease out the effects that lead to this, what we call motivated underreporting.
