Welcome back. So this is the last section of this lecture and in this section we're going to look at a couple of high profile cases where the results of different types of studies have been contradictory. And we're going to start by looking at a study of vitamin c on coronary heart disease. This is just one example that we've seen of many types of studies that have shown in observational studies that vitamin supplements seem to be protective against disease, whether it's beta carotene against cancer, or selenium or vitamin E. And we go on from those observational studies to do clinical trials and we get much different answers in the clinical trials. So in this case, we see two estimates, one for women and one for men, that indicated people with higher levels of vitamin C were at significantly less risk for cardiovascular disease. But when we went on to do that in a large clinical trial there was no effect of vitamin C supplementation on cardiovascular disease. So seemingly contradictory results. So why is that? Potentially, you know, when we design clinical trials, we focus on higher risk populations because we want to be able to observe outcomes. So we have a powerful enough study. So, maybe in the clinical trial the people were far enough down the road in kind of their latent history of cardiovascular disease that supplementation with a vitamin wasn't going to stop a process that has already started. Maybe we didn't have the right choice of antioxidant regimen. Potentially the trials are too short to see longer term benefits, but, at least in some cases, like the Physician's Health Study was a clinical trial that looked at betacarotene for prevention of cancer, and that was 12 years, and they didn't see any effect. And one important area that we always think about is residual confounding. Do we have healthy users, that the people who take vitamin supplements, or have higher levels of vitamin C, there's so many things that are different about those people than the people who are in the control group and we're not taking care of that. So, some authors, recently took a more careful look of that issue of residual confounding. And so here, let me go through this big table first, this is from the British Women's Heart Study, which was a study of over 4,000 older women, between the ages of 60 and 80. So they took these 4,000 women and broken them up into four groups based on their serum level of vitamin C. So there's four quartiles. Column 1 is the lowest quartile to column 4 with the highest levels of vitamin C. And then they looked at a whole battery of different characteristics. Some, I said these were older women, but even some from their childhood, did they come from a more working class families? Were there bathrooms in there house as children? And some socioeconomic indicators currently about what kind of work they had done and did they have access to a car. They also looked at behavioral and lifestyle risk factors, whether they were smokers, ex-smokers. And they also looked at some biomarkers for how good their nutrition was as a child. What was their adult height and their leg length and the ratio of that? As a marker for, kind of their childhood conditions. And they found for everyone of these indicators that it was associated with their vitamin C status. So, if they came from a working class family as a child, they were more likely to have lower Vitamin C levels than people who came from more professional classes and this was true for every one of these indicators. And it was also independently true. It wasn't like you could pick one of those indicators, throw it into a model, and it would adjust away the socio-economic status. Everyone of these indicators was individually associated. So if you adjusted for the other ones you still saw this effect. So this is a strong argument that residual confounding can be important in these type of studies, especially with dietary interventions. That for some interventions, the residual confounding issue is a big one. And either we have to collect a lot of data to be able to take it into account as Lawlor and his group did and model it or we have to find other ways to deal with it. And there are indeed some investigators that suggest that we should take sort of a human genome approach to residual confounding and that we can collect every bit of data about a person and put it into the model. Now that might be quite expensive and time-consuming, but you should be aware of those when you look at some observational studies. And some points to be made is that you can't assume that these are dependent factors, that if you take some measures of socioeconomic status that you can cover the whole spectrum and, all of the effects, as we saw in the last slide, that there were a lot of independent effects. They may not be linear. In this particular study, they seem to be linear and there may even be interactions between them. So, there may be effect modification having come from a poor, family growing up and what your state is as an adult. So it may, be more complicated. So, that is one potential explanation for a lot of the differences we've seen between observational studies of vitamin and mineral supplementation in the clinical trials. At the turn of the century, which seems like a really odd phrase, the results of the Women's Health Study came out. And this was, again, very controversial. There had been the Nurses' Health Study and other observational studies that had indicated that hormone replacement therapy for women going through menopause or after menopause was protective against coronary artery disease. In the Nurses' Health Study it indicated that the risk of coronary artery disease was 40% less in women taking hormone replacement therapy. And then, 2002 this big clinical trial, the Women's Health Initiative was published with exactly the opposite results. That not only was it not protective, but it might increase the risk for coronary heart disease. So why was this? Is this again a case of residual confounding? And you can imagine that, you know, physicians and patients who decide to go on hormone replacement therapy are making intelligent choices. They're you know, less likely to be smokers, they just have a better risk profile and maybe there's some life course confounding operating and maybe that's why the Nurses' Health Initiative shows a protective effect, whereas the clinical trial shows just the opposite. And, and in fact, few of the studies had adjusted for socioeconomic status, current or historical. And one of the arguments they make in the Nurses' Health Study is that this was a pretty homogeneous group, even though it's a pretty large group of women in the thousands. But they all have a nursing education you know, in a professional class. So the socio economic effects would be less. It could be that you get a better prognosis among adherers, because in the Nurses' Health Study to be in the group that was considered on treatment you had to keep adhering to treatment to stay in that group. And sometimes people who adhere to treatment they, again, sort of residual confounding that reflects on more than just their adherence to treatment. That they just may have more heakthy life styles. And of course, if you're going in to get a regular prescription of a hormone replacement therapy, you possibly could have more contact with the medical care than people who are not. Other things that have been brought up that we'll talk about in more detail in the next few slides, is could there have been some information bias? Was the follow-up and ascertainment of outcomes similar in the Nurses' Health Study, as in the Women's Health Initiative? And finally, whether there's a survival bias, are these, participants really the same? And we are going to talk about those two issues in more detail. So, here are some results of different outcomes from the Nurses' Health Study and the Women's Health Initiative. So you can see that they were in agreement about the risk of breast cancer. That use of hormone replacement therapy, did increase the risk of breast cancer. There was some reduction in both studies, on the risk of colorectal cancer. They agreed on the increased risk for pulmonary embolism and stroke. But the one area they disagreed was in this coronary artery disease or coronary heart disease that here's the protective effect from the Nurses' Health Study versus the Woman's Health Initiative which showed that it put to an increased risk. Why is that? There is a potential for information bias due to how they assertain those outcomes. So if the outcome is non-fatal, MI, a heart attack, or a death due to coronary heart disease, maybe they ascertain those outcomes differently in those two studies. So, the Women's Health Initiative, being a clinical trial, had, you know, set out diagnostic criteria. That included both silent MIs and more symptomatic MI, which the Nurses' Health Study didn't have silent MIs. So in the Women's Health Initiative they were able to detect those silent heart attacks by doing regular EKGs on all of the participants. So they could see if there was evidence of a silent MI or an atypical heart attack, which may be more common in women than in men, and that wouldn't be recognized clinically. And there are estimates that 30% of heart attacks in women are indeed the silent atypical MIs. But indeed, in the Women's Health Initiative where they had the data on this, they only saw that 3% of the outcomes that they classified were the silent MIs. But it's interesting to note that the only other observational study that included this assessment of silent MIs that also saw an increased risk of hormone replacement therapy associated with an increased risk in coronary heart disease was the Framingham study where they looked at silent MIs and says well. Another thing is that in the Nurses' Health Study the people who were looking over the records and defining the cause of death weren't masked to the treatment group, because it's sort of hard to mask the medical records and the death certificates and all the information you would be be reviewing. It's difficult to blind those records completely and so that was not done as securely as it was done in the Women's Health Initiative, which, you know, is a prospective clinical trial that they are recording those data in such a way to maintain the masking. So one of the reasons that there could be a difference in the results of these two studies is just how they counted, the silent MIs and atypical symptoms. So, in the observational study, we have, you know, all the women who were enrolled and some of them have MIs, but some of them were asymptomatic. And those asymptomatic ones were more likely to be missed in the, Nurses Health Study than in the Women's Health Initiative. And even among the symptomatic MI's, the atypical ones may have been less likely to be classified, because this was based on the nurse's report of whether they had an MI or not, and it may not have been recognized clinically as an MI. It may have been attributed to some other cause, these symptoms. So some of those could have not been included in the overall count. And there's a potential that there was some bias that if there was this feeling among the clinicians and even the nurses themselves that HRT was protecting them from heart disease they were less likely to classify those atypical events as being MI, if they were receiving hormone replacement therapy. So in the randomized study there was better ascertainment of asymptomatic and symptomatic MIs because of the procedures of, you know, blinding the participants, having masked assessments of outcomes, having those periodic EKGs. So indeed, this ascertainment bias combined with some misclassification of events and the residual confounding we talked about in terms of the vitamin studies, that could plausibly explain at least some of the differences in the results between the Women's Health Initiative and the Nurse's Health Study. Ascertainment biased didn't appear to effect the other outcomes like breast cancer and stroke, but there weren't strong beliefs about hormone replacement therapy being protective. And certainly the patient's expectation could influence the outcome that if someone has the belief that they're on a treatment that's going to prevent heart disease that has, maybe, some placebo effect. So the conclusion was that whether you're a randomized study or an observational study, you should try to have masked evaluation of outcomes, and you should have an objectively defined outcome that are collected in a routine manner. So, another possible difference between the randomized clinical trials and the observational studies is survival bias. Were they really comparing apples to apples? Because in the observational studies, women came into those studies as current users or they have started using HRT during the study, but did we really account for the fact that some women had been using those for quite a while? So the comparisons were between users versus non-users without regard to how long people had been using the therapy. In the randomized clinical trial, the Women's Health Initiative, for the most part it was initiators versus non-initiators. So it was mostly women who had never taken hormone replacement therapy. And they were randomized to either get a placebo or to get the therapy. So the little bit different starting points if you think about that in those populations. So here are some results that are just from the Women's Health Initiative. And the Women's Health Initiative had a large clinical trial, and it also had an observational study. And so the next few slides I'm going to show you are comparing the results from the clinical trial to the observational study. Which, in the clinical trial, remember you've got new users verses placebo and in the observational study you sort of have a mix of people who been on therapy for a while or may have started. But they saw results similar to the comparison that we just went over with the Nurses' Health Study, where in the clinical trial there was increased risk of coronary heart disease and in the observational studies some indication of a protective effect of HRT against coronary heart disease. And in fact they saw this for other outcomes as well. The same pattern for stroke and venus thrombosis. But they looked at how long the people had been taking the hormones. So here we have all the women enrolled in the clinical trial, and we can see that randomization worked pretty well because about 83% of the women in the placebo group had never used HRT before. Compared to 82% in the group assigned to the hormone replacement therapy. And there was some women, but pretty small numbers and equally distributed in both groups, who has some history of HRT use in the clinical trial, but predominantly we have a clinical trial of new-initiators. In the observational study, you know, these women came to the study and they got classified in the control group versus being in the estrogen group based on what they were taking. So most of the women in the control group were never users. They never used the HRT therapy or if they had used it in the past they had discontinued it. Well, all the women that were classified as users of hormone replacement therapy were indeed users of hormone replacement therapy. And they had, you know, taken it for two to more than five years. And the predominant usage was more than five years. So you can see that these are quite different than the group in the clinical trial, who were all new users. Some of the investigators from the Women's Health Initiative had the wisdom to say, well, let's look at these results stratified by how long people had been users of hormone replacement therapy. So if we look at less than two years in the clinical trial you would indeed see that 80% of the women, there was an increased hazard ratio associated with the therapy compared to placebo. And if we look in the much smaller group in the observational study who had used hormone replacement therapy less than two years versus those who'd never used it there is a suggestion of increased risk in that group associated with hormone replacement therapy. Now very few cases, as we saw the distribution. There were very few women who had used hormone replacement less than two years. And you can see a pattern of decreasing risks with increasing years of usage in both the clinical trial and the observational study. And this is true for the other clinical outcomes such as stroke and venous thrombosis or embolism. So, now these results are starting to agree, when we take into account where the women started at base line. And, in fact, another group of investigators went back to the Nurse's Health Study and did a simalir type of analisys where they controlled for the amount of time women had been using hormone replacement therapy. So, if they looked overall they saw some protective effect in this particular analysis. But if they stratified their analysis to look at the effect of hormone replacement therapy for women who had used it for less than two years versus those, who didn't use it then, they saw an increase in risk in that first two years for coronary heart disease versus when women had used the therapy for more than two years, again, they saw that the risk went down over time. So it sort of, you know, goes back to a very classical epidemiological concept of incidents versus prevalence bias, that you can't compare prevalent and incident cases and think that they are comparable. So in a sense, they were really, in the initial comparison of the results from observational studies and the trials, they weren't really comparing apples to apples. The women were different and, in terms of the extent of their use of a hormone replacement therapy. So some women in the observational study actually had this immortal time where they survived long enough to get into the observational study, and the women who died early on were less likely to get into the study, so they had some immortal time. The women who died didn't enroll in the study, and in the randomized clinical trial, you're comparing instant cohorts. And this was compounded by the risk varying over time, that indeed, there may be some initial higher risk that levels out over time. So you have both of the survival bias and maybe a true pharmacological effect of the risks diminishing over time compounding to make these type of studies look different if you don't control for the amount of time the women had been using the therapy. Other reasons that have been brought up is that there's potentially different formulations, that in the observational studies, they used different market formulations. Whereas in the clinical trial, they used one specific formulation. And also, there could be some missed classification of users in the Nurses' Health Study, because they only classified women every two years based on questionnaires. A woman could start and stop therapy within that two year period and never be classified as a user. So there could be some information bias in how they classified women into the groups there. I think one of the really interesting things about this whole story as it evolved is sort of highlighted in this article from 2006 in the New York Times, that after the results of the Women's Health Initiative came out there was a dramatic decrease in prescriptions for hormone replacement therapy. And that got seen in the population as a big drop in breast cancer, a known risk of hormone replacement therapy. And sort of for me, as a clinical trialist, this is really validation of what we do, to be able to see our results translate and change clinical practice and have important health effects on people's lives, not that I was personally involved in the Women's Health Initiative. But I'll take credit anyway. [LAUGH] One thing I want to talk about, though, with observational studies that they can be very good at, is looking at unintended outcomes of treatment. Because if you have unintended outcomes and unsuspected outcomes of treatment, then you can break that link where the risk of the outcome is somehow related to the prescription. So you don't have that confounding by indication and a lot of times for adverse events, which are hard to detect in clinical trials, because we generally don't have big enough groups, that maybe observational studies is a good place to look for unsuspected adverse events in treatment, because the prognosis for the adverse event probably doesn't influence how the drug gets prescribed. And so we can look over longer follow up and bigger databases to see if a particular drug is associated with differeent adverse events that were just too infrequent to detect in most clinical trials. So, this is a strength of observational studies that I just wanted to recognize. So, in conclusion I've got a couple of slides that just sort of go over the pros and cons of the different types of studies. And for randomized clinical trials, as we've talked about all quarter is, you know, the pro is that they're well-designed experiments, that they have a lot of internal validity, so we can avoid selection bias by randomizing. We don't have to worry as much about residual confounding and we can, you know, ensure that we are comparing comparable groups. We have protocols that reduce information bias, sometimes we use masking, we have standardized assignments of outcomes. We really have a experiment design to be able to detect small to moderate effects and we can you know, eliminate things like the survival bias we saw in some observational studies by controlling the timing of treatment. Some of the problems with clinical trials, is they, aren't always generalizable. So, their external validity isn't always good. It may be a highly select population with a rigid protocol. So how does that information really translate when we're trying to make healthcare decisions for broader populations? Certainly we can't do clinical trials to evaluate things that we think our harmful. We cannot randomize people to smoking or to overeating or things like that. So there has to be some expectation that our intervention is going to have good effects. Often the trials, because they're small and because they're expensive, it's hard to do clinical trials over long periods of time, but people have diseases and deal with diabetes over their entire lifespan. But it's difficult to impossible to have a clinical trial of diabetes regimens that lasts 20 years. But that's really the time frame that we have to think about interventions in. Observational studies have the benefit of less logistical problems. You don't have to convince people to be randomized. We generally don't mass their treatments. So we can sort of place them easier into a regular healthcare environment and if we can get more people involved and more clinicians involved we may have broader populations that are more representative. So the results are more applicable to the general population. And we allow in an observational study, for the tailoring of treatments that really happens. You know, that people's dose gets changed because of adverse events or a new drug gets added. So those are all kind of benefits in a observational study that you might be able to really have a study that mimics how the treatments are used in the population better. But the cons as we talked about are that you could have selection bias, you know, this residual confounding can be real and can really lead you astray. You just don't have that guarantee of randomization and there could be other differences that are systematic between the control group and the experimental group, that are really due to the treatment and are due to bias. And unless we carefully plan to observe outcomes in an objective manner that unmasked assessments of outcomes can cause bias because people know what treatment the patient is on and that may influence how they judge the outcome. And if we do include all of those controls like a standard data collection and standardized assessment of outcome. Well, then the observational study can get almost as expensive as the clinical trial. So, I just want to end by summarizing that indeed, the impetus for putting the randomized clinical trial as the gold standard wasn't really to replace observational studies. It was to replace historical control studies, that that was where they really saw the bigger bias. But we have to remember that perspective observational studies can be good enough, but they need to be designed like a clinical trial. [LAUGH] They have to have, you know, uniform assessment of outcome, by uniform procedures, and when we do get different answers, we need to take a close look at why those answers might be different, and ensure that they're really asking the same question, like we saw in the example of the hormone replacement therapy. The different question was is, people who are continuosly using HRT versus new initiators. We have to recognize that sometimes we get different answers within the same type of study, so there can be heterogeneity even within the answers you get in randomized clinical trials and, you know, we need to take that into account when we're putting together evidence. So we need to rely on the overall evidence, sort of going back to the beginning of the lecture of how we put together all the pieces of evidence, and examine each piece critically to see what weight we should give it. Observational studies certainly give us the opportunity to look for adverse events of treatments unintended outcomes and that would be a good use of larger observational study databases and again all types of studies should be incorporated in the synthesis of information to promulgate guidelines for treatment. And finally I want to come back to Sir Bradford Hill, who was the person we talked about in the randomization lecture, who really introduced the idea of randomization into medical research. And even he recognized that if you have treatments that are a grand slam, you know, you have a uniform mortality outcome if you don't give this treatment. And you give this treatment to one person and they survive, that you might not need to randomize things. You don't need to randomize clinical trial for every medical question. And indeed, you can't have one for every medical question.
