- Welcome everybody. I'm Janet Rich-Edwards. I am one of the two
science faculty advisers for the Radcliffe program. John is the other. I cover the life sciences. And I'm an epidemiologist. So for me to be able to have
this day is really a thrill. It also marks the
beginning of sort of a year of epidemiology
and epidemics work that we're going
to be focusing on. So epidemiology
we usually define as the study of the incidence,
prevalence, distribution, and control of disease
in a population. Now you contrast
that with medicine which is about detection,
prevention, and cure of disease in individuals. And so we consider
epidemiology to be the science, the investigative
branch of population health. And when we think about various
levels of biomedical research, you think of-- generally, we talk about
three, maybe four levels. You talk about your basic
or pre-clinical science, which you think of as bench
research, usually experiments, usually with animals
or cells or cell lines. Then you think about
the level that we call clinical research,
which very often involves recruiting patients to a study. That study might be
observational in nature, watching the course of disease
in a group of patients, or it might be
experimental, think of randomized clinical trials
for drugs as clinical research. Then we get to the level
of population research, or epidemiology. And here we're usually talking
about very large collections of data about individuals
within a population. Sometimes we go out
and collect these data by forming what we call cohorts,
an observational cohort. So you may have heard of, say,
the Nurses' Health Study, which has been following 121,000
nurses with questionnaires every two years since 1976. Or you might be talking about
secondary data analysis. You might be
analyzing data that's collected for other purposes,
say vital statistics registries. You might, for
example, go to Norway where they've got fantastic
ability to link all the records through sort of a citizen ID. And you might be
able to, say, link a woman's records of her
births to the records of her hospitalization
or even mortality and make connections
statistically by using these secondary data sources. And there are many different
methods of epidemiology and you'll hear about
many of them today. One of the tricks
about epidemiology and the fact that it's
largely observational is that we very rarely
can perform experiments. And when you think about
what is the gold standard of biomedical
proof, you're really thinking about experiments or
randomized clinical trials, usually placebo
controlled, usually blinded to the investigator
and the participant and the clinicians involved. Epidemiology can rarely have
that kind of experiment. And so we're challenged
to take questions-- to distinguish cause
from correlation. And to give you a
really basic paradigm-- you know, if you think about-- you could do a study looking
at causes of lung cancer. And you could go
out in a population and survey them about their
cigarette smoking habits and ask them about the matches,
whether they carry matches or not. And if you aren't really
careful about your inference, you might conclude that carrying
matches in your pocket causes lung cancer, right? Now, we all know
that's a correlation. Carrying matches is correlated
with smoking cigarettes, which is the actual cause
of lung cancer. That seems so patently obvious,
painfully obvious, if you will. But in epidemiology we're often
dealing with very closely woven together potential
causes of disease that will also be correlated
with non-causes of disease. And our challenge as scientists
is to tease this apart. Because where we
would all like to go is to translation where you
take the science of epidemiology and you translate that into
effective public policy, effective new possibly
clinical protocols. But you want it to be based on
a pretty reasonable inference of cause. And one of the reasons we chose
the four scientists you'll hear about today is
because a lot of them focus on how can we get
better at distinguishing cause from correlation. So I think you'll be hearing
quite a bit about that today. So the four young
scientists we have here I'm really thrilled about. And each of them will talk
for about a half an hour and we're going to
group them into two. The first two are
going to be more, what you might be familiar with,
focused on infectious disease epidemiology. Our roots as a field, when
you think the word epidemics, you think of
infectious disease, you think of cholera outbreaks in
London in the 19th century. We have since evolved. And epidemics has
now writ much larger. So we have, consider, we have
epidemics of heart disease. We certainly have
epidemics of obesity. We have epidemics
of mental health. And so the second
half of the day is focused on some of these
other noninfectious disease epidemics, and particularly
epidemics in mental health issues and how
they're distributed throughout the population in
terms of health disparities. So to kick you off--
so what we'll do is we'll have two speakers
speak back-to-back and then we'll do 20 minutes
of questions after that. And then give you guys
break and then we'll go to the second
half of the program. So I will introduce to you
our first two speakers. So Neal Goldstein is
an infectious disease epidemiologist at Christiana
Health Care in Newark, Delaware, and also hold
a faculty appointment at Drexel University. His specialty is using secondary
data sources, particularly medical records. And his research spans
several disciplines, including vaccine preventable
diseases, sexual minority health, pediatric infectious
disease, and women's health around pregnancy. Most recently, he's
focused on this issue of translational epidemiology. So how do we move from
knowledge generation to application and advocacy. And his talk today is
going to be about infection in hospitals, whether we can
be moving from a model that's based on the
likelihood of disease being a function of the
patient, the characteristics of the patient, to really
change that and look at functions of the system. Our second talk will
be Alana Brennan who is an instructor in the
Department of Global Health and in epidemiology
at Boston University. Her main focus is
to apply bio stats and epidemiological
observation to cohorts in sub-Saharan Africa,
particularly around HIV and antiretroviral therapies. In particular, her work has
been related to changing South African National
treatment guidelines, early versus deferred
pediatric anti-- it's so hard to say--
antiretroviral treatment, and as well as the economic
outcomes of patients receiving that
kind of treatment. She also focuses on when a
clinical trial is not feasible, right? Often for logistical or
often for ethical reasons. How can we do
quasi-experiments which are research designs
in which investigators study quasi-random variation
exposures that occur naturally. And she'll talk
a bit about that. So with that introduction,
Neal and Alana, actually, why don't you both come on up. And Neal, you can take it away. [APPLAUSE] - Thank you for that
very nice introduction. It's really an honor
to be here today. So I want to-- before I get into
the talk itself, I just want to give a
little bit of background as to why I chose
this particular area. As was stated, I did my
training in infectious disease epidemiology. And when I got to doing my
post-doc at this hospital, it was a community hospital. It's a large community hospital,
but it still very much focused on patient care. So the research that
was ongoing there was all about what are the
characteristics of the patients and what leads them to become
infected with some disease. So I wanted to bring more
of my traditional training as an infectious
disease epidemiologist into the hospital environment. And specifically, I worked in
the neonatal intensive care unit. So I'm going to
tell you everything I know about neonatology. It's only in two slides, though. And to start we need to focus
on why do the patients get sick in the environment. Well, one way is we can just
look at the characteristics during pregnancy. So did the mother have some
kind of markers of inflammation like chorioamnionitis? Was she group b strep carrier? And then these can lead
to vertical transmission, so from mom to the baby. There's also, though,
the characteristics of the babies themselves,
in that maybe they were born pre-term. So this would mean
they would have an underdeveloped immune system,
underdeveloped skin system. So these are what I would
call these intrinsic factors. And these are
definitely correlates of infection, in that
if you are pre-term, if you had a mom
that had chorio you are likely to become
colonized and infected. So the questions I
wanted to ask were all about what are other factors
that can lead to infections. So you can-- certainly, it can
be caused by your care team, your health care workers. If they are colonized or
infected with some organism, they can very easily
pass it to you. It also may be a
function, though, of the environment
itself, or even what we call fomites, which are
just kind of inanimate objects that harbor organisms so this
could be a medical device or it could even be as
simple as your cell phone. So when you deal
with when you go through an epidemiology
program, especially when you're in a communicable
disease mode, you will talk about this notion
of the epidemiologic triad. The idea is that if you can
block one of these pathways, you can potentially stop
transmission of some organism. So it's funny that cholera
was mentioned already, because I'm going to use
cholera as the example here. The host would be
people, it would be us. The agent would be
the cholera bacteria. And then the environment could
be where the cholera resides. So the idea is that if we
can block these pathways, we can stop cholera
transmission. For example, between the cholera
bacteria itself and the host you can make the host immune by,
say, giving them vaccinations. You've blocked that
pathway you can also block the pathway between the
agent and the environment, say, by coordinating
the water supply. And lastly, you can block
the environment host pathway by, say, making
sure that people have access to clean, potable water. So we're blocking these
pathways and we're potentially stopping cholera from
infecting the host. There's also this notion of a
vector that kind of operates in between these three areas. The easiest example of that
would be Zika virus or malaria. The vector in this
case is a mosquito. And that it functions,
it operates kind of between all these pathways here. So as we then switch
from this classical, you know, you're
in school, you're getting trained in
epidemiology to now how can we apply
this at the hospital, we can start substituting
things on here. So the host where I worked
it became the neonate. The agent that I'll
talk about are really health-- mainly associated
with health care acquired infections. So I'm going to talk about
methicillin-resistant staph aureus, M-R-S-A or MRSA. Going to talk about
something called sepsis, which is a
bacteria in the blood. We'll talk about a
variety of organisms that can lead to that. But I'm also going to
talk about colonization of a few other organisms,
pseudomonas and klebsiella. So those are our pathogens. The environment now becomes the
neonatal intensive care unit. The intermediary or the
vector is the health care worker or visitor in that they
are within this environment and they will interact
with the babies as well as potentially
the pathogenic organisms. Three research
questions that we were looking at for this
particular group of analyzes. The first is looking outside of
what are the intrinsic factors, so what are the
characteristics of the baby, we wanted to start incorporating
these extrinsic factors. Like, how does occupancy
lead to a greater risk for becoming septic? And I'll get into what
that means in a few slides. But also, how does the
patient care network or who the providers
are interacting with, how does that influence
possible spread of MRSA. And lastly, if we
can identify maybe some of these
other factors, what is it about the
environment itself, what is it about the
NICU, the bed space, the location of
the patient, that may have some correlation to
becoming colonised or infected. And I just want to-- I have one slide here that's
kind of my methods-y slide. Because I want to just give
you a sense for how we're able to do this type of study. It's not as if we
are prospectively recruiting parents to recruit
their children into this study. These data already
exist, which is really-- that's kind of one
of my wheelhouses is I like doing secondary
analysis of data that's already in the
electronic medical record. I feel like we have
so much information that already exists
that was delivered during routine patient care, we
can mine that and then ask some of these interesting questions. So what we have in the
electronic medical record, it can be thought of as a
cohort of neonates in that they are admitted to the neonatal
intensive care unit, and while they're in the NICU
we know what happens to them. We can follow them through
time and see did they develop one of these
outcomes of interest or did they have some other-- did they not have the outcome. And that would be were they
discharged from the unit, were they transfer
somewhere else, or did they expire while
they were in the unit. So these three research
questions, now, we'll go through them one at a time. I'm putting the epidemiologic
triad back up here on the slide so you can get a sense for
which are the pathways-- and I'm not sure if
it shows up that well, but they're highlighted in red. So the intrinsic factors that
we would traditionally focus on are really what's happening
between the pathogen and the neonate. But this talk is all about
these extrinsic factors. So that's these other
pathways that we're beginning to look at. So the first question
that we're tackling is this notion of occupancy. And the reason that
this question came about is because it's
almost in some way an easy way to explain if
there's an outbreak of, say, MRSA in the unit we
could just be like, oh, it was really
crowded at that time. You know, maybe that's
why there's some breakdown in hygiene going on. Maybe that's what's
going on here. So we wanted to say, all
right, is it really related to occupancy and what
does one actually mean by occupancy,
because it can be defined a whole host of ways. For example, it could
just be the number of babies in the
unit at any one time, and this is what we
would call this census. Or could be something
to do with the demands on the staff in the unit. And a proxy for this would be
what's the overall acuity, say, what is the proportion of
pre-term babies in the NICU or what's the proportion
of very low birth weight babies in NICU. Or it could be a function of,
say, just staffing levels. What's the nurse
to patient ratio. And one thing I want
to emphasize with this is that it's not
occupancy itself that leads to these outcomes
of being infected or colonized, it's this notion of being
a proxy for something else. Because maybe it's a breakdown
in sterile technique to access lines that's happening
as a result of increased demands on the unit. Well, getting right into
the results for this one. We looked at two measures, here. We looked at the notion
of just the number of occupied beds or the census,
we looked at the acuity. And what we saw was that
there was a modest risk increase in late onset
sepsis that was statistically associated with acuity,
not so much for census. And although this is,
again, a very modest effect, the way that I initially
did this analysis was it's kind of like for
each percentage point increase in acuity in the unit-- so what I would
stress for this one is that if we're looking at,
say, the proportion of very low birth weight babies and
we were comparing a unit that had like 10% very low birth
weight babies to a unit that had 40% very low birth
weight babies, now all of a sudden we're talking about
a 60% association of late onset sepsis. So it becomes a little bit
more important of a predictor. Conclusions from this question-- we saw that in times
of high unit acuity, we may need some additional
infection control precautions. As well as if you're
doing this type of work where you're looking at health
care associated infections, it's probably a good idea to
have some measure of occupancy in your analysis. All right, so switching
to what I think are the more
interesting ones, here. The second research
question is how does the patient care
network of a NICU influence transmission of a MRSA. Again, the triad's up there. You can see highlighted
in red which pathways we're operating on. And the outcome I'm interested
for this study is MRSA. And MRSA is a not uncommon
health care associated infection and the
consequences are particularly dire in neonates
because about a third of the ones that become
colonized with MRSA will go on to develop
invasive infection. So it's important
pathogen for us to target in the
hospital environment. And what we know is that about
20% to 40% of a patient's flora are the result of
cross-contamination via the health worker. They're this vector,
they're this intermediary that's going between patients. And one of the main correlates
of this transmission is proper hand hygiene. In fact, hand
hygiene is in itself composed of a couple
of different things. There could be the
notion of compliance. So that just how
many of your staff are doing hand hygiene, whether
it's the gel hand sanitizer, whether it's using sinks. But then you also have this idea
that you could be compliant, but are you doing it correctly. And that's this
notion of efficacy. Because it's comprised
of these two things I don't think you could
ever get 100% effectiveness. Because even if everyone
in your unit is doing it, we don't know that they
are doing it correctly. So to do this kind
of work we wanted to know how's the notion of
which providers that interact with a baby connect
infants together and then how could
this, perhaps, influence the spread of MRSA. So we needed to ascertain what
I was calling the patient care network. The idea is that I can go into
the electronic health record and I can see which providers
interacted with which babies. And then I can begin to link
providers and babies together by saying they took
care of this baby and then they went
on to take care of this baby and this baby. So you can develop these
grafts and we'll see one in a few minutes. But you have this
network that forms. And it's akin to something you
may have heard something called a social network
analysis and it's all about your
social networks are and how that can spread ideas. It's a similar
sort of idea, here. To give you a sense,
though, for what the type of interactions
we're looking at in the health record, as you can imagine, when
you're in an ICU environment, that's the most
intensive care that you would have in the hospital. So there's all sorts
of interactions that are happening. Some of those
interactions are kind of meaningless for our work
here in that they're just patient documentation
sort of things, those really don't contain any
risk of spreading a disease. Other ones, though,
they are more meaningful in that they have the
propensity to spread MRSA here. So we ranked the ones
in the medical record and I thought that
it would really be the more invasive
procedures are the ones that we would see here. It would be the chest tube
placement, central line, that kind of stuff. But in fact, the
majority of interactions that we see in our NICU are
just really routine things like taking vital signs, since
this is a population of babies, doing like a stool assessment. Nevertheless, this
still has a propensity to spread an organism
from one baby to another. And so this picture up here
that looks like a bird's nest, this is actually our
NICU on a given day. So what you have is
each red dot is a baby. And then the black lines are
connections between babies by health care
providers in common. When I first saw
this, you know, it suggests that, man, there
really is a lot going on a given day in the unit. But each one of these lines is
a potential for hand hygiene to break down, whether
it's not being compliant or it's not efficacious because
it wasn't done correctly. So we conducted this
simulation study here to model what this NICU network
looks like over a typical nine day admission. That that's the median
length of stay in our NICU. So I'm going to switch over,
see if I can play a video. All right, so what
we're looking at here is each dot represents
a baby in the NICU. The blue dots are babies
that are susceptible to MRSA colonization, the red dots
are babies that are already colonized with MRSA. The lines that you're seeing
drawing and as they update, these are interactions
that providers are having that link babies together. It's updated every hour. So the idea is that you have
these one hour care bundles. I look at everything that's
happened with a given provider for one hour
and all the babies they've interacted with. And then I begin to connect
these babies together. So now we can do
this simulation model where we follow over
time these interactions. And what I do here
is I allow for there to be the possibility
of MRSA transmission that's considered on, say,
hand hygiene effectiveness or some characteristic
of the baby. I don't know if you noticed,
but we started with two red dots up there. That meant there were two
MRSA-colonized babies. And here we are 80 hours
or so into this NICU and we're already at four
colonized babies here. So if I let this go
through the whole way, you end up with about five
MRSA-colonized infants. So it clearly demonstrates that
there is a lot of opportunity for MRSA transmission to occur. So this suggests to us that
there's a lot of opportunities for MRSA transmission to occur. And now that we've had the
environment constructed where we can
mathematically model it, we can begin asking questions
such as when we vary the amount of MRSA that's in
the unit at any given time, how does that affect
the spread of it, we can vary the hand
hygiene compliance, we can very care
attributes of the baby. So we know that infants that are
born pre-term or infants whose trachea is intubated
require additional care. And on the basis
of this, we came up with these graphs
here the y-axis we're looking at
the post-simulation, so that means after nine
days in our simulated NICU, we're looking at the prevalence
of MRSA colonization. And then the group
along the bottom, these four groups that I have
here, one two three four, this is just kind of seeding our
NICU with starting prevalence of MRSA, so this would be one
baby out of the total amount, or about 2% of the babies
have MRSA, and so on. The gray bars then
represent the varying amount of hand hygiene effectiveness. The two extremes, the
white bar and then the black bar, white bar would
be no hand hygiene at all. So this is just assuming
that no one follows any hand hygiene at all, it gives
us a usable comparison. The black bar represents 100%
where everyone's doing it and everyone's
doing it correctly. So we acknowledge
that reality exists somewhere in between here. When I first started
with this analysis I had a bet going
with our ID physician. I thought there would
be a plateauing effect. I really thought there would be
this point of no return, where we just didn't see
any improvement. And he said that,
no, I think you are going to see a linear trend. You're going to keep
getting bang for the buck. I think in some way, the
jury's still out on that one. I kind of see that-- I agree with him that
there is this linear, you keep getting return
for your money here. But there does seem
to be a little bit of plateauing that occurs, in
my opinion, towards the bottom here. But if we focus just
on this one over here, on the one that has 2%
starting prevalence, I don't have the
numbers in front of me. I just wanted to talk about
the overall picture here. But the number-- after
you follow this NICU, at 2% prevalence, even
with 88% hand hygiene effectiveness, which I think
is kind of a theoretic maximum, that still suggests
that post simulation you have about two, and
in fact, a little bit more than two babies
colonized with MRSA. In my opinion, then, that really
suggests that hand hygiene alone is not enough. There's never a
good enough level of hand hygiene in the
hospital environment. So that's one of our conclusions
from this work is that there's no good enough level. And the conclusion
number two in some sense that may be a bit
obvious or intuitive. The greatest risk
of MRSA colonization was for those who have
the most contacts. The neat thing about this
work is we're actually able to quantify what that is. And then to point
number three on here, in times of a high number of
initially colonized babies with MRSA, it really
warrants other interventions in then NICU. And these would
be things like we have isolation
rooms in our NICU, so placing the babies
in the isolation rooms. This is also something
known as cohorting. Or when you have
multiple babies with MRSA we like to group them together
in a certain area of the NICU. It could also just
be things like, should we consider
decolonizing them, should we do universal gloving
in addition to hand hygiene. It's all food for thought
for our infection control department. So the final research
question here is, what are the characteristics
of the NICU itself? We know that occupancy
seems to matter, we know that hand
hygiene is important. But what is it about the
environment of the NICU that may lead babies to becoming
colonized and infected with various organisms? All right, so a brief
recap of where we've been. We saw-- and I just mentioned
this-- that acuity of the unit seemed to be an
important predictor. The amount of care
required seemed to be an important predictor. And that's a picture
of our NICU up there. So we want to know, what is
it about the NICU itself? In fact, this is nothing new. And I'm returning to
cholera because it's such an instructional example. How many people have
heard of John Snow? And not the Game of
Thrones Jon Snow, but the-- All right, so it's about
half, maybe a little bit more than half. In some sense, he's
one of the originators of this field of epidemiology. And you have to
put yourself-- now for this picture
to make sense, you have to put yourself back
into mid 1800s London, there's a cholera outbreak. And what John Snow did that
was so innovative at the time, and, again, may seem kind
of obvious to us today, was he just put the
cases on a map of London. And those are these
dark rectangles up here, the black rectangles. So he's putting these
places on a map and notices that they really
appear to be centered around one of these wells
in London on Broad Street. And lo and behold, when
they turned off access, when they locked the well
so people can no longer draw their water from it, this
particular outbreak subsided. This is what we would call
today in the field spatial epidemiology. And also is something we
often say in the field, in public health in
particular, that place matters. So I wanted to know, does
place matter in our NICU? We're looking at a
bunch of outcomes now, we're looking at
MRSA but we're also looking at sepsis again,
as well as being colonized among those babies that
are being ventilated, are they colonized
with certain organisms that have the potential to
cause invasive infection. And we're going to consider a
few different characteristics of our unit. Our NICU is what's known
as an open pod design. So you have these pods
and within a given pod you can have up to three
babies, occasionally four babies that
share this space depending on the
demands of the baby. But some of the areas in
the NICU you can only fit-- some of the pods you can
only get a single incubator in there, single bed. Other ones you can
get up to four. So does that seem to matter? If you're in an area with
there only being a single bed, are you less likely,
say, to become colonized? As well as-- we're
looking at historic data here from our electronic
medical record. So at one point in our unit
we had a leaking ceiling tile and it turned out that the
water supply was contaminated with pseudomonas and it was
leaking near one of the pods. So we said, well, can
we use this as a marker, then, to see did more
kids that were in the pod during this period of
time get pseudomonas. Also at one point, and this
was really bizarre to me, but this predated when
I was at Christiana. In one of the rooms-- the NICU is divided up
in these three rooms. One of the rooms had carpet. It's like, why would you
guys have carpet in an ICU? Because you have environmental
services that comes through, right, and they're
vacuuming it and it's kicking up all sorts of dust
and particles and organisms, as well, into the environment. But we're able to look
at that back in time and say, all right, if you were
in the room when it had carpet, did that cause an increased risk
of colonization or infection? And, as well, we threw in a
couple of markers of fomites. There is equipment--
there's spaces in the NICU where you can
fit more equipment for babies that require more care. There's also refrigerators that
store breast milk and medicine sprinkled throughout the unit. And one of the
neonatologists was asking me, you know, that
she thought it was related to the refrigerators. She thought, I think
those refrigerators have a contaminated-- some part of them
is contaminated. Can we somehow model to see
if being near a refrigerator leads to an increased
risk for infection? And one final thing I
wanted to consider here is that, since we're doing
this spatial analysis, say I'm in a pod and a baby in a
neighboring pod becomes septic. They have some
kind of bacteremia. Does that then
influence my risk? Does proximity to other babies
influence a given baby's risk? So to do this work
first thing we needed was a spatial
representation of our NICU. Now it's not like if I
was doing work in Boston, it's pretty easy to find
a map of Boston to them plot cases of whatever. But there wasn't
this map of the NICU so we had to create
a map of the NICU. This is what it looks like. It's a u-shaped NICU. And one of the easiest
things I can do is create what's known
as a choropleth map. And you see these
all the time, right? If you look at like diabetes,
obesity, heart disease in the US, you see
these colored maps. And so it's very nice to
get a quick qualitative sense as to what's going on. So I picked one of
our outcomes here. The way that you
would read this map is the darker the shade
of red, the more numbers of these outcomes that occurred
within one of these pods. So I want to do-- this is the point
in the talk where it becomes a little interactive. I want to do a quick survey. There's two options here. One is that you think,
just by looking at this, it looks like it's
just randomness. The other option
is that you think there are patterns to
what's going on here. In other words, there's some
spatial correlation going on. All right, so show
of hands, then. Who thinks this is
just randomness? This is random noise, what
we're looking at up here. One take? All right, I've got
a couple of takers. Looks like two or three. And then raise your
hand if you think that there's clear spatial-- like, there's patterns,
here, there's hot spots. So the majority think
there are patterns. I fell into the camp where
I, when I first saw this, was like, there's clearly
patterns going on. If you look at the
bottom area here, there's definitely
something going on here. There's definitely
something going on here. So we're going to return to
that question in a minute. For this final
research question, we had three sub questions
that we wanted to investigate. First, was it random
or are there patterns, are there hot spots? Second, what about the
environment itself? Does this explain our
infections and colonizations? As well as does, you know, if a
neighboring baby becomes sick, does that influence
my particular risk? So to the first question,
was the distribution random? We do some fancy
statistics and what we end up with is that for
two of our outcomes, MRSA and sepsis, there
were patterns to it. It wasn't just randomness. For the other two
of our outcomes, though, it statistically
appeared to be random. So I got bad news in that the
outcome we were looking at, which was klebsiella, in fact
was statistically random. So that just goes to
show you can't always-- a qualitative
interpretation sometimes doesn't lead you down the road
to the correct conclusion. All right, now to the
environment itself. On an earlier slide I brought
up the list of factors that we looked at. So this was things like does
being in a refrigerator matter? Does being in what we call
the carpeted quiet room, did that matter? So we looked at all
these factors, right? And what we found
was that none of them really mattered all that
much, except for pseudomonas. So now we're dealing
with being colonized with this bacteria in
your endotracheal tube, this is among things
that were intubated. And we saw that for them,
being in an area that required more medical equipment
seemed to suggest like a nearly two-fold or two and a 1/2
forward increase odds of being colonized with this organism. So that was of interest. And these models that
we're doing right here, they're controlled for all
of these characteristics of the babies themselves. So we're accounting for the
fact that some of these babies are going to have been born very
pre-term, have very low birth weight, may have had a
central line present. As well as how long that
they're intubated for, right? Because it stands to reason that
the longer you're intubated, the more likely your
endotracheal tube will pick up some kind of bug, acknowledging
that all the tubes start out sterile. And finally this notion of, does
a neighbor influence your risk? Here, again, doing
some fancy statistics. But what we ultimately find
is that MRSA colonization appear to be the only one where
if there was a neighbor that was MRSA-colonized, that
I was at increased risk for perhaps being
MRSA-colonized. When I first saw this result
I got very excited by it. But then I thought that-- well, this is actually probably
just returning something that's already existing in our unit. In that MRSA-colonized
babies are already being placed in
the isolation rooms and they're already
being cohorted to certain areas in the NICU,
so it's probably actually just returning that. It's just telling me
that, yeah, there's this spatial auto-correlation
but you're the one causing it. It's not that MRSA
is more communicable than other organisms. So something I thought was going
to be more interesting, maybe not. We still have to talk about that
with the infection prevention group. The conclusions, though,
from this final research question here. What we saw was that the
intrinsic baby-specific factors-- so, again, these are
things like your birth weight, your gestation, did
you have a central line-- those appear to explain
most of the outcomes. But we still see
statistical evidence that there's extrinsic
factors at play here. We identified maybe
a few of them. I still think there's
more to be done here, that there's more factors
we just haven't uncovered. Because with this
type of study, where you're working with data
that's already existing in the medical
record, you kind of can only test for
what you have data on. So if we were designing a
prospective study from this, we could really
investigate a lot more. Maybe there's other
factors out there we just haven't uncovered. But definitely with MRSA
colonization, you know, it warrants isolation
and cohorting given the potential for transmission. When I give this
talk at our hospital I like to conclude with, so
what should we do as the NICU? Like, what should our
infection prevention group do? But I'm guessing
that none of you are with the Christiana
Care NICU and if you are, see me in advance because
that's a tremendous coincidence that you're here right now. So I thought that
would be nice to end it with what can we do as potential
patients in the health care system? Really one of the
easiest things to do is the notion of
hand hygiene, right? It's making sure
that your providers have done hand hygiene when
you're seeing with them. You're not going to insult
them by they come into the room and you don't see
them scrub down, you're not going to
insult them by saying, can you please use a gel
sanitizer while you're here. As well as you should
do that yourself when you're in a
hospital environment. They're gross places to be
because there's organisms all over. No matter how vigilant
environmental services can be in the hospital,
there's still more organisms there than you
would just encounter walking around elsewhere. But that's another thing that
you can do in addition to hand hygiene, ensuring that your
staff have hand hygiene, they follow those
practices, making sure the environment is clean. Like, you should be
able to look around and it should be clean
where you are in there. If it's not clean,
talk to someone to get environmental
services in there to go ahead and clean the area. Well, there's other things
that we can do as well. Like ensuring that we prescribe
antibiotics correctly. Because kind of what are the
implications behind this work is that if you are
over-prescribing an antibiotic, you can develop what's
known as resistance to it. And particularly with
MRSA, right, that's already resistant
to the methicillin class of antibiotics. So we see that by
being more vigilant with antibiotic prescribing,
you can potentially reduce the burden
that's caused by these. So that would just
be, you know, if you feel like you have
the cold, there's no need for an
antibiotic, right? I think most people know that. But if you're a senior
provider and they give you prescription for an
antibiotic, know the reason why they're giving
that antibiotic, make sure that it's the
appropriate class for what you have. These are all good
questions that they would be happy to talk to you with. So in closing, clearly,
work of this magnitude, I couldn't do without
a whole host of people that are much smarter
than myself, many of them are listed on this
slide over here. And I also want to acknowledge
the Society for Epidemiologic Research because
they were kind enough to select a portion of this
work for one of their awards. Thank you for listening. Here's my contact information. I'm happy to chat
about any of this with you now or after the fact. [APPLAUSE] - All right so I'm very
excited to be here today. My name, again, is
Alana Brennan and I'll be talking about the use of
quasi-experimental designs for evaluating HIV care and
treatment in sub-Saharan Africa and observational work. Specifically about
two analyses I've done over the last four years. I'll give you a brief context
of HIV in sub-Saharan Africa and then also South Africa. Talk about observational
versus experimental studies. And then talk about the two
specific quasi-experimental studies that we did using
propensity score matching and regression discontinuity
to evaluate HIV care and treatment. And this is all
secondary data analyses. So the HIV epidemic--
many people are probably very familiar of what's
going on worldwide-- but in case you're not, there's
roughly 37 million people who are HIV positive worldwide. And the vast majority of them
are in sub-Saharan Africa where there's 26 million
who are HIV positive. And just to give you an idea
of how treatment has rolled out in this setting,
treatment wasn't available in resource-limited
settings until about 2004. And this is based on a
directive by the World Health Organization and
guidelines that we're bringing antiretroviral therapy
into government sector clinics. And so in 2004 treatment
became available. And it was based on certain
eligibility and guidelines because of resource and
financial constraints. Over time, in 2010, as
better drugs became available and the eligibility
criteria changed, more people became
eligible for treatment. 2013 there was another increase
in eligibility or expansion of eligibility criteria. And then currently we're at
what we call universal test and treat or treat all. And that means that
anybody who is HIV positive and comes into a
clinic, they are eligible for treatment that day. And this, even though the
policy was started in 2015, most countries didn't
roll it out until 2016. September of this last year. And universal test
and treat is what it is, as far as I understand,
what is protocol in high income countries since roughly 2004. And then what has happened
also over this time period is we've gone from this
huge pill burden of patients having to take two to
three handfuls of drugs twice a day, two,
three times a day, to what one is a fixed
dose combination. So it's one drug, twice a day. So it's actually improved
patient care substantially. And patients are--
this is easy to do. Easier to do, to adhere to. And this graph is
just showing where we're at in different regions
throughout the world in regards to antiretroviral therapy
coverage for those that are eligible for HIV. And I'm just highlighting here
in the red box southern Africa and eastern Africa
regions because those are the regions that the
data is from that I'll talk about today. And if we look at--
this is 2010 to 2015. And in 2010 in the
southern African and east African region, we
see that about 25% of people who are
eligible for treatment actually got treatment for HIV. We see a substantial increase
to 2015 where about 55% of the population
that is eligible is actually accessing
antiretroviral therapy. And although there's
still a large unmet need, this jump is massive. A massive undertaking. So although there is about
50%, roughly, at least in eastern and southern Africa,
that are in need of treatment, this is not a small feat
to get to this point. So briefly about
observational studies and experimental studies. And Janet had talked about this. Most of the work that
I do, until recently, is in the world of observational
analytical research. And so our typical
prospective cohort design is where you have a
population of people, you choose a
subsample, you classify them as exposed or unexposed. And then you follow
them forward over time to assess who develops disease
or your desired outcome. And then you compare the
risk or rates in the exposed to the unexposed to look at the
magnitude of an association. And as Janet had
mentioned, basically, we're only looking at what we
believe to be correlations or associations causality
is difficult to establish when you're in
observational research. Where we can do that is in
the experimental study design, specifically the
randomized clinical trial, which is our gold standard. So you have your population,
your underlying population, you choose a
subsample, and then you randomize that sample to either
received treatment or control. And then again, follow
them forward over time to assess your desired outcomes
and to compare the risk or rates amongst the
treated to the control to look at the
actual causal effect. So if you're randomisation-- the
key here is that randomisation. If you're
randomisation held, you can actually establish causality
in a randomized clinical trial, which is what we're
always trying to go for. So in my mentorship
for the last five years with Professor Matthew
Fox, he's always challenged us to
think about when we're working
observational research, how can we get as close
to a clinical trial design as possible with
secondary data analyses. Because we know in
experimental studies we can balance confounders
in measured and unmeasured because of the
randomization, if it holds, but that's not a luxury
in observational research. So quasi-experimental
designs, the two that I'll talk about
today, can help us, if the assumptions
are met and they're applied appropriately and
the question is appropriate, help us get closer to that
establishing causality. So the first analysis
that I'll talk about is looking at outcomes of stable
HIV positive patients that have been down-referred from a
doctor-managed antiretroviral clinic to nurse-managed
primary health care clinic for monitoring and treatment
of HIV in South Africa. And this is a
prospective cohort study where we use propensity
score matching to balance confounding between
exposed and the unexposed. So we're trying to
simulate clinical trial. So here we see that the context
of this is in South Africa. And South Africa, just to
give you a bit of background, is the largest HIV
epidemic in the world. About seven million people
are HIV positive, about 50% of those are actually
receiving care for HIV. So again, there's a huge unmet
need, but that 50% is not a small feat, by any means. In the clinic that
the data is from and where I've worked
at for about four years a while back is
Themba Lethu Clinic, which is associated with one of
the largest secondary hospitals in Johannesburg. It's called Helen
Joseph Hospital. And it's central Jo-berg. So the population
is a lot of migrants back and forth,
Malawi's, Zimbabwean, but also a lot of South
African nationals. And so what was going on in
this time, roughly around 2007, is all of the model of care for
HIV treatment in South Africa was doctor-initiated and
then doctor-managed care. So they always
felt that HIV care had to be managed by doctors. They were the only
ones who could do it, nurses weren't going to
be capable of doing that. But what they started to see
was this overrun clinics, right? There were long lines,
patients were leaving, getting lost to follow up. It was just very difficult to
access care in a timely manner. So what they started to do is
de-centralize or down-refer patients who were stable to
local primary health care clinics that were either
within the vicinity of the actual main clinic,
which is Themba Lethu, or a primary health
care clinic that was closer to the patient's home. And so this began in
2007 where Themba Lethu began to down-refer their stable
patients to primary health care clinics. In the primary
health care clinics the patients are managed by
nurses, not by doctors, OK? So at the same time that this
was going on my colleagues were competing what was
called the CIPRA trial. It was a non-inferiority
trial comparing nurse to doctor-managed
outcomes in patients. And what they showed was
nurses can manage patients in a clinical trial
setting and have comparable outcomes to doctors. But what this doesn't
answer is can nurses do this in an
observational setting where there's not close
supervision or strict inclusion criteria of these patients. So this is where we went to
the Themba Lethu data set. It's a massive database. It's a longitudinal
prospective cohort and the data is collected
in a database capturing system called Therapy Edge
which was designed specifically for HIV care and treatment. And then at each
clinic we collect demographic information,
clinical information, drugs, and conditions. And currently there's
about 35,000 patients that are on treatment
in the database. The study population
for this analysis was patients who were
adults and initiated onto a standard first line
regimen at Themba Lethu Clinic after 2004. And they had to meet
these eligibility criteria for being down-referred to a
primary health care clinic. And so that was
stable on treatment, they had to have no
opportunistic infections, they had to have a higher CD4
count, be virally-suppressed. And then the
clinicians, also, there were some subjective
measures there of whether the clinician felt
they were good candidates for down-referral. And so in this population,
we were limited once we-- the eligible patients,
rather, there were about 5,000
eligible patients. And they were all eligible
for down-referral. So they all met these criteria. But 1,579 were
actually transferred to the primary
health care clinic. So we'll consider them
the exposed group. And the other
remaining 3,421 were eligible but not transferred. So we'll refer to them
as the unexposed group. And we used logistic regression
to model the propensity of receiving exposure. And I'll go through this
in a little more detail. And then we matched patients
exposed and unexposed based on their
propensity score, OK? So we're trying to balance
observed and unobserved confounders between the
exposed and the unexposed. And then we conducted
a survival analysis to look at our outcomes. So when we're talking about
modeling a patient's propensity score, we're looking
at their probability of receiving exposure. And exposure in
this case is being down-referred to the
primary health care clinics. Because remember, they're
all eligible for it. It's just those that are
down-referred are the ones that are actually exposed. So we modeled that based on a
set of observed confounders, which is gender, age, CD4
count, hemoglobin, body mass index, and other factors. And so what we have is we
have the entire population. The red circle is our
population of patients that are down-referred to the
primary health care clinic. The blue line is those that
are not down-referred but are eligible. So we modeled the probability
that the entire population would have the exposure
of being transferred to the primary
health care clinic. And then we matched those
with that overlap of who had similar propensity scores. And then that matched sample,
that overlapped sample, is what we use for our analysis. And I'll break that down
in a little more detail. But just a basic calculation
of a propensity score. So this is a simple
two-by-two table. On the left you
have patients that have the actual
confounder of interest. On the right you have
those that don't have it. And then we have simple
exposed and unexposed groups within both of those strata. You can ignore disease, d
equals one, d equals zero. It doesn't even
matter in this case because we're just looking
at exposure and confounders. So to model someone's
propensity score, the probability of receiving-- probability of
being exposed given that you have a
confounder is simply 84 over the total in the strata. Which gives us a
probability of 0.62 or 62%. The same thing goes for
the confounder, or those without the confounder. So if we want to model the
probability of exposure given that you don't have
the confounder, is simply 177, the
total exposed, divided by the total in the strata. And that gives us a probability
or propensity of 0.44, 44%. So you can think
of in our analysis we have person A who's
been down-referred, person B who is eligible
for down-referral but not down-referred. So we have our exposed
and our unexposed. And they both have a predicted
probability or propensity of 85% of being down-referred. So they're identical on
those observe confounders, in regards to how we created
our propensity score, and their exposure just differs. So you can think of them as
essentially randomly assigned, OK? At least on observed
confounders. And I'll keep saying that
because we can't say anything about unobserved. So this is the actual
data from the study. So we had 5,000 patients. On the left we had
1,579 which were down-referred to the clinic. 3,421 which were not
down-referred but eligible. So on the left is exposed,
on the right is unexposed. After getting rid of some
due to exclusion criteria, we had a final population
of exposed of 693 and a final population
of unexposed of 2,968. And so we had the
propensity scores for that entire population. And then we matched the 693
to the 2,968 one to three, OK? So we had a final population
of 693 exposed and 2,079 unexposed. So that's our final
analytical population matched under propensity to be exposed. And this table, this
first table is basically what you would see in table
one of a clinical trial, right? What we're trying
to show here is that we're just balanced
unobserved characteristics between exposed and
unexposed groups. And you can't probably see it,
the numbers are quite small, but you can trust me. It is published, so
you can actually check. So our exposed group
is down-referred and our unexposed
is not down-referred and all this shows is that
these characteristics, the proportions across
both, are comparable. But there's no
significant difference. That the propensity
score matching did work and it helped us balance
observed confounders between the two groups. And so what we were able to
conclude with this analysis that we couldn't in the
randomized clinical trial is that patients managed
by nurses compared to patients managed by doctors
for HIV care and treatment can have, if not comparable,
if not better outcomes in this case. Because what we showed, and I
won't go into all the results, but what we showed was
that those patients that were treated by nurses had an
80% decrease in risk of death, 70% decrease in risk of loss,
and a 40% decrease in risk of viral load rebound or
failure, in this case. So they had better outcomes. And it's important to
note that the outcomes in these-- the events
were quite few in number because these are
stable patients. So our conclusion
is that the results are comparable, if
not maybe better, with patients that
are managed by nurses. The next analysis here, and this
is from my dissertation work, using a regression discontinuity
design to evaluate a policy change for HIV in 2010. This is an application
of a method that I'll continue to use
going forward in my career. If it's appropriate, obviously. So just a little bit
more background here. As I mentioned before,
in 2004 the WHO rolled out treatment in
resource limited settings. And at that time,
first line therapy was two drug classes,
and three drugs total. And you don't have to worry
about the fancy names. The drugs were two nucleoside
reverse transcriptase inhibitors, which was
lamivudine, and then stavudine or zidovudine. And then one
non-nucleoside reverse transcriptase inhibitor. And again, the drug
names are not important, but you will hear
me say them a lot. So stavudine and zidovudine
are the two older drugs that were of interest
in this analysis. And then in 2010, the WHO
updated their guidelines when new drugs became available
in resource limited settings. So the drug tenofovir, which
has been used in the high income countries since early
2000, finally made its way to resource limited settings. It was delayed
because of the cost, and because one of the main
side effects of tenofovir is renal insufficiency
and potentially renal failure, which is really
expensive, but rare. Also during that time, efficacy
trials were coming out, and safety trials, saying that
the side effects of tenofovir compared to these older
drugs is a better toxicity profile and less side effects. So it's a better drug, better
quality of life for patients. And then also, the
Clinton Foundation did a really good job of
getting the prices down. Even though tenofovir is
still four times the cost of the older drugs,
it's still much cheaper than it was back
in the mid 2000s. So what the change
in the WHO guidelines set us up to do
here, and this is more of the quintessential
quasi-experimental design, propensity score. There wasn't a cut off, and
there wasn't, obviously, any randomization
outside of trying to simulate that with
propensity score. But this is more of the
quintessential ones. What we're doing in this case is
we're not assigning, randomly, patients to receive a
treatment or control, but they're randomly assigned
based on a policy change. And I'll explain
this in more detail, but as long as nothing else
is changing, the population or outcomes within
that population, that policy change is
essentially random. And patients on either
side of that policy change can be considered similar. So we can get what is
called local randomization. At that time, there was a
handful of observational work. Some of which I had
put out previously, myself and my colleagues. That was all
prospective cohort work. And so what was coming
out was this kind of mixed bag of, when comparing
the old drug to the new drug, results. Some were showing a
higher risk of mortality, some were showing a
higher risk of loss, some were showing that
they were comparable. But we know that in
observational research, a lot of this is based
on the assumption that when we balance
based, or adjust for observed confounders, that
we're balanced on unobserved. And that's a strong assumption. Also what was
happening at that time is two of the studies had
included patients that were initiated onto
the new drug prior to the change in the policy. So these patients had
contraindications, or were sicker. So they couldn't take the
drugs that were available, they could only
take the new drugs. This leads to what we refer to
as confounding by indication, which is going to cause
them to be at increased risk of poor outcomes. Regression discontinuity is best
shown visually, at least when I learned it, visuals helped. So this is a method
that's been used in econometrics in
the educational world since the 1960s. And it's kind of
foreign to epidemiology because I think people
get really nervous when I don't adjust for anything. So there's been some
concern and difficulty in getting it published,
but we'll sort that out. So what this is
is, in this graph we have time on the
horizontal axis. And then you have what we will
refer to as the threshold, or the cutoff. Which in this case,
this dotted line represents the date
of policy change. And then based on where
in this analysis patients will be assigned to
either the control group or the experimental
group depending on the date of initiation
of treatment for HIV. So if a patient
initiates treatment prior to the guideline change,
they're in the control group. If they initiate treatment
after the guideline change, they're in the
experimental group. And if there is no
effect of the exposure, no effect of that
change in the guideline or change in treatment,
then we would expect to see something like this. No discontinuity or
break at that threshold. But if it's the appropriate
method, what we will see is this break at the
threshold, and I'll show you this in the results. So this break at the
threshold is the actual effect of the intervention. So as long as nothing else is
changing along this timeline, outcomes are continuous,
patient demographics and clinical characteristics
are the same, this is essentially randomized. Right around the threshold. So we're getting close to
simulating a randomized trial. The data source that we
use for this analysis is a huge database for
HIV care and treatment. It's the International
Epidemiological Database for Evaluation of HIV and AIDS. It's a NIH funded project where
they pool observational HIV databases into regions
all over the world. The region that we used for
this was southern Africa. Specifically, we used
Zambia in South Africa. This analysis is a
prospective cohort study where we're applying a
regression discontinuity design. It's a fairly big
population, over 52,000, the majority of which
came from Zambia, where there's over 36,000. Their HIV positive treatment
naive, non-pregnant, patients were greater than
16 years of age. Then they had to
initiate treatment 12 months prior,
or 12 months after, the change in the guideline. And that date of change
varies between South Africa and Zambia. South Africa was right along
lines with the World Health Organization recommendation,
in April 1, 2010. That's when they said change,
and South Africa did that. Zambia changed a lot
earlier because they got a huge donation from
the pharmaceutical companies to roll out tenofovir, the
new drug, at an earlier time. So that's why it's
three years earlier. But these two dates are the
thresholds, or the cutoff, that we used in the analysis. Then, all patients had the
potential for 24 months of follow up to get a look at
our desired outcomes, which were death, attrition, which
is a combination of death and loss,
immunological response, which is based on
your CD4 change, which measures your immune
status, virologic failure, and this is HIV failure. So two consecutive viral
loads greater than 1,000. And then single-drug
substitution, and this is an indicator of
toxicity from a drug. And we're only looking at
that actual substitution within that one class of
stavudine, zidovudine, and tenofovir. We're not looking
at anything else, just seeing if those drugs
are being switched out. Coming back to this graph,
regression discontinuity can be implemented whenever
exposure is determined by the threshold rule. So now that data [? verite ?]
initiation is that threshold. And I'll refer to what's called
the assignment variable, which is going to assign patients to
either side of that threshold. And that's the date
they initiate ART. So if a patient initiates ART
before the date of guideline change, they are
considered in the old drug era for the control group. If a patient initiates
after the guideline change, they're in the new
drug era of tenofovir. And then, again, as long as
there is no change in outcomes, the patient population
is consistent over time, we can assume that
those patients that are just above and
below the threshold are essentially randomized. And we call it
local randomization because you can only say
that right around that line. The further you go
away from that line, the assumption doesn't hold. And so they're
essentially random, or balanced unobserved
and unobserved factors, and the only difference should
be the treatments that they receive, is that exposure. I'm going to go
through this quickly, if you're interested in the
linear regression equation. It's a simple linear
regression equation. We have a constant. We have time and
then interaction with the actual treatment. We have an exposure indicator. And then we have this beta2. Within a randomized
clinical trial, you're estimating what
we refer to as the intent to treat effect. So you're analyzing
it based on what people were assigned to receive
for treatment or placebo. So this beta2 is what
we're interested in. We're not controlling
for anything else or any other factors
because we're assuming that they're balanced. And there's ways to check that. There is one main threat
to regression discontinuity in regards to the
validity of it, and that is manipulation
of the assignment variable, so that data [? verite ?]
initiation by the individual. And you can think of potentially
two different scenarios, which I think are highly unlikely,
but you could see happening. Where a patient knows that
there is a new drug coming out after a certain date,
and so they hold off on going into the clinic to
initiate care until that drug is available. That's highly unlikely. Patients aren't really
in tune to what's going on at a national level
and what the policies are. So that's most likely unlikely. You could also see a scenario
where a clinician could delay treatment because they
know this new drug is going to be available. But that's unethical. These patients are coming into
the treatment extremely sick, their CD4 count
is very, very low. So to send them away
without drugs is unethical. So that could be occurring,
but we don't believe it is, and there's ways
to test for that. We can test for that by looking
at if our observed confounders are balanced on either
side of the thresholds. So is our patient
population consistent? And then we can also see if
there's that manipulation, if people are delaying treatment
until after the guideline change, by just
basically graphing the numbers of initiates
bend in weekly intervals in our time period
to see if there's any, what we refer
to as bunching, on the right side
of the threshold. So the results. This is basically a table 1 in
a randomized clinical trial. And all it's showing you is
we have South Africa here, we have Zambia here. We have our observed
confounders here, which the only things we
could actually measure were age, gender, CD4 count,
weight and hemoglobin. These are all clinical and
demographic characteristics of patients. What we saw, there was
no significant difference between our exposed and
our unexposed group, or a treatment and control on
either side of the threshold. On either side of that
line, they were similar in South Africa and in Zambia. And that's all that's showing. To see if there was any of that
manipulation of the assignment variable, people were
delaying treatment until after the guidelines
actually happened, this is just the
graph where we're looking at the number
of patients initiating. So we have the red
line-- and this is for the other graphs
I'll show you-- is always the threshold or the cutoff
in Zambia and in South Africa. And then patients,
again, are initiated on either side of this threshold
and either on the control group, or the treatment group. Then we have just number
of initiates on the y-axis. Then time on the x-axis,
so 12 months on this side, and 12 months on this side. These black arrows
represent seasonal trends in South Africa, so
we expect to see dips in patients, the numbers
initiating treatment. What we're looking at here
is, if those patients who could have initiated over here
were delaying for some reason, we would expect to see this
huge increase in patient numbers on the right side
of that red line, because they were manipulating
the date that they actually initiated treatment. We don't see that. We see a slight dip in
South Africa and in Zambia, but then the numbers
basically increase to the numbers that were
previously there, or previously being initiated. So we don't feel like there's
evidence of manipulation of the assignment variable. This figure is just showing
the actual probability of tenofovir. So again, to orient
you, red line is the date of the
guideline change, and then control or
treatment on either side. And this is just the
proportion of patients that are initiating tenofovir. This is the reason that we felt
like regression discontinuity was a good design option,
because you can see right at this threshold, the
exposure substantially increases in South Africa. We go 81%, a risk
difference of 81%. And a risk difference
of 41% in Zambia. And remember, they delayed,
and it wasn't nationally done, so Zambia didn't have
as stark of a contrast. So that's great. That's where we're like, OK,
regression discontinuity. There is a discontinuity
in our exposure at that policy change, which we
were very excited about that. This was the only
outcome that there was a significant difference. The orientation's
the same, we're looking at the proportion
of single-drug substitution on the y-axis,
and time on the x. South Africa and Zambia. And single-drug
substitution is just that indicator of toxicity. So we see a significant
decrease in toxicity, a risk difference of negative 15. And then a risk difference
of a decrease of negative two in Zambia. So not as large, but still
a significant decrease. And this was expected
since tenofovir has a better side
effect and toxicity profile from the other drugs. Then these other graphs, this
is just South Africa alone. These are the other outcomes. And remember how I said if
we had expected that there'd be no effective
exposure on the outcome, we would just see a flat line. There was really no effect on
any of these other outcomes. We have death, attrition,
mean CD4 count, and viral load failure. So there really was
not a difference. The exposure did not
affect the outcome. Which should be. These drugs are
comparable, they're effective at treating
HIV if they're taken. Then this is just Zambia. And we see again, we didn't
look at viral load failure, but you see death, loss,
and then CD4 count. They're flat. So the policy change, or
that exposure to tenofovir, didn't have a
significant increase or decrease in the
outcomes in this case. In summary, the study
allowed us to say that starting patients on
tenofovir standard of care reduced single-drug
substitutions, so reduced toxicity. But there were no changes
in other observed outcomes, in this case. Strengths were huge
sample size, you'd be hard-pressed to get
this in a clinical trial. It'd be very expensive
and you would spend your whole entire life,
and your children's life, trying to get that. If the assumptions are met
in a regression discontinuity design, it's less
vulnerable to major threats to validity, like a lot
of observational research that we do is, because
of the assumptions. We can check those
threats to validity by looking at if our observed
confounders are balanced, or if there's any manipulation
of the assignment variable. And we saw that there was none. Hopefully what I've shown
you by these two examples is that, as we move
from these descriptive studies in observational
world into analytical, down towards the randomized clinical
trial, our gold standard, quasi experimental designs can
help us, if applied properly and the assumptions are met,
to get a little bit closer to that establishing causality. So instead of
correlation, we can say that an exposure actually
causes the outcome that were of interest. Thank you. [APPLAUSE] - That was absolutely
fascinating. Thank you. And now we have 15 or 20
minutes for your questions. And while you are thinking
of your questions, maybe I'll ask one
or two of mine. Neal, I was struck
by the slide up there that showed the proportion
of patient contacts by what the procedure was. Checking vital signs was right
there at the top, like 36%, 38%, something like that. As you think about the
propensity for those contacts in that network to
cause infection, do you think about changing the
frequency of those procedures? Is there any talk
about, maybe you don't need to take vital
signs quite as often? I'm sort of wondering how you
think about the procedures in that mix. - So the procedures
themselves, we actually ranked the procedures
based on propensity to spread in organisms. So something like
vital signs, it was like the lowest threshold
that's possible that we thought could actually spread MRSA. But some of the
other procedures, like the stool description and
testing procedure that I think was the second or third
most common, yeah. I think that does warrant
reviewing that policy to say, why are we
doing this so much? Unless there's a good
reason for doing it, this is a clear opportunity
for MRSA transmission to occur. Even though it's done
under non-sterile gloving, there still is the potential
for something to happen. These are the kinds of
conversations that, this works, spawned with our infection
control group at the hospital. And actually, I
hadn't particularly thought of that, so I want
to go back right now and-- - Excellent, great. - --say, how often are we
actually doing these things? And do we need to
do them as often? - Yeah. A really, really
interesting analysis. Thinking also about acuity, and
the increased risk of acuity, which is that mix of severity
of the cases in your NICU. Is that an individual
characteristic, or is that also as a
proportion of your population. It's sort of interesting
to think about the tension. Like when the acuity
is high, is it the highly acute
cases who get sick, or is it everybody's
risk that increases? - Yes it's both. It's both. It's the highly acute
patients because they are at the greater risk. They're having more done
to them and they're sick, or they may have a less of
a developed immune system and skin system. But then it's that
breakdown that happens overall, especially
if there's a code in the unit. If there's a code in the
unit, then all the demands get focused right away on
that very critical infant. And then you have
to think about, as the groups
disperse from the code to go back to caring with
the other, at that moment, they are most vulnerable
to spreading something if that particular
patient happened to have been
colonized or infected. - Really Interesting. So Alana, I think the
regression discontinuity is really interesting. And I'm thinking of
other opportunities for doing that, and also
thinking about sometimes we get natural experiments
or disasters that afford the same kind of
opportunity or threshold as a policy change. We once did a
study where we were looking at the risk of preterm
delivery in a pregnancy cohort that we had going
on here in Boston. And we were interested in
the role of acute stressors. And September 11 happened right
in the middle of that cohort, so we were able to look
on either side of it. But one of the things I
remember we wrestled with is how much time is
eligible before the event, and then after the event. And does that need to
be comparable length, and how do you define
that against the risk of-- the risk to these
things, of course, is there's perhaps a
secular change going on in the background. I wondered if you'd
talk about that. - So in this
analysis, you can have a window around the
threshold if you want. Something like, I
mean there's obviously a date with September
11, but there could be some
window around that, where it could help you kind
of tease out the discontinuity more clearly. But you're right. With something like a natural
disaster or a terrorist attack, there are other
underlying factors that may not be consistent. Where that local randomization
or continuous outcomes assumptions that
you have to make would be actually consistent,
those assumptions would be met, is what I would be a
bit concerned about. Just because of everything
that also could be changing around that time date. We're using this going forward. In South Africa, we're building
a national level cohort of HIV, and then also with
chronic diseases, diabetes and cardiovascular disease, by
doing a lot more big data work. We're using a laboratory system
that they have in South Africa to link labs to each other to
create individuals and create prospective cohorts and then
assess policy changes over time where it's like a
clear cut policy, and there isn't this kind of-- I think a natural
disaster or anything around that may be a little
bit more difficult to make sure the assumptions are met. But it'd be interesting to try. - Let's open up to questions. - I'm a particle
physicist, so I come from a complete different
domain of how we do analysis. One of the things
I was curious about is, we have this set of
tools called Monte Carlos. And basically, we take a model
of the system we're studying, including random stuff,
diffusion, that sort of thing. It just struck me that when
you try to do a validation, the nice thing
about Monte Carlo is that you can run 100 times what
you just did for the HIV study, or 100 times what you
just did, and then you can judge the
statistical significance rather than just using the
more refined statistical tools. I was wondering if
there's any attempt to move towards
that in the field that you're using these
secondary sources? - Yes. In fact, I have
many colleagues that would be quite encouraged
by your comment there. Some of the other
work that I do, I use a lot of
Bayesian methods, where we bring in prior information. We don't start out assuming
everyone's at equal risk. We assume some groups are
at greater risk than others, and then we run these
simulations, many hundreds of thousands of times. So there's definitely a movement
of foot in the field to go more towards that direction. But from a teaching
aspect, it's still being taught in a
traditional framework. Just like, here's
how to get started. So in some sense, it
has its own momentum and it's being taught this way. But there are more and more
people using the techniques that you're describing. - And I think as myself
and my colleagues move into more of the
big data world, which is kind of a broad term, a
lot of those techniques we're talking about applying. So yes, there's movement. But you're right, it's taught
more in a traditional sense. - I don't know if I'm
speaking for you as well. Our field, I feel,
uses a lot of methods that come from other places. For example, social sciences. The methods that I'm
using for the work that I presented on multilevel
analysis, a lot of that comes from the social sciences. So I feel that your
group is pioneering some of these methods, and
speaks well for our group and it'll just take a little
bit of time to catch up to you. - Other questions? - Hi. It's a question for Neal. How much variation are there
in NICUs in other places? Is there a, kind
of, gold standard where they don't have
these problems at all? - Absolutely, critical
care management of babies differs institution
to institution. So in some sense,
I wonder with you, how generalizable are our
findings to other NICUs. A lot of the day to day
what goes on in NICU is fairly well prescribed,
and they follow protocols and guidelines. So in some sense, there
will be a variation. I would love to actually
repeat this analysis with a whole group of NICUs
and see what's out there. But if another NICU is
doing some infection control procedure because they
have a flag in their EMR to say it's a high census right
now or it's a high unit acuity, then our findings may not
necessarily apply to them. That is kind of a
next step, I think, with this work is
let's see if we can replicate this in other NICUs. Is this something just
specific to Christiana care, or is this truly a
generalizable issue that's happening across the board. - As a follow up
question, the size of the NICU that you described
is considerably larger than the community hospitals
that I've worked at. It would be interesting to
compare the size of the NICU in terms of the outcomes,
because this would certainly have a significant
effect on advice as to whether regionalize
this kind of care or whether it can be as
diffusive as it is now. - Yes. I'm dealing with a level 3 NICU
that can go up to 70 patients. And if you consider a smaller
regional hospital that maybe has like a 10 or even just
a very, very small 5 bed NICU, I think that's a great
question to explore. - All right. I know we have tea and coffee
and cookies waiting outside, so let's take a break. I want to thank you guys,
these were really fascinating-- [APPLAUSE]
