Consider the following sequence of events:
-In 1945, an 82 day long battle between US Marine and Army forces and the Imperial Japanese
Army was fought in the prefecture of Okinawa. This battle was considered one of the bloodiest
in the pacific and resulted in the death of almost 150,000 Okinawans, roughly a third
of the population. Due to widespread destruction, and foodstuffs
in particular being stolen or deliberately destroyed, a huge number of civilians were
left starving and struggled to return to a sufficient diet after the war. -Just four years later In 1949, data from
US national archives indicated that 85% of Okinawansâ calories came from carbohydrates
with sweet potatoes comprising 69% of all calories and 1% of calories coming from Fish. Then, several decades later, A 2016 paper
pointed out that the Okinawanâs 1949 post war diet has a ratio of protein to carbohydrate
that is similar to an experimental high carb diet used to improve lifespan in rodents. Then just this year In January 2019, a BBC
article referring to this paper comes out with the headline âA high-carb diet may
explain why Okinawans live so long.â Okinawan people did historically eat their
fair share of sweet potatoes - the typhoon resistant tuber made for a good staple crop. However, is it fair to make conclusions about
Okinawans longevity based on their diet right after the war? In any case, when you dig in a bit, it becomes
apparent that this BBC article lacks some very important context. The idea of this video is to give some insight
into the shortcomings of research in order to help you understand what makes for a weak
or strong piece of supporting evidence for this or that health claim. Letâs say a detective wants to determine
who killed John. He will follow clues and investigate evidence
while considering the strengths and shortcomings in each piece of evidence. For example, a witness saying they saw someone
that sort of looked like Count Jackingtonâs butler is much weaker evidence than a security
camera capturing an image of the butler. Itâs good to take a similar approach when
trying to make conclusions from research. Last year, I picked up this book with the
title âThe Best Diet: Simple and Evidence based guide to healthy eatingâ written by
a doctor Tsugawa at UCLA. I saw this book around when I was making a
video on Butter Coffee and the cover of this book has a big red X next to the word âButter
Coffeeâ so I picked it up to make sure I wouldnât have to delete my video. On page 31 that it says âButter is a bad
fat as shown by several studies.â Now, the first, most obvious step to evaluate
a claim is to investigate the evidence the claim is based on. Thereâs a reference number next to this
sentence about butter, so I go to reference (4) in the very back of the book and it has
a footnote saying âThe idea that butter is bad comes from observational studies in
which butter seems to raise the âbadâ LDL cholesterol. However, the evidence that butter intake affects
your risk for disease is not particularly strong.â This footnote goes on to point out that a
2016 paper did not show an association between butter, heart attack and stroke. This book also brings up a very often debated
topic: Eggs. Eggs, especially the yolk, can be a cheap
source of good nutrients like fat soluble vitamins which arenât contained in the majority
of typically eaten foods. However, the book recommends limiting your
egg intake to only one a day. If only this book came out earlier, then this
poor 88 year old man could have been warned. He ate 20 to 30 eggs a day for 15 years as
of 1991. Interestingly, he maintained normal plasma
cholesterol despite the ludicrous amount of dietary cholesterol he consumed- weâll talk
about how this affected him in a moment and Weâll come back to why it's being said to
eat only one egg a day, but before we do that, allow me to explain a couple concepts. The first is that of confounding variables:
Here we have the grabbing headline âHigh-Fat Diet Linked to Anxiety, Depression.â If we take a look at the study theyâre basing
the article on, we see that the high-fat diet they used - D12451 from Research Diets Inc.
, contains 20% protein, 45% fat and 35% carbohydrate - this is relatively high fat. However, half of the carbohydrate is refined
table sugar⌠by weight thereâs almost as much pure sugar as there is fat. So, Do think this might confound the effect
of fat? Now, Biology is incredibly complicated - there
are so many variables that may affect a given output. So, a common challenge is isolating the effect
of one food or gene on disease risk from the effects of all the other foods and genes that
could also potentially increase that disease risk. I had a chat about the various challenges
in scientific research with Eli Lyons,the CEO of the synthetic biology
company Tupac Bio. In his current position, and as a PhD candidate
at the University of Tokyo, Eli has a decade of experience regularly reading through research
papers. Afterwards I followed up on Skype to ask him
about the challenge isolating variables. âIn some of my work, Iâve done statistical
analysis on oncogenes or high throughput mutagenesis. Oncogenes are cancer causing genes, or, genes
that when mutated may drive cancer. And, what commonly occurs though is that in
a tumor, for example, you may have many genes that are mutated. However, not all of the mutated genes are
actually driving the cancer. So, the ones that are driving it are called
driver cancer genes. And, so how do you isolate the effects or
determine which genes are the driver cancer genes and which are like carrier mutations. Itâs also more complex because there may
be some interactions between the driver cancer genes and some genes that are mutated and
the interactions are very complex, but the impact may be largely due, the majority of
the impact may be due to the driver genes for example. And so, really, itâs isolating how large
of an impact or, how much of the cancer is due to gene A - a mutation in gene A, and
how much of the cancer is due to a mutation in gene B for example.â (2)This brings us to my next point, the importance
of context: A good example for why itâs hard to isolate
things from context is protein. There seems to be some concern about protein
for people on a low carb diet. One of the goals of doing a low carb diet
is keeping your insulin low, and to achieve that people replace the carbs with fat or
protein, but protein ironically seems to raise insulin levels. However, does the context matter? Does protein by itself reliably raise insulin
levels? If we take a look at this study in canines
as presented by Dr. Benjamin Bikman, we see that dogs receiving an infusion of glucose
get spikes in their insulin levels when given the amino acid alanine. So, it looks like protein does raise insulin. But what about dogs without the glucose infusion? The dogs not receiving glucose didnât see
their insulin change to any noticeable degree. So then, imagine how this fact would confuse
the data in for example a study looking at how protein affects risk for diabetes, an
insulin driven disease. You might look at how many servings of meat
people are having per day and then look at who develops diabetes, but the physiological
effects of a hamburger patty tucked in a whole wheat bun and served with french fries are
going to be much different from a steak served only with butter and rosemary. Another good example of the importance of
context comes from the work of Dave Feldman. Dave is an independent lipid researcher who
has developed something called the Lipid Energy Model. Heâs actually been interviewed on this channel
before. We all know that we are supposed to keep our
LDL bad cholesterol as low as possible to prevent heart disease. However, in our interview, Dave explained
the logic behind why when it comes to heart disease, LDL - the so called bad cholesterol
isnât all that important in the context of high HDL and low triglycerides. That is, you donât need to worry all that
much about sky high bad cholesterol if your HDL is high and your triglycerides are low. âThe NHANES data has certainly been exciting
because while itâs true that if you look at LDL by itself, it's associated with higher
mortality, when you look at it grouped with high HDL and low triglycerides, itâs associated
with low mortality.â This huge NHANES data set that Dave recently
got his hands on is showing that this idea that HDL and triglycerides are more important
than LDL cholesterol indeed pans out surprisingly well. âI first removed everybody that had a low
LDL, so everyone with 159 mg per dL and lower, I took out. I went ahead and separated out everybody with
HDL cholesterol of 49 or lower. And then finally, I took out everybody above
100mg/dL of triglycerides. This was pretty exciting because now I could
actually look at what the mortality data was that was left. And that mortality was pretty exciting because
not only did they have an all cause mortality that was lower than the average, but, believe
it or not, diseases of the heart were extraordinarily low. The youngest person in that group that was
left over, once all three of these markers were accounted for, was 68. The oldest in the group? 94. And outside of those two, everybody else died
in their 80âs. A total of 18 total deaths from diseases of
the heart, and almost everyone died of old age.â So, letâs go back to that earlier point
about the recommendation to eat only one egg a day. The author of the earlier mentioned book explains
that, according to a 2013 meta analysis, those who ate more than one egg a day had a 42%
higher risk for developing type 2 diabetes than those who hardly ate eggs. What a meta analysis does is pile the data
from multiple studies together to try and make more accurate conclusions. So hereâs another point for investigating
a claim - the cumbersome task of actually digging through the referenced study. So, Letâs look at this meta analysis that
reference 3 points to. Then, letâs go to Table 1 and see what studies
are used for the data on Type 2 diabetes. Looking at references 51, 37 and 41 we get
these three studies: The data from this study[37] did suggest that
high levels of egg consumption are associated with increased risk for type 2 diabetes. However, this study[51] found that âNo statistically
significant associations between egg consumption and diabetes.â and this other study also
[41] found âno association between egg consumption or dietary cholesterol and increased risk
of incident T2D." But, by taking these three studies with differing
conclusions and pooling the data together in a meta-analysis, the conclusion becomes
âcompared with those who never consume eggs, those who eat 1 egg per day or more are 42%
more likely to develop type 2 diabetes.â At first, this seems like a good idea - more
data, so a more accurate picture, right? However, in this study [37], the women eating
the most eggs are smoking the most, eating higher amounts of trans fat, eating 500 more
calories per day and exercising the least. The men who ate more eggs also drank more
alcohol and smoked more. The researchers do take these unhealthy habits
into account and make adjustments when analyzing the data, but it is very ambitious to assume
you can quantify the effects of all things on diabetes risk, and then subtract these
to accurately understand how just eggs by themselves affect diabetes risk. In any case, the studies used in this meta
analysis not all adjusting for potential confounding variables. This one only adjusts for Age and Sex. This one doesnât even account for how many
calories the people ate along with the eggs - what if the people eating two eggs a day
are getting those two eggs from a Dennyâs Grand Slam seven days a week? By the way, remember the guy who ate 25 eggs
a day for 15 years? The 88 year old had no health complaints other
than poor memory and loneliness after his wife passed away. Also, he had no history of stroke, heart disease
or diabetes. So, while this kind of meta analysis study
is a clue to the puzzle of eggs, I think youâd agree itâs not as strong a piece of evidence
as it appeared at first glance. Of course the other reason for us being told
to not eat eggs comes from the theory that fat and cholesterol cause heart disease. The very first clue for this theory comes
from research by Nikolai Anichkov. Anichkov found that feeding cholesterol to
rabbits had them develop very high levels of blood cholesterol and atherosclerosis. ...But rabbits are herbivores, and their natural
intake of cholesterol hovers right around zero milligrams. So, letâs move onto my next point: the shortcomings
of using animals as a model for understanding humans. âWe were talking about the problems with
model organisms...â âYea⌠The basic idea is that the mice that they
use in experiments are not very diverse. Right, so theyâre kind of like clones andâŚ
the way theyâre breed is similar to having a breed of dog. If you did all of your experiments on golden
retrievers. Is that really representative of what would
happen if you did the experiment on ten different dog breeds?â To give you a picture of how this can affect
research, consider the work of Lewis Dahl. In 1963, he fed rats a high salt diet and
found that some, but not all developed high blood pressure. He then went on to selectively breed rats,
producing a strain of rats that were genetically sensitive to salt. Then, in 1970, he fed these salt sensitive
rats commercial baby food and about half of these salt sensitive rats died. So he concluded that the high salt content
of the baby food formula was to blame. After his study was published, the US senate
issued a mandate for lowering salt in baby foods. Now, think about that for a moment. Does anything sound odd to you about this
sequence of events? Anyhow, letâs get back to our discussion
on mice models: â...Itâs well known that mice models are
not always very good and the pharmaceutical industry knows that really well - it's really
easy to... thereâs a lot of literature on it, but itâs really easy to think about
- which is well, how do drugs get approved for humans? Well, one is they do early stage pre-clinical
work which is usually on cell lines and then on mice and then they move maybe to canines
and apes or something and then they start human trials. But, you may be familiar that with the phenomena
that in clinical trials 1, the drug passed but it failed in clinical 3 trial. But if you imagine like, it failed at one
of these human based trials but, well, it passed the mouse trial. Right, so I think that right there gives you
some idea of that well, the mouse model did not model what we expected to happen in humans. Why does that happen? Well⌠mouse is different from a human, and
also the model they make where the mouse has a certain type of tumor, that tumor may not
perfectly model the tumor in humans. Diabetes in mouse might not be the same...
like the model they make.â âThis situation where like youâll pass
the mouse phase, but then you fail at the human phase. Thatâs not a rare occurence?â âNot rare at all, itâs probably the opposite.â With all this said, studies based on mouse
models are still pieces of evidence - not to be completely dismissed whenever we donât
like their findings. But, we should try and investigate the specifics
of why a particular mouse model wouldnât be appropriate for emulating humans. âI guess One of the easiest different to
point out is just that their lifetimes are shorter. So⌠they mature faster. Right, so, Iâm just kind of going through
really easy differences to spot between mouse and human ... in my old lab I was doing some
retinal development research, like studying how the eye develops, but like, somehow itâs
kind of a weird model to use because mice are born blind⌠And then like humans are not⌠so thatâs
kind of odd. These are just some things youâd want to
start to think about when youâre thinking is this a good model to use of humans.â Letâs say itâs a typical Saturday morning,
youâve just made your coffee and are sitting down to read a paper like this one and you
see the words âhigh fat diet induced obesity.â But you are considering doing a keto diet
to lose some weight, and you think âIf a high fat diet is a reliable way to produce
obesity in rodents⌠Why would I want to do a high fat keto diet?â However, we should first investigate if there
are some specific metabolic differences between rodents and humans. The high-fat diet for the mice used in this
paper, D12492, is 20% protein, 60% fat and 20% carbohydrate. An actual ketogenic diet for humans would
need to be restricted to around 10% or even 5% carbohydrate, but at 20% of calories coming
from carbohydrate, this rodent chow is actually a relatively low carb diet for a human. Now Itâs thought that most of the weight
loss magic from a keto or low carb diet comes from lowering insulin and entering ketosis. However, rodents donât enter ketosis nearly
as easily as humans. According to Dr. Benjamin Bikman, in rodent
experiments, without calorie restriction, to get rodents into ketosis, you need to reduce
their diet down to just 1% carbohydrate, 9% protein and 90% fat. Even a diet that is 95% fat barely rodents
gets rodents into ketosis. By the way, a 95% fat diet would be like an
entire cup of butter and about 80g or 8 thin slices of bacon for the day - but any more
bacon than that would be too much protein. Simply put, the amount of carb restriction
that qualifies as low carb or keto for a human does not qualify for a rodent. These kinda specific differences should be
acknowledged when using rodents as models for humans. Letâs move on to my next point: Food vs.
Compounds in food - Weâll start with chocolate. In his book âDoctoring Data,â Dr. Malcolm
Kendrick talks about a headline he saw saying âChemicals found within chocolate protect
against heart disease.â He explains that, according to the research,
âcatechins and procyanidins, found in dark chocolate, inhibit the enzyme Angiotensin
Converting Enzyme (ACE). When ACE is blocked, blood pressure drops.â This is actually how blood pressure lowering
drugs work, so it makes for compelling reasoning behind a compelling headline. The only catch is that there wasnât any
actual clinical effect⌠while dark chocolate did result in an 18% drop in ACE activityâŚ
there was no actual drop in blood pressure in those taking cocoa extract. Another example is the idea that a compound
in such and such food has been found to cause disease, so that food itself must cause disease. For example, there is the idea that heterocyclic
amines in cooked meat cause cancer. However, the studies finding that these heterocyclic
amines are cancerous were giving rodents amounts of HCAs equivalent to 1000 to 100,000 times
the normal amount consumed by humans. As this paper says, âComparison of the carcinogenic
dose in rodents and the actual human daily intake suggests that the latter is definitely
too low for cancer production to be explicable in terms of HCAs alone.â[R] One last example is red wine - youâve probably
heard that it protects against heart disease. This review article from the Journal of Cardiovascular
Disease Research discusses the quote âaccumulating evidence that suggests that red wine possesses
a diverse range of biological actions and may be beneficial in the prevention of CVD.â However, if we look at the references, most
of them are looking at compounds within red wine - namely polyphenols and resveratrol. One study found that the rat equivalent of
one glass of red wine worth of polyphenols had beneficial effects against heart disease. Then again, does that mean a glass of red
wine itself with its 200mg of polyphenols and 10000mg of alcohol prevents heart disease
in humans? By the way, the other compound in red wine
resveratrol has gained a lot of attention for its promising anti aging effects. However, one of the researchers investigating
this compound has said that, in order to get the anti aging benefits, âthe sad news is
youâd need to drink about 1000 bottles a day, which I donât recommend.â In any case, the point is: claims like âsuch
and such food prevents or cause diseaseâ are very different from claims like âcompounds
in such and such food prevent or cause disease.â My next point on what to consider when evaluating
pieces of evidence, is why and from where certain ideas arose. For example,
why did people think to start looking at red wine to see if it benefitted heart disease? Well, sometime around 1991, people were trying
to make sense of the fact that the French ate very large amounts of saturated fat yet
had low rates of heart disease. One idea was that this so called âFrench
Paradoxâ could be explained by Franceâs high red wine consumption. So This is what's called an ad-hoc hypothesis
- a hypothesis added to a theory in order to save it from being falsified. In this case, because saturated fat and cholesterol
must cause heart disease, it was assumed that there must be some protective factor in the
French diet. Put another way the logic is âletâs construct
a new hypothesis (red wine prevents heart disease) to explain data that does not support
our initial hypothesis (fat causes heart disease).â Now, just because a hypothesis is ad-hoc,
doesnât mean itâs wrong, but it deserves scrutiny. And what really deserves scrutiny are things
that go quickly from the idea stage to clinical practice. An example provided in the earlier book âDoctoring
Data,â is how a good idea arguably killed millions of people. From the early 1900âs for about 50 years
or so, it was thought that strict bed rest for about six weeks was the appropriate prescription
for someone who had just had a heart attack. It sort of makes sense, after such a traumatic
event it sounds like it would be best to let the heart rest and keep exertion minimal so
as not to stress the heart. And I really mean minimal - to quote Thomas
Lewis, a prominent physician from the 1930âs: âThe patient is to be guarded by day and
night nursing and helped in every way to avoid voluntary movement, or effort.â So what exactly is wrong with bed rest? As Dr. Kendrick explains, First, lying in
bed stationary for six weeks means that there is a very good chance of developing a deep
vein thrombosis (DVT) in the legs. A high percentage of these break off, travel
to the lungs, and block the arteries in the lungs causing a pulmonary embolus (PE) â an
event with a very high mortality rate. In fact, even a several hour plane flight
carries this risk. In 1977, the term âTravelerâs Thrombosisâ
was coined for people developing deep vein thrombosis on flights. Low oxygen, low humidity , and low cabin pressure
at high elevations plus sitting motionless in a chair for several hours is a good recipe
for thrombosis. The second issue with bed rest is that without
any exercise, and especially after a heart attack, the heart atrophies very rapidly. It becomes weaker, and deadly heart rhythms
develop, so you are far more likely to die of ventricular fibrillation. Dr. Kendrick estimates that hundreds of thousands
of people were dying from bed rest each year, and this approach wasnât being questioned
until the mid-1950âs. So where did the idea come from? Well, in 1912 Dr James Herrick of Chicago
published an article titled âClinical features of sudden obstruction of the coronary arteries,â
Where he essentially described the first documented heart attack. In that article, he stated, âThe importance
of absolute rest in bed for several days is clearâ postinfarctionâŚâ To quote Dr. Kendrick â...Herrick managed
to describe the worldâs first heart attack in 1912 and then, without missing a beat,
he immediately knew that strict bed rest was an essential form of treatment â for a condition
never before described.â Another example of seemingly good ideas harming
people is that of Hormone Replacement Therapy. It had been recognised that women under 60
had far lower rates of heart disease than men of that age. For various reasons it became accepted that
female sex hormones were what was protective against heart disease. According to Dr. Kendrick, one key piece of
evidence amidst the limited amount of evidence for this concept was a 1987 observational
study - observational studies by the way are widely accepted to be very weak pieces of
evidence in general. Yet, the idea was still accepted so well,
in fact, that replacing the declining female sex hormones in menopausal women became incorporated
into the 1992 American College of Physiciansâ guidelines. In the US, failure to prescribe hormone replacement
therapy for menopausal women was akin to medical malpractice. Later, a randomized primary prevention trial
using hormone replacement therapy involving nearly 17,000 women published its results
in JAMA in 2002. This trial found that â...there was a 29%
increase in coronary heart disease risk...â I wonder how many women would have accepted
hormone replacement therapy if they knew the practice originated from a sorta good idea
backed by a weak observational study. My very last point is... the circular situation
where existing ideas can influence research in a way that biases the research towards
acting as evidence for that idea. So⌠what do I mean by that? Dave Feldman who we spoke with earlier has
a good example of what Iâm talking about: âOne of my problems with cholesterol research
is it often lumps soft endpoints with hard endpoints and this can be a bit of a challenge
because our existing opinion on cholesterol can make a difference in how the data is recorded. So, to give you an example on the patient
side, letâs say that you and I have a steak dinner tonight and then afterwards we both
go our separate ways. But, each of that night experiences a thirty
minute prolonged chest pain. And for me, this is the warning I had been
hearing about from my doctor this whole time. After all, heâs been telling me about my
high cholesterol and I need to do something about it or Iâm going to have a heart attack. Sure enough when I go to the hospital, it
does in fact prove true that I did have a non-fatal myocardial infarction. So all of that data then becomes record. You on the other hand, did also have a myocardial
infarction, but the difference is because your cholesterol has been low, you went ahead
and took a TUMS because you felt like it was heartburn, went to sleep, and that data never
ended up anywhere inside of the hospital record. Thatâs a big deal is both of us are already
part of a study. But this also plays into the hands of medical
professionals, because after all, if they would likewise have the same opinion that
high LDL is a risk factor just the same as something like C-Reactive Protein, then that
may be relevant for a judgement call on the margins. Certainly a lot of heart attacks that you
survive are not on the margins. And most medical professionals would agree. But, some are on the margins and thatâs
a soft endpoint. I like hard endpoints like mortality because
theyâre pretty easy to diagnose. Everyone knows whether you lived or died,
and as such, that data is a lot stronger to look at in the long run.â ...And, there are several studies that use
the subjective, soft endpoint Dave is talking about here. So for now, this concludes my points on what
to keep in mind when trying determine if a piece of evidence behind this or that health
claim is strong or not. I realize some of this can be confusing or
disheartening and that nobody has time to dig through 50 research papers just to decide
what to eat for breakfast... but for those of you who want to dig deeper things to deeper
understand what makes us healthy and why, hopefully these points serve as tools to help
you analyze articles and research more effectively.
A mealtime video about meals? Hell yeah!
If you liked this type of video you will love the much more concise and clear healthcare triage. Here is one example: https://youtu.be/p18WGG7Cpro
Nutrition Facts is a great channel for getting summaries of the basic research https://www.youtube.com/user/NutritionFactsOrg
I don't agree with everything this channel has to say. Though if you're interested in reading more about why nutrition science is so hard, the science behind the calorie is a good place to start:
Death of the Calorie
This guy is a fraud and spreads misinformation that kills people. He has a video about why the carnivore diet, which excludes vegetables, is healthy. He also frequently pushes low carb diets which have been repeatedly linked to increased death.
I don't get it.. How are these studies keep getting published? Shouldn't Peer Review prevent that?!ďťż
Not sure why this video has so many uovotes. He's just talking about how studies and experiments can be biased, skewed by other factors or completely out of context. Not really specific to Nutritional science.
this guy is the real MVP, I've been following up on him for months!
from the guy telling you to fear gluten and trying mewing just in case. I won't be giving him more views anytime soon.