Muscle for Life with Mike Matthews - Good Science, Bad Science: The Difference and Why It Matters

Starting point is 00:00:00 Hello, hello, and welcome to another episode of Muscle for Life. I'm Mike Matthews. Thank you for joining me today. And this episode is special because it is one of the chapters of my newest book, well, newest audio book in this case, but it's also available in digital and hard copy formats as well. And it is called Fitness Science Explained, and you can get it right now at fitnesssciencebook.com. And this book is a crash course in reading, understanding, and applying scientific research.

Starting point is 00:00:34 And it teaches you in very simple terms what most people will never know about how to use science to optimize your health, fitness, and lifestyle. Fitness Science Explained covers all of the big moving parts, including the basics of the scientific method, the differences between randomized trials and observational studies, the power of the placebo effect, the importance of sample sizes, the anatomy of statistical analysis, and much more. You'll also learn in the book how to get access to full-text studies without spending a fortune and the most popular journals out there for exercise, nutrition, and supplementation. And you will get a scientist

Starting point is 00:01:20 formulated cheat sheet that will help you quickly and accurately estimate the quality of research that you want to review. In my opinion, the cheat sheet alone is worth the cost of the book. It is very, very practical. So whether you want to discover and use evidence-based methods for building muscle or losing fat faster, or maybe reducing your risk of disease or dysfunction, fat faster or maybe reducing your risk of disease or dysfunction or just maximizing some other aspect of your body, mind, or life, this book will show you the way. Also, to celebrate this joyous occasion, I am giving away $1,500 in Legion gift cards. And all you have to do for a chance to win is head over to fitnesssciencebook.com, buy a copy of the book, any format, and then forward your receipt email to launch at legionsupplements.com. And that's it. You are

Starting point is 00:02:13 entered in the giveaway. You got to act fast though, because winter is coming. No, because winners will be chosen this Friday, September 4th. Oh, and you can also increase your chances of winning by buying extra copies of the book. Again, any formats.. Oh, and you can also increase your chances of winning by buying extra copies of the book. Again, any formats. So specifically, if you buy three copies, you're going to get five giveaway entries. So that is a plus 400% chance to win. If you buy five copies, you're going to get eight giveaway entries, and that is a plus 700% chance to win. And if you buy 10 copies, you're going to get 15 giveaway entries, which is a plus 1400% chance to win. And you are going to get an autographed copy of the book. So for instance,

Starting point is 00:02:52 if you buy the paperback ebook and audio book, that is three copies, five entries to win. And if you were to buy three paperbacks, as well as the ebook and audiobook. That's five. So you get eight entries to win and so forth. Anyway, to get your copy or copies of Fitness Science Explained, just head over to fitnesssciencebook.com now. Also, if you like what I am doing here on the podcast and elsewhere, definitely check out my sports nutrition company, Legion, which, thanks to the support of many people like you, is the leading brand of all natural sports supplements in the world. And we're on top because every ingredient and dose in every product is backed by peer-reviewed scientific research, every formulation is 100% transparent. There are no proprietary blends, for example,

Starting point is 00:03:46 and everything is naturally sweetened and flavored. So that means no artificial sweeteners, no artificial food dyes, which may not be as dangerous as some people would have you believe, but there is good evidence to suggest that having many servings of artificial sweeteners in particular every day for long periods of time may not be the best for your health. So while you don't need pills, powders, and potions to get into great shape, and frankly, most of them are virtually useless, there are natural ingredients that can help you lose fat, build muscle, and get healthy faster, and you will find the best of them in Legion's products. To check out everything we have to offer, including protein powders and protein bars,

Starting point is 00:04:31 pre-workout, post-workout supplements, fat burners, multivitamins, joint support, and more, head over to www.buylegion.com, B-U-Y Legion.com. And just to show how much I appreciate my podcast peeps, use the coupon code MFL at checkout, and you will save 20% on your entire first order. So again, if you appreciate my work and if you want to see more of it, and if you also want all natural evidence-based supplements that work, please do consider supporting Legion so I can keep doing what I love, like producing more podcasts like this. Chapter 1. Good Science, Bad Science. The Difference and Why It Matters. The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. Isaac Asimov

Starting point is 00:05:25 One month, media headlines blazon that research has confirmed that one food or another reduces your risk of cancer, diabetes, obesity, or some other nasty health condition. Hooray, you think. Time to load up on said food. Then, sometime later, after it has become a staple in your meal plans, the other shoe drops. New research refutes earlier findings and demonstrates that it actually increases your risk of disease and dysfunction. What the heck? How can scientific research just turn on a dime like that and do a full 180? Oh well, you think. A few months of eating this way can't have been that harmful. Life goes on. Then it happens again, and again, and again.

Starting point is 00:06:11 Eventually, you conclude that science can't seem to make up its mind on anything, and you stop paying attention. Fortunately, this isn't true. It may appear that there is a study to prove or disprove just about any assertion, but this illusion isn't the fault of science itself, but rather the widespread misunderstandings about the scientific process, media sensationalism, and sometimes even fraudulent research. Let's take a closer look at the nine main reasons that science can appear to be so confusing and contradictory. 1. Media misrepresentation. Attention spans are shorter than ever these days, and when news organizations have just a few hundred words or seconds to report on health matters, they can't afford to discuss the nuances of complicated scientific research. Instead,

Starting point is 00:07:03 they need titillating headlines and easily digested soundbites that draw eyeballs and clicks and bounce around in social media and water cooler conversations. That inevitably leads to misinformation. The two most common ways this occurs are 1. Confusing correlation with causation 2. Oversimplification and Sensationalism Let's go over each. Confusing Correlation with Causation

Starting point is 00:07:30 Quite a bit of health-related research is based on observational data, meaning that scientists observe groups of people going about their lives, collect various types of data, and then look for correlations between different variables. Correlation is a mutual relationship or connection between two or more things. For example, it was through observational research that the link between smoking and lung cancer was first discovered. In the famous British doctor's study of 1954, scientists sent out questionnaires to British doctors asking them about their smoking habits. The scientists then looked at which doctors got lung cancer

Starting point is 00:08:08 and found that doctors who reported smoking were more likely to get the disease. This type of research is a fantastic tool for documenting phenomena, forming hypotheses, and pointing the way for further research. But it can never be used to conclusively determine the cause of the phenomena observed because there are many ways for variables to be related without one causatively influencing the other. For instance, ice cream intake goes up in the summer as does the incidence of drowning. So, you could say that there's a strong correlation between eating ice cream and drowning. This does not mean that eating ice cream

Starting point is 00:08:45 causes people to drown, however, which is how your average media outlet might explain it. A good example of this is how the media has reported that drinking diet soda can make you fat, cause and effect, cut and dried. These warnings were based on research that showed that people who drank diet soda more often also tended to be more overweight, which may or may not be true. What if diet soda isn't causing weight gain, but instead obese people tend to switch to diet soda in hopes of losing weight? That is just one of a number of alternative hypotheses that could explain the correlation, and that's why further, more rigorous research is needed to identify the true cause. Well, in this case, that additional research has already been done, and scientists found that the correlation between obese people

Starting point is 00:09:38 and drinking diet soda was, in fact, due to their efforts to lose weight. In other words, diet soda was more popular among overweight people trying to lose weight because it contains fewer calories than regular soda. Furthermore, when it's used in this fashion to reduce overall calorie intake, diet soda consumption is associated with weight loss, not gain. Unfortunately, the media makes this type of mistake all the time. Studies show that news outlets tend to report on observational research more than randomized controlled trials, which can establish correlation, and which you'll learn more about soon, as well as lower-quality studies that should be taken with a grain of salt.

Starting point is 00:10:22 Oversimplification and Sensationalism The media will often oversimplify or distort the results of a study to make a nice, catchy, clickbait headline. Tim Caulfield of the University of Alberta has coined a term for this, scienceploitation. For example, a popular UK website once ran the headline, a glass of red wine is the equivalent to an hour at the gym, says new study, with a sub-headline of new research reveals skipping the gym in favor of the pub

Starting point is 00:10:52 is okay. Perfect, many people thought. Time to exercise less and drink more. If you actually read the scientific paper, though, you'll quickly realize that it isn't what the study found. paper, though, you'll quickly realize that it isn't what the study found. Instead, it found that a compound in grapes and red wine called resveratrol may increase exercise capacity, how well people or animals tolerate intense exercise, in rats, who are already exercising. There was no red wine involved in this study, and it never showed that people should stop working out. Another example of this is when the media reported on a study from the New England Journal of Medicine with headlines claiming that drinking coffee could make you live longer. However, not only did the media make the

Starting point is 00:11:36 mistake of confusing correlation and causation, they also failed to mention that the study only involved people who had already lived to at least 50 years of age, had no history of cancer, heart disease, or stroke, and didn't smoke. There were many other limitations to the study as well, which the scientists mentioned in the paper, but the media failed to report on. Why can the media get away with this? There are likely three reasons why this type of reporting continues unabated. Journalists often have little

Starting point is 00:12:06 formal training in science and thus are unable to ensure their stories are scientifically accurate. The general public also has little formal training in science and is thus unable to differentiate good from bad reporting. Sensationalism sells, so there's always an incentive for the media to spend scientific research in sensationalistic ways. Keep in mind that most of these organizations rely on advertising revenue to survive, and advertising revenue is driven by website visits. Thus, from a business perspective, writing articles that get a lot of clicks is far more important than being scientifically accurate, especially when it would reduce the virality of the content. 2. Cherry-picking versus going by the weight of the evidence. It's very common to have dozens or even hundreds of published studies on any given

Starting point is 00:12:56 topic, and in many cases, the results aren't all in agreement. Sometimes the differing or even contradictory results come from differences in how the studies were designed and executed. Sometimes shenanigans are afoot, and sometimes it's just random chance. This is why scientists consider the weight of the evidence available as opposed to the findings of a single study. Think of a scale, with one group of studies more or less in agreement on one side, and another group that indicates otherwise on the other side. The scale will favor whichever side has more evidence to support its assertion, which you could say is where the weight of the evidence lies. Thus, a good scientist will

Starting point is 00:13:36 say, given the weight of the evidence, this explanation is most likely true. Unfortunately, due mainly to ignorance, personal biases, and the media's love of controversy, research is often cherry-picked to make claims that go against the weight of the evidence. In other words, people often pick out and play up studies that they don't realize are flawed, that they just personally agree with, or that will make for great headlines. A perfect example of cherry-picking occurs among some of the more zealous advocates of low-carb dieting. They often hold up a few studies as definitive proof that low-carb diets are better for losing fat and claim there's no room left for discussion or debate. When you

Starting point is 00:14:17 peruse these studies though, you'll find glaring flaws in how they were carried out and when you collect and analyze all of the available research on the matter, you'll find there is no practical difference in fat loss between low- and high-carb diets, so long as calories and protein intake are matched. In other words, so long as people are eating the same amount of calories and protein, the amount of carbohydrate they're eating won't meaningfully impact their fat loss. In the final analysis, dietary adherence, not carb intake, is the biggest predictor of weight loss success. Thus, a scientist worth their salt would say, the weight of the evidence indicates that

Starting point is 00:14:56 there are no differences in fat loss between low and high carb diets, so long as calories are restricted and protein intake is adequate. Accordingly, individuals should choose the diet that they can best stick to for maximum results. Yup, the old adage is true. In many ways, the best weight loss diet is the one you can stick to. 3. Different Quality Levels of Studies As I mentioned earlier, there are often a large number of studies published on a particular topic, and some are better than others. There are many factors to consider when assessing the quality of a study, ranging from the type of research, observational or otherwise, it is to how well it's designed, how many participants there were, whether humans or animals were involved, and more.

Starting point is 00:15:41 Thus, when you're working to determine the weight of the evidence, you have to consider not only the number of studies on each side, but the quality as well. For example, if I have 10 studies with only 10 subjects each that points to one conclusion as well as two studies with a thousand subjects each that points to another conclusion, then the weight of the evidence lies with the latter, even though the former conclusion has more individual studies on its side. As you'll learn later in this course, sample size, which is the number of samples measured or observations used in a study, is a major determinant of the quality of research. A perfect example of how ignoring the quality of research can result in misleading conclusions is antioxidant supplementation. There's low quality evidence in the quality of research can result in misleading conclusions is antioxidant supplementation. There's low-quality evidence in the form of observational research and small-scale trials on animals and humans

Starting point is 00:16:33 that suggests antioxidant supplementation may reduce the risk of cancer. And high-quality research in the form of randomized clinical trials that shows antioxidant supplementation doesn't. Guess which research the media and mainstream health gurus decided to champion? Yep, the low-quality research, and antioxidant supplements started flying off the shelves. 4. Science moves slowly. Contradictions are a natural part of the scientific process. Many conclusions in science are tentative because they're based on the best evidence available at the time. However, as time moves on and as scientists accumulate more data and evidence, newer findings and understandings can overturn older ones. This is particularly true

Starting point is 00:17:18 when there's little data and evidence to begin with. A good example of this process is the story of butter versus margarine. Three decades ago, as evidence accumulated that the saturated fat in butter may be related to heart disease risk, scientists recommended that people switch to margarine to reduce their saturated fat intake. However, evidence then began to accumulate that the chemically modified fats, trans fat, and margarine were even worse than saturated fat in regard to heart disease risk. Based on this newer evidence, scientists revised their recommendations to continue to limit butter but also eliminate margarine and trans fats from diets. 5. Science often deals in shades of gray rather than black and white. Science is full of nuance, and therefore

Starting point is 00:18:05 research usually doesn't lend itself well to headlines and soundbites, which is what most people want. Simple, neat, black or white answers to their questions. Unfortunately, though, many scientific topics operate more in shades of gray, and especially when the evidence isn't strong. There is often a lot of uncertainty in the realm of science, which the general public finds uncomfortable. They don't want informed guesses. They want certainties that make their lives easier, and science is often unequipped to meet these demands. Moreover, the human body is fantastically complex, and some scientific answers can never be provided in black or white terms. All this is why the media tends to oversimplify scientific research when presenting it to the public.

Starting point is 00:18:51 In their eyes, they're just giving people what they want as opposed to offering more accurate but complex information that very few people will read or understand. A perfect example of this is how people want definitive answers as to which foods are good and bad. Scientifically speaking, there are no good and bad foods. Rather, food quality exists on a continuum, meaning that some foods are better than others when it comes to general health and well-being. Take sugar, a molecule that most people consider bad. In and of itself,

Starting point is 00:19:26 sugar, a molecule that most people consider bad. In and of itself, it's not a harmful substance, and one of its components is necessary for life, glucose. Research shows that when it's consumed in moderation as part of a calorie-controlled diet, it doesn't cause adverse health effects or fat gain. However, when sugar is added to highly processed foods to enhance their palatability and energy density, these foods become easier to overeat, and the resulting increase in calorie consumption and fat gain can become a health issue. That doesn't make for a good tweet or elevator pitch to a book publisher, though, and so the research on sugar tends to be separated into two buckets, one that shows it's good and another that shows it's bad. This creates the

Starting point is 00:20:06 illusion of incongruity when in fact, it's just a case of missing the forest for the trees. If you like what I'm doing here on the podcast and elsewhere, definitely check out my sports nutrition company Legion, which thanks to the support of many people like you is the leading brand of all natural sports supplements in the world. Six, lack of reproducibility slash replication. A very important concept in the realm of science is replication or reproducibility. For a scientific finding to be considered true, it needs to be reproduced, meaning that other scientists should be able to achieve

Starting point is 00:20:51 the same results by repeating the experiment. This is important because if other scientists can't replicate the results, then it's likely the initial results were a fluke. The media loves to report on hot new studies with new findings, but often such studies are small pilot experiments that have yet to be reproduced with larger sample sizes and better study designs. Often, later studies end up refuting the results of the original

Starting point is 00:21:17 breakthrough research, giving the appearance of conflicting evidence. In reality, the initial results were happenstance. This is why it's important to be cautious when viewing small studies with new or unusual findings. One of the greatest examples of this happened in the 1980s. Two scientists held a press conference, saying they'd been able to make atoms fuse at room temperature, cold fusion. However, they hadn't reproduced their results, and other scientists weren't able to reproduce the results either. By late 1989, most scientists considered the prospect of cold fusion dead. It's also important to take note of the labs conducting research. If one lab consistently produces a certain result, but other labs can't reproduce it, then the research coming from the former lab should be viewed with skepticism. For example, one lab has produced astounding muscle-building results with the supplement HMB,

Starting point is 00:22:10 but other labs haven't been able to reproduce anything close, which calls the positive results into question. 7. Poor Research Design Slash Execution Sometimes a study produces unusual results simply because it's poorly designed and executed. A perfect example of this is research out of the Ramazzini Institute in Italy that supposedly showed that aspartame caused cancer in rats. The research was heavily criticized by leading scientific organizations for having many flaws, including the fact that the control rodents had unusually high cancer rates, and the fact that when independent scientists asked to double-check the data, the institute flat-out refused. In most cases, organizations like this are outed among the scientific community,

Starting point is 00:22:57 but by that time, the story has already made its way through the media cycle, convincing many that once again, the scientific process doesn't make any sense. 8. Unpublished Research When scientists do a study, they collect the data, analyze it, write up the results, and submit the write-up to a scientific journal for publication. The study then goes through a process of peer review, which consists of other independent scientists reviewing it for flaws. Based on their findings, the study is either accepted for publication or rejected. The peer review process isn't without flaws, but it's the first line of defense against bad research getting published

Starting point is 00:23:36 and then propagated by the media. Thanks to peer review, if a study is published in a scientific journal, you can at least know it's gone through some type of quality control. This isn't the case with unpublished research. For example, scientists often present new research at conferences that has yet to be peer-reviewed or published. Sometimes the media catches wind of these reports and runs with them before they've gone through the peer-review process. And sometimes, scientists will themselves promote the findings of studies that haven't been peer reviewed or published. One case of this was on September 30, 2011, when Martin Lindstrom reported on his unpublished neuroimaging iPhone study in the New York Times.

Starting point is 00:24:17 He reported that people experience the same feelings of love in response to their iPhones ringing as they did in the company of their partners, best friends, or parents. Many scientists criticize Lindstrom, stating that his data didn't support such a conclusion. But, since Lindstrom had bypassed peer review, his dubious conclusions were all that most people ever heard or saw. Companies that sell products often report unpublished research as authoritative proof of their effectiveness.

Starting point is 00:24:46 You should be wary of such research because it hasn't been scrutinized by independent scientists and is often designed and executed in such a way as to guarantee positive results. For example, the creator of a cold exposure vest claimed that his product could help people burn up to 500 extra calories per day. This startling promise was based on research he conducted himself where people wore the vest for two weeks and lost fat. This trial was never peer-reviewed or published in any scientific journal, and if it had been submitted for review, it would have been rejected for egregious design flaws. For instance, the alleged increase in energy expenditure was based on unreliable estimates of body composition rather than direct, validated measurements of energy expenditure.

Starting point is 00:25:31 9. Fabricated Research Fabricated research essentially means research that's been made up. While fabricated research isn't nearly as common as everything else we've covered so far, it still exists and can lead to great confusion. Scientists may falsify data for a number of reasons, including to gain money, notoriety, and funding for further research, or merely to add another publication to their name. One of the most famous cases of fabricated research came from Andrew Wakefield. In 1988, he published a paper in the prestigious journal Lancet that showed an association between the measles, mumps, rubella,

Starting point is 00:26:10 MMR vaccine, and autism in children. However, it was later discovered that he had fabricated some of his data. Independent researchers discovered that Wakefield's descriptions of the children's medical cases differed from their actual medical records. Wakefield's paper was eventually retracted from the journal, but to this day his fraudulent research is still used to support the claim that vaccines may cause autism, despite numerous studies showing no such relationship. Scientific research can seem like a quagmire of misinformation, contradiction, and outright lies. When you look under the hood, though,

Starting point is 00:26:45 you quickly find that the media selectively picks studies designed to generate the most controversy, spins the findings for maximum dramatic effect, and withholds information about how they were conducted. In other cases, a shenanigans start before the studies hit your Facebook feed. Poor study designs skew the results and some scientists accidentally

Starting point is 00:27:06 or intentionally falsify their data. Despite all of that, it's still the best system we have for answering this simple question, what's probably true and what isn't? To understand how honest, intelligent researchers go about answering that question, we need to take a closer look at the scientific

Starting point is 00:27:25 method. Key takeaways. The media often misrepresents scientific studies by confusing correlation with causation and oversimplifying and sensationalizing data. The media gets away with this because journalists and the public often have little formal training in science, and sensationalism sells, so there's always an incentive for the media to spend scientific research. It's very common to have dozens or even hundreds of published studies on any given topic, and in many cases, the results aren't all in agreement. Unfortunately, due mainly to ignorance, personal biases, and the media's love of controversy, research is often cherry-picked to make claims that go against the weight of the evidence.

Starting point is 00:28:07 In other words, people often pick out and play up studies that they don't realize are flawed, that they just personally agree with, or that will make for great headlines. There are many factors to consider when assessing the quality of a study, ranging from the type of research, observational or otherwise,

Starting point is 00:28:23 it is to how well it's designed, how many participants there were, whether humans or animals were involved, and more. Thus, when you're working to determine the weight of the evidence, you have to consider not only the number of studies on each side, but the quality as well. Contradictions are a natural part of the scientific process. Many conclusions in science are tentative because they're based on the best evidence available at the time. However, as time moves on and as scientists accumulate more data and evidence, newer findings and understandings can overturn older ones. This is particularly true when there's little data and evidence to begin with. For a scientific finding to be considered true, it needs to be reproduced,

Starting point is 00:29:06 meaning that other scientists should be able to achieve the same results by repeating the experiment. This is important because if other scientists can't replicate the results, then it's likely the initial results were a fluke. Peer review consists of other independent scientists reviewing research for flaws. Based on their findings, the study is either accepted for publication or rejected. Scientists often present new research at conferences that has yet to be peer-reviewed or published. Sometimes the media catches wind of these reports and runs with them before they've gone through the peer-review process, and sometimes scientists will themselves promote the findings of studies that haven't been peer-reviewed or published.

Starting point is 00:29:45 Fabricated research isn't common, but it can lead to great confusion. Scientists may falsify data for a number of reasons, including to gain money, notoriety, and funding for further research, or merely to add another publication to their name. Despite its many flaws, the scientific process is still the best system we have for answering this simple question, what's probably true and what isn't? All right, well, that's it for this episode. I hope you enjoyed it and found it interesting and helpful. you don't mind doing me a favor, please do leave a quick review on iTunes or wherever you're listening to me from in whichever app you're listening to me in, because that not only

Starting point is 00:30:32 convinces people that they should check out the show, it also increases search visibility. And thus it helps more people find their way to me and learn how to get fitter, leaner, stronger, healthier, and happier as well. And of course, if you want to be notified when the next episode goes live, then simply subscribe to the podcast and you won't miss out on any new stuff. And if you didn't like something about the show, please do shoot me an email at mike at muscleforlife.com, just muscle, F-O-R, life.com, and share your thoughts on how I can do this better. I read everything myself and I'm always looking for

Starting point is 00:31:12 constructive feedback, even if it is criticism. I'm open to it. And of course, you can email me if you have positive feedback as well, or if you have questions really relating to anything that you think I could help you with, definitely send me an email. That is the best way to get ahold of me, mikeatmuscleforlife.com. And that's it. Thanks again for listening to this episode. And I hope to hear from you soon.

Muscle for Life with Mike Matthews - Good Science, Bad Science: The Difference and Why It Matters

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.