How Science News Outlets Can Lie To You (Yes, Even If They Cite Studies!)

10almonds is reader-supported. We may, at no cost to you, receive a portion of sales if you purchase a product through a link in this article.

Each Monday, we’re going to be bringing you cutting-edge research reviews to not only make your health and productivity crazy simple, but also, constantly up-to-date.

But today, in this special edition, we want to lay out plain and simple how to see through a lot of the tricks used not just by popular news outlets, but even sometimes the research publications themselves.

That way, when we give you health-related science news, you won’t have to take our word for it, because you’ll be able to see whether the studies we cite really support the claims we make.

Of course, we’ll always give you the best, most honest information we have… But the point is that you shouldn’t have to trust us! So, buckle in for today’s special edition, and never have to blindly believe sci-hub (or Snopes!) again.

The above now-famous Tumblr post that became a meme is a popular and obvious example of how statistics can be misleading, either by error or by deliberate spin.

But what sort of mistakes and misrepresentations are we most likely to find in real research?

Spin Bias

Perhaps most common in popular media reporting of science, the Spin Bias hinges on the fact that most people perceive numbers in a very “fuzzy logic” sort of way. Do you?

Try this:

  • A million seconds is 11.5 days
  • A billion seconds is not weeks, but 13.2 months!

…just kidding, it’s actually nearly thirty-two years.

Did the months figure seem reasonable to you, though? If so, this is the same kind of “human brains don’t do large numbers” problem that occurs when looking at statistics.

Let’s have a look at reporting on statistically unlikely side effects for vaccines, as an example:

  • “966 people in the US died after receiving this vaccine!” (So many! So risky!)
  • “Fewer than 3 people per million died after receiving this vaccine!” (Hmm, I wonder if it is worth it?)
  • “Half of unvaccinated people with this disease die of it” (Oh)

How to check for this: ask yourself “is what’s being described as very common really very common?”. To keep with the spiders theme, there are many (usually outright made-up) stats thrown around on social media about how near the nearest spider is at any given time. Apply this kind of thinking to medical conditions.. If something affects only 1% of the population (So few! What a tiny number!), how far would you have to go to find someone with that condition? The end of your street, perhaps?

Selection/Sampling Bias

Diabetes disproportionately affects black people, but diabetes research disproportionately focuses on white people with diabetes. There are many possible reasons for this, the most obvious being systemic/institutional racism. For example, advertisements for clinical trial volunteer opportunities might appear more frequently amongst a convenient, nearby, mostly-white student body. The selection bias, therefore, made the study much less reliable.

Alternatively: a researcher is conducting a study on depression, and advertises for research subjects. He struggles to get a large enough sample size, because depressed people are less likely to respond, but eventually gets enough. Little does he know, even the most depressed of his subjects are relatively happy and healthy compared with the silent majority of depressed people who didn’t respond.

See This And Many More Educational Cartoons At Sketchplanations.com!

How to check for this: Does the “method” section of the scientific article describe how they took pains to make sure their sample was representative of the relevant population, and how did they decide what the relevant population was?

Publication Bias

Scientific publications will tend to prioritise statistical significance. Which seems great, right? We want statistically significant studies… don’t we?

We do, but: usually, in science, we consider something “statistically significant” when it hits the magical marker of p=0.05 (in other words, the probability of getting that result is 1/20, and the results are reliably coming back on the right side of that marker).

However, this can result in the clinic stopping testing once p=0.05 is reached, because they want to have their paper published. (“Yay, we’ve reached out magical marker and now our paper will be published”)

So, you can think of publication bias as the tendency for researchers to publish ‘positive’ results.

If it weren’t for publication bias, we would have a lot more studies that say “we tested this, and here are our results, which didn’t help answer our question at all”—which would be bad for the publication, but good for science, because data is data.

To put it in non-numerical terms: this is the same misrepresentation as the technically true phrase “when I misplace something, it’s always in the last place I look for it”—obviously it is, because that’s when you stop looking.

There’s not a good way to check for this, but be sure to check out sample sizes and see that they’re reassuringly large.

Reporting/Detection/Survivorship Bias

There’s a famous example of the rise in “popularity” of left-handedness. Whilst Americans born in ~1910 had a bit under a 3.5% chance of being left handed, those born in ~1950 had a bit under a 12% change.

Why did left-handedness become so much more prevalent all of a sudden, and then plateau at 12%?

Simple, that’s when schools stopped forcing left-handed children to use their right hands instead.

In a similar fashion, countries have generally found that homosexuality became a lot more common once decriminalized. Of course the real incidence almost certainly did not change—it just became more visible to research.

So, these biases are caused when the method of data collection and/or measurement leads to a systematic error in results.

How to check for this: you’ll need to think this through logically, on a case by case basis. Is there a reason that we might not be seeing or hearing from a certain demographic?

And perhaps most common of all…

Confounding Bias

This is the bias that relates to the well-known idea “correlation ≠ causation”.

Everyone has heard the funny examples, such as “ice cream sales cause shark attacks” (in reality, both are more likely to happen in similar places and times; when many people are at the beach, for instance).

How can any research paper possibly screw this one up?

Often they don’t and it’s a case of Spin Bias (see above), but examples that are not so obviously wrong “by common sense” often fly under the radar:

“Horse-riding found to be the sport that most extends longevity”

Should we all take up horse-riding to increase our lifespans? Probably not; the reality is that people who can afford horses can probably afford better than average healthcare, and lead easier, less stressful lives overall. The fact that people with horses typically have wealthier lifestyles than those without, is the confounding variable here.

See This And Many More Educational Cartoons on XKCD.com!

In short, when you look at the scientific research papers cited in the articles you read (you do look at the studies, yes?), watch out for these biases that found their way into the research, and you’ll be able to draw your own conclusions, with well-informed confidence, about what the study actually tells us.

Science shouldn’t be gatekept, and definitely shouldn’t be abused, so the more people who know about these things, the better!

So…would one of your friends benefit from this knowledge? Forward it to them!

Don’t Forget…

Did you arrive here from our newsletter? Don’t forget to return to the email to continue learning!

Recommended

  • Gymnema Sylvestre: The “Sugar Destroyer”
  • Lower Cholesterol Naturally
    Cholesterol conundrum? Learn how lifestyle choices like stress management, exercise, and heart-healthy diets can safely lower your LDL levels without statins.

Learn to Age Gracefully

Join the 98k+ American women taking control of their health & aging with our 100% free (and fun!) daily emails:

  • Dandelion Greens vs Collard Greens – Which is Healthier?

    10almonds is reader-supported. We may, at no cost to you, receive a portion of sales if you purchase a product through a link in this article.

    Our Verdict

    When comparing dandelion greens to collard greens, we picked the dandelion greens.

    Why?

    Collard greens are great—they even beat kale in one of our previous “This or That” articles!—but dandelion greens simply pack more of a nutritional punch:

    In terms of macros, dandelions have slightly more carbs (+3g/100g) for the same protein and fiber, and/but the glycemic index is equal (zero), so those carbs aren’t anything to worry about. Nobody is getting metabolic disease by getting their carbs from dandelion leaves. In short, we’re calling it a tie on macros, though it could nominally swing either way if you have an opinion (one way or the other) about the extra 3g of carbs.

    In the category of vitamins, things are more exciting: dandelion greens have more of vitamins A, B1, B2, B3, B6, B7, B9, C, E, and K, while collard greens have more vitamin B5. An easy and clear win for dandelions.

    Looking at the minerals tells a similar story; dandelion greens have much more calcium, copper, iron, magnesium, phosphorus, potassium, and zinc, while collard greens have slightly more manganese. Another overwhelming win for dandelions.

    One more category, polyphenols. We’d be here until next week if we listed all the polyphenols that dandelion greens have, but suffice it to say, dandelion greens have a total of 385.55mg/100g polyphenols, while collard greens have a total of 0.08mg/100g polyphenols. Grabbing a calculator, we see that this means dandelions have more than 4819x the polyphenol content that collard greens do.

    So, “eat leafy greens” is great advice, but they are definitely not all created equal!

    Let us take this moment to exhort: if you have any space at home where you can grow dandelions, grow them!

    Not only are they great for pollinators, but also they beat the collard greens that beat kale. And you can have as much as you want, for free, right there.

    Want to learn more?

    You might like to read:

    Collard Greens vs Kale – Which is Healthier?

    Enjoy!

    Share This Post

  • New study suggests weight loss drugs like Ozempic could help with knee pain. Here’s why there may be a link

    10almonds is reader-supported. We may, at no cost to you, receive a portion of sales if you purchase a product through a link in this article.

    The drug semaglutide, commonly known by the brand names Ozempic or Wegovy, was originally developed to help people with type 2 diabetes manage their blood sugar levels.

    However, researchers have discovered it may help with other health issues, too. Clinical trials show semaglutide can be effective for weight loss, and hundreds of thousands of people around the world are using it for this purpose.

    Evidence has also shown the drug can help manage heart failure and chronic kidney disease in people with obesity and type 2 diabetes.

    Now, a study published in the New England Journal of Medicine has suggested semaglutide can improve knee pain in people with obesity and osteoarthritis. So what did this study find, and how could semaglutide and osteoarthritis pain be linked?

    Pormezz/Shutterstock

    Osteoarthritis and obesity

    Osteoarthritis is a common joint disease, affecting 2.1 million Australians. Most people with osteoarthritis have pain and find it difficult to perform common daily activities such as walking. The knee is the joint most commonly affected by osteoarthritis.

    Being overweight or obese is a major risk factor for osteoarthritis in the knee. The link between the two conditions is complex. It involves a combination of increased load on the knee, metabolic factors such as high cholesterol and high blood sugar, and inflammation.

    For example, elevated blood sugar levels increase the production of inflammatory molecules in the body, which can damage the cartilage in the knee, and lead to the development of osteoarthritis.

    Weight loss is strongly recommended to reduce the pain of knee osteoarthritis in people who are overweight or obese. International and Australian guidelines suggest losing as little as 5% of body weight can help.

    But losing weight with just diet and exercise can be difficult for many people. One study from the United Kingdom found the annual probability of people with obesity losing 5% or more of their body weight was less than one in ten.

    Semaglutide has recently entered the market as a potential alternative route to weight loss. It comes from a class of drugs known as GLP-1 receptor agonists and works by increasing a person’s sense of fullness.

    Semaglutide for osteoarthritis?

    The rationale for the recent study was that while we know weight loss alleviates symptoms of knee osteoarthritis, the effect of GLP-1 receptor agonists was yet to be explored. So the researchers set out to understand what effect semaglutide might have on knee osteoarthritis pain, alongside body weight.

    They randomly allocated 407 people with obesity and moderate osteoarthritis into one of two groups. One group received semaglutide once a week, while the other group received a placebo. Both groups were treated for 68 weeks and received counselling on diet and physical activity. At the end of the treatment phase, researchers measured changes in knee pain, function, and body weight.

    As expected, those taking semaglutide lost more weight than those in the placebo group. People on semaglutide lost around 13% of their body weight on average, while those taking the placebo lost around 3% on average. More than 70% of people in the semaglutide group lost at least 10% of their body weight compared to just over 9% of people in the placebo group.

    A man outdoors holding his knee.
    Osteoarthritis of the knee is the most common type of osteoarthritis. SKT Studio/Shutterstock

    The study found semaglutide reduced knee pain significantly more than the placebo. Participants who took semaglutide reported an additional 14-point reduction in pain on a 0–100 scale compared to the placebo group.

    This is much greater than the pain reduction in another recent study among people with obesity and knee osteoarthritis. This study investigated the effects of a diet and exercise program compared to an attention control (where participants are provided with information about nutrition and physical activity). The results here saw only a 3-point difference between the intervention group and the control group on the same scale.

    The amount of pain relief reported in the semaglutide trial is also larger than that reported with commonly used pain medicines such as anti-inflammatories, opioids and antidepressants.

    Semaglutide also improved knee function compared to the placebo. For example, people who took semaglutide could walk about 42 meters further than those on the placebo in a six-minute walking test.

    How could semaglutide reduce knee pain?

    It’s not fully clear how semaglutide helps with knee pain from osteoarthritis. One explanation may be that when a person loses weight, there’s less stress on the joints, which reduces pain.

    But recent studies have also suggested semaglutide and other GLP-1 receptor agonists might have anti-inflammatory properties, and could even protect against cartilage wear and tear.

    While the results of this new study are promising, it’s too soon to regard semaglutide as a “miracle drug” for knee osteoarthritis. And as this study was funded by the drug company that makes semaglutide, it will be important to have independent studies in the future, to confirm the findings, or not.

    The study also had strict criteria, excluding some groups, such as those taking opioids for knee pain. One in seven Australians seeing a GP for their knee osteoarthritis are prescribed opioids. Most participants in the trial were white (61%) and women (82%). This means the study may not fully represent the average person with knee osteoarthritis and obesity.

    It’s also important to consider semaglutide can have a range of side effects, including gastrointestinal symptoms and fatigue.

    There are some concerns that semaglutide could reduce muscle mass and bone density, though we’re still learning more about this.

    Further, it can be difficult to access.

    I have knee osteoarthritis, what should I do?

    Osteoarthritis is a disease caused by multiple factors, and it’s important to take a multifaceted approach to managing it. Weight loss is an important component for those who are overweight or obese, but so are other aspects of self-management. This might include physical activity, pacing strategies, and other positive lifestyle changes such as improving sleep, healthy eating, and so on.

    Giovanni E. Ferreira, NHMRC Emerging Leader Research Fellow, Institute of Musculoskeletal Health, University of Sydney and Christina Abdel Shaheed, Associate Professor, School of Public Health, University of Sydney

    This article is republished from The Conversation under a Creative Commons license. Read the original article.

    Share This Post

  • The Art and Science of Connection – by Kasley Killam, MPH

    10almonds is reader-supported. We may, at no cost to you, receive a portion of sales if you purchase a product through a link in this article.

    We can eat well, exercise well, and even sleep well, and we’ll still have a +53% increased all-cause mortality if we lack social connection—even if we technically have support and access to social resources, just not the real human connection itself. And as we get older, it gets increasingly easy to find ourselves isolated.

    The author is a social scientist by profession, and it shows. None of what she shares in the book is wishy-washy; it has abundant scientific references coming thick and fast, and a great deal of clarity with regard to terms, something often not found in books of this genre that lean more towards the art than the science.

    On which note, for the reader who may be thinking “I am indeed quite alone”, she also offers proven techniques for remedying that; not in the way that many books use the word “proven” to mean “we got some testimonials”, but rather, proven in the sense of “we did science to it and based on these 17 large population-based retrospective cohort studies, we can say with 99% confidence that this is an effective tool to mediate improved social bonds and social health outcomes”.

    To this end, it’s a very practical book also, and should bestow upon any isolated reader a sense of confidence that in fact, things can be better. A particular strength is that it also looks at many different scenarios, so for the “what if I…” people with clear reasons why social connection is not abundantly available, yes, she has such cases covered too.

    Bottom line: if you’d like to live more healthily for longer, social health is an underrated and oft-forgotten way of greatly increasing those things, by science.

    Click here to check out The Art And Science Of Social Connection, and get connected!

    Share This Post

Related Posts

  • Gymnema Sylvestre: The “Sugar Destroyer”
  • Hello Sleep – by Dr. Jade Wu

    10almonds is reader-supported. We may, at no cost to you, receive a portion of sales if you purchase a product through a link in this article.

    We’ve reviewed other sleep books before, so what makes this one stand out?

    Mostly, it’s because this one takes quite a different approach.

    While still giving a nod to the sensible advice you’ve already read in many places (including here at 10almonds), Dr. Wu looks to help the reader avoid falling into the trap (or: help the reader get out of the trap, if already there) of focussing so much on getting better sleep that it becomes an all-consuming stressor that takes up much of the day thinking about it, and guess what, much of the night too, because you’re busy working out how sleep-deprived you’re going to be tomorrow.

    Instead, Dr. Wu recommends to work with your body rather than against it, worry less, and ultimately sleep better. Of course, the “how” of this is what makes most of the book.

    She does also give chapters on things that may be different for you, based on such things as hormones, age, or medical conditions.

    The writing style is pop-science but with frequent references to scientific papers as appropriate, making good science very accessible.

    Bottom line: if you’ve tried everything else and/but good sleep still eludes you, this book will help you to end the battle and make friends with your sleep (a metaphor the author uses throughout the book, by the way).

    Click here to check out Hello Sleep, and indeed get better sleep!

    Don’t Forget…

    Did you arrive here from our newsletter? Don’t forget to return to the email to continue learning!

    Learn to Age Gracefully

    Join the 98k+ American women taking control of their health & aging with our 100% free (and fun!) daily emails:

  • Artichoke vs Brussels Sprouts – Which is Healthier?

    10almonds is reader-supported. We may, at no cost to you, receive a portion of sales if you purchase a product through a link in this article.

    Our Verdict

    When comparing artichoke to Brussels sprouts, we picked the sprouts.

    Why?

    Finally, a vegetable that beats artichoke—after it previously beat heart-of-palm, asparagus, and even cabbage! It was still close though, which is impressive for artichoke, considering what a nutritional powerhouse Brussels sprouts are:

    In terms of macros, the only meaningful difference is that artichoke has slightly more carbs and fiber, so artichoke gets the most marginal of nominal wins in this category.

    In the category of vitamins, however, artichoke has more of vitamins B3, B9, and choline, while Brussels sprouts have more of vitamins A, B1, B2, B6, C, E, and K, giving sprouts the clear victory here, especially with much higher margins of difference (e.g. 58x more vitamin A, as well as 7x more vitamin C, and 10x more vitamin K).

    When it comes to minerals, artichoke has more copper, magnesium, phosphorus, and zinc, while Brussels sprouts have more iron, manganese, potassium, and selenium, resulting in a 4:4 tie, and the small margins of difference are mostly comparable, with the exception that sprouts have 8x more selenium. So, Brussels sprouts win this category very marginally on that tie-breaker.

    Adding up the sections we see that macros and minerals gave a small win each to artichoke and sprouts respectively, while the vitamins category was an overwhelming win for sprouts, so—with this deciding factor in mind—sprouts win the day today.

    Want to learn more?

    You might like to read:

    Sprout Your Seeds, Grains, Beans, Etc

    Enjoy!

    Don’t Forget…

    Did you arrive here from our newsletter? Don’t forget to return to the email to continue learning!

    Learn to Age Gracefully

    Join the 98k+ American women taking control of their health & aging with our 100% free (and fun!) daily emails:

  • Ending Aging – by Dr. Aubrey de Grey

    10almonds is reader-supported. We may, at no cost to you, receive a portion of sales if you purchase a product through a link in this article.

    We know about how to slow aging. We know about diet, exercise, sleep, intermittent fasting, and other lifestyle tweaks to make. But how much can we turn back the clock, according to science?

    Dr. Aubrey de Grey’s foundational principle is simple: the body is a biological machine, and aging is fundamentally an engineering problem.

    He then outlines the key parts to that problem: the princple ways in which cells (and DNA) get damaged, and what we need to do about that in each case. Car tires get damaged over time; our approach is to replace them within a certain period of time so that they don’t blow out. In the body, it’s a bit similar with cells so that we don’t get cancer, for example.

    The book goes into detail regards each of the seven main ways we accumulate this damage, and highlights avenues of research looking to prevent it, and in at least some cases, the measures already available to so.

    Bottom line: if you want a hard science overview of actual rejuvenation research in biogerontology, this is a book that presents that comprehensively, without assuming prior knowledge.

    Click here to check out Ending Aging and never stop learning!

    Don’t Forget…

    Did you arrive here from our newsletter? Don’t forget to return to the email to continue learning!

    Learn to Age Gracefully

    Join the 98k+ American women taking control of their health & aging with our 100% free (and fun!) daily emails: