Health Care AI, Intended To Save Money, Turns Out To Require a Lot of Expensive Humans
10almonds is reader-supported. We may, at no cost to you, receive a portion of sales if you purchase a product through a link in this article.
Preparing cancer patients for difficult decisions is an oncologist’s job. They don’t always remember to do it, however. At the University of Pennsylvania Health System, doctors are nudged to talk about a patient’s treatment and end-of-life preferences by an artificially intelligent algorithm that predicts the chances of death.
But it’s far from being a set-it-and-forget-it tool. A routine tech checkup revealed the algorithm decayed during the covid-19 pandemic, getting 7 percentage points worse at predicting who would die, according to a 2022 study.
There were likely real-life impacts. Ravi Parikh, an Emory University oncologist who was the study’s lead author, told KFF Health News the tool failed hundreds of times to prompt doctors to initiate that important discussion — possibly heading off unnecessary chemotherapy — with patients who needed it.
He believes several algorithms designed to enhance medical care weakened during the pandemic, not just the one at Penn Medicine. “Many institutions are not routinely monitoring the performance” of their products, Parikh said.
Algorithm glitches are one facet of a dilemma that computer scientists and doctors have long acknowledged but that is starting to puzzle hospital executives and researchers: Artificial intelligence systems require consistent monitoring and staffing to put in place and to keep them working well.
In essence: You need people, and more machines, to make sure the new tools don’t mess up.
“Everybody thinks that AI will help us with our access and capacity and improve care and so on,” said Nigam Shah, chief data scientist at Stanford Health Care. “All of that is nice and good, but if it increases the cost of care by 20%, is that viable?”
Government officials worry hospitals lack the resources to put these technologies through their paces. “I have looked far and wide,” FDA Commissioner Robert Califf said at a recent agency panel on AI. “I do not believe there’s a single health system, in the United States, that’s capable of validating an AI algorithm that’s put into place in a clinical care system.”
AI is already widespread in health care. Algorithms are used to predict patients’ risk of death or deterioration, to suggest diagnoses or triage patients, to record and summarize visits to save doctors work, and to approve insurance claims.
If tech evangelists are right, the technology will become ubiquitous — and profitable. The investment firm Bessemer Venture Partners has identified some 20 health-focused AI startups on track to make $10 million in revenue each in a year. The FDA has approved nearly a thousand artificially intelligent products.
Evaluating whether these products work is challenging. Evaluating whether they continue to work — or have developed the software equivalent of a blown gasket or leaky engine — is even trickier.
Take a recent study at Yale Medicine evaluating six “early warning systems,” which alert clinicians when patients are likely to deteriorate rapidly. A supercomputer ran the data for several days, said Dana Edelson, a doctor at the University of Chicago and co-founder of a company that provided one algorithm for the study. The process was fruitful, showing huge differences in performance among the six products.
It’s not easy for hospitals and providers to select the best algorithms for their needs. The average doctor doesn’t have a supercomputer sitting around, and there is no Consumer Reports for AI.
“We have no standards,” said Jesse Ehrenfeld, immediate past president of the American Medical Association. “There is nothing I can point you to today that is a standard around how you evaluate, monitor, look at the performance of a model of an algorithm, AI-enabled or not, when it’s deployed.”
Perhaps the most common AI product in doctors’ offices is called ambient documentation, a tech-enabled assistant that listens to and summarizes patient visits. Last year, investors at Rock Health tracked $353 million flowing into these documentation companies. But, Ehrenfeld said, “There is no standard right now for comparing the output of these tools.”
And that’s a problem, when even small errors can be devastating. A team at Stanford University tried using large language models — the technology underlying popular AI tools like ChatGPT — to summarize patients’ medical history. They compared the results with what a physician would write.
“Even in the best case, the models had a 35% error rate,” said Stanford’s Shah. In medicine, “when you’re writing a summary and you forget one word, like ‘fever’ — I mean, that’s a problem, right?”
Sometimes the reasons algorithms fail are fairly logical. For example, changes to underlying data can erode their effectiveness, like when hospitals switch lab providers.
Sometimes, however, the pitfalls yawn open for no apparent reason.
Sandy Aronson, a tech executive at Mass General Brigham’s personalized medicine program in Boston, said that when his team tested one application meant to help genetic counselors locate relevant literature about DNA variants, the product suffered “nondeterminism” — that is, when asked the same question multiple times in a short period, it gave different results.
Aronson is excited about the potential for large language models to summarize knowledge for overburdened genetic counselors, but “the technology needs to improve.”
If metrics and standards are sparse and errors can crop up for strange reasons, what are institutions to do? Invest lots of resources. At Stanford, Shah said, it took eight to 10 months and 115 man-hours just to audit two models for fairness and reliability.
Experts interviewed by KFF Health News floated the idea of artificial intelligence monitoring artificial intelligence, with some (human) data whiz monitoring both. All acknowledged that would require organizations to spend even more money — a tough ask given the realities of hospital budgets and the limited supply of AI tech specialists.
“It’s great to have a vision where we’re melting icebergs in order to have a model monitoring their model,” Shah said. “But is that really what I wanted? How many more people are we going to need?”
KFF Health News is a national newsroom that produces in-depth journalism about health issues and is one of the core operating programs at KFF—an independent source of health policy research, polling, and journalism. Learn more about KFF.
Subscribe to KFF Health News’ free Morning Briefing.
This article first appeared on KFF Health News and is republished here under a Creative Commons license.
Don’t Forget…
Did you arrive here from our newsletter? Don’t forget to return to the email to continue learning!
Recommended
Learn to Age Gracefully
Join the 98k+ American women taking control of their health & aging with our 100% free (and fun!) daily emails:
-
Pumpkin Seeds vs Watermelon Seeds – Which is Healthier?
10almonds is reader-supported. We may, at no cost to you, receive a portion of sales if you purchase a product through a link in this article.
Our Verdict
When comparing pumpkin seeds to watermelon seeds, we picked the watermelon.
Why?
Starting with the macros: pumpkin seeds have a lot more carbs, while watermelon seeds have a lot more protein, despite pumpkin seeds being famous for such. They’re about equal on fiber. In terms of fats, watermelon seeds are higher in fats, and yes, these are healthy fats, mostly polyunsaturated.
When it comes to vitamins, pumpkin seeds are marginally higher in vitamins A and C, while watermelon seeds are a lot higher in vitamins B1, B2, B3, B5, B6, and B9. An easy win for watermelon seeds here.
In the category of minerals, despite being famous for zinc, pumpkin seeds are higher only in potassium, while watermelon seeds are higher in iron, magnesium, manganese, and phosphorus; the two seeds are equal on calcium, copper, and zinc. Another win for watermelon seeds.
In short, enjoy both, but watermelon has more to offer. Of course, if buying just the seeds and not the whole fruit, it’s generally easier to find pumpkin seeds than watermelon seeds, so do bear in mind that pumpkin seeds’ second place isn’t that bad here—it’s just a case of a very nutritious food looking bad by standing next to an even better one.
Want to learn more?
You might like to read:
Seed Saving Secrets – by Alice Mirren
Take care!
Share This Post
-
How Jumping Rope Changes The Human Body
10almonds is reader-supported. We may, at no cost to you, receive a portion of sales if you purchase a product through a link in this article.
Most popularly enjoyed by professional boxers and six-year-old girls, jumping rope is one of the most metabolism-boosting exercises around:
Just a hop, skip, and a jump away from good health
Maybe you haven’t tried it since your age was in single digits, so, if you do…
What benefits can you expect?
- Improves cardiovascular fitness, equivalent to 30 minutes of running with just 10 minutes of jumping.
- Increases bone density and boosts immunity by aiding the lymphatic system.
- Enhances explosiveness in the lower body, agility, and stamina.
- Improves shoulder endurance, coordination, and spatial awareness.
What kind of rope is best for you?
- Beginner ropes: licorice ropes (nylon/vinyl), beaded ropes for rhythm and durability.
- Advanced ropes: speed ropes (denser, faster materials) for higher speeds and more difficult skills.
- Weighted ropes: build upper body muscles (forearms, shoulders, chest, back).
What length should you get?
- Recommended rope length varies by height (8 ft for 5’0″–5’4″, 9 ft for 5’5″–5’11”, 10 ft for 6’0″ and above).
- Beginners should start with longer ropes for clearance.
What should you learn?
- Initial jump rope skills: start with manageable daily jump totals, gradually increasing as ankles, calves, and feet adapt.
- Further skills: learn the two-foot jump and then the boxer’s skip for efficient, longer sessions and advanced skills. Keep arms close and hands at waist level for a smooth swing.
For more on all of this, enjoy:
Click Here If The Embedded Video Doesn’t Load Automatically!
Want to learn more?
You might also like to read:
How To Do High Intensity Interval Training (Without Wrecking Your Body)
Take care!
Share This Post
-
Brussels Sprouts vs Spirulina – Which is Healthier?
10almonds is reader-supported. We may, at no cost to you, receive a portion of sales if you purchase a product through a link in this article.
Our Verdict
When comparing Brussels sprouts to spirulina, we picked the sprouts.
Why?
Pitting these two well-known superfoods against each other, we get the following:
Looking at the macros first, spirulina has a little more protein, while sprouts have more carbs and nearly 10x the fiber. So, we call this a win for sprouts.
In terms of vitamins, sprouts have a lot more of vitamins A, B6, B9, C, E, K, and choline, while spirulina has a little more of vitamins B1, B2, and B3. An easy win for sprouts.
In the category of minerals, sprouts have more calcium, magnesium, manganese, phosphorus, potassium, selenium, and zinc, while spirulina has more copper and iron. Another clear win for sprouts.
Adding up the sections makes the winner very clear: Brussels sprouts enjoy a well-earned victory.
Want to learn more?
You might like to read:
Take care!
Share This Post
Related Posts
-
Escape From The Clutches Of Shame
10almonds is reader-supported. We may, at no cost to you, receive a portion of sales if you purchase a product through a link in this article.
We’ve written before about managing various emotions, including “negative” ones. We put that in “scare quotes” because they also all have positive aspects, that are just generally overshadowed by the fact that the emotions themselves are not pleasant. But for example…
We evolved our emotions, including the “negative” ones, for our own benefit as a species:
- Stress keeps us safe by making sure we take important situations seriously
- Anger keeps us safe by protecting us from threats
- Disgust keeps us safe by helping us to avoid things that might cause disease
- Anxiety keeps us safe by ensuring we don’t get complacent
- Guilt keeps us safe by ensuring we can function as a community
- Sadness keeps us safe by ensuring we value things that are important to us, and learn to become averse to losing them
- …and so on
You can read more about how to turn these off (or rather, at least pause them) when they’re misfiring and/or just plain not convenient, here:
While it’s generally considered good to process feelings instead of putting them aside, the fact is that sometimes we have to hold it together while we do something, such that we can later have an emotional breakdown at a convenient time and place, instead of the supermarket or bank or office or airport or while entertaining houseguests or… etc.
Today, though, we’re not putting things aside, for the most part (though we will get to that too).
We’ll be dealing with shame, which is closely linked to the guilt we mentioned in that list there.
See also: Reconsidering the Differences Between Shame and Guilt
Shame’s purpose
Shame’s purpose is to help us (as a community) avoid anti-social behavior for which we might be shamed, and thus exiled from the in-group. It helps us all function better together, which is how we thrive as a species.
Shame, therefore, is often assumed to be something we can (and possibly should) use to ensure that we (ourselves and/or others) “do the right thing”.
But there’s a catch…
Shame only works negatively
You may be thinking “well duh, it’s a negative emotion”, but this isn’t about negativity in the subjective sense, but rather, positive vs negative motivation:
- Positive motivation: motivation that encourages us to do a given thing
- Negative motivation: motivation that encourages us to specifically not do a given thing
Shame is only useful as a negative motivation, i.e., encouraging us to specifically not do a given thing.
Examples:
- You cannot (in any way that sticks, at least) shame somebody into doing more housework.
- You can, however, shame somebody out of drinking and driving.
This distinction matters a lot when it comes to how we are with our children, or with our employees (or those placed under us in a management structure), or with people who otherwise look to us as leaders.
It also matters when it comes to how we are with ourselves.
Here’s a paper about this, by the way, with assorted real-world examples:
The negative side of motivation: the role of shame
From those examples, we can see that attempts to shame someone (including oneself) into doing something positive will generally not only fail, they will actively backfire, and people (including oneself) will often perform worse than pre-shaming.
Looking inwards: healthy vs unhealthy shame
Alcoholics Anonymous and similar programs use a degree of pro-social shame to help members abstain from the the act being shamed.
Rather than the unhelpful shame of exiling a person from a group for doing a shameful thing, however, they take an approach of laying out the shame for all to see, feeling the worst of it and moving past it, which many report as being quite freeing emotionally while still [negatively] motivational to not use the substance in question in the future (and similar for activity-based addictions/compulsions, such as gambling, for example).
As such, if you are trying to avoid doing a thing, shame can be a useful motivator. So by all means, if it’s appropriate to your goals, tell your friends/family about how you are now quitting this or that (be it an addiction, or just something generally unhealthy that you’d like to strike off your regular consumption/activity list).
You will still be tempted! But the knowledge of the shame you would feel as a result will help keep you from straying into that temptation.
If you are trying to do a thing, however, (even something thought of in a negative frame, such as “lose weight”), then shame is not helpful and you will do best to set it aside.
You can shame yourself out of drinking sodas (if that’s your plan), but you can’t shame yourself into eating healthy meals. And even if your plan is just shaming yourself out of eating unhealthy food… Without a clear active positive replacement to focus on instead, all you’ll do there is give yourself an eating disorder. You’ll eat nothing when people are looking, and then either a) also eat next to nothing in private or else b) binge in secret, and feel terrible about yourself, neither of which are any good for you whatsoever.
Similarly, you can shame yourself out of bed, but you can’t shame yourself into the gym:
Let it go
There are some cases, especially those where shame has a large crossover with guilt, that it serves no purpose whatsoever, and is best processed and then put aside.
For example, if you did something that you are ashamed of many years ago, and/or feel guilty about something that you did many years ago, but this is not an ongoing thing for you (i.e., it was a one-off bad decision, or a bad habit that have now long since dropped), then feeling shame and/or guilt about that does not benefit you or anyone else.
As to how to process it and put it aside, if your thing harmed someone else, you could see if there’s a way to try to make amends (even if without confessing ill, such as by acting anonymously to benefit the person/group you harmed).
And then, forgive yourself. Regardless of whether you feel like you deserve it. Make the useful choice, that better benefits you, and by extension those around you.
If you are religious, you may find that of help here too. We’re a health science publication not a theological one, but for example: Buddhism preaches compassion including for oneself. Judaism preaches atonement. Christianity, absolution. For Islam, mercy is one of the holiest ideals of the religion, along with forgiveness. So while religion isn’t everyone’s thing, for those for whom it is, it can be an asset in this regard.
For a more worldly approach:
To Err Is Human; To Forgive, Healthy (Here’s How To Do It) ← this goes for when the forgiveness in question is for yourself, too—and we do write about that there (and how)!
Take care!
Don’t Forget…
Did you arrive here from our newsletter? Don’t forget to return to the email to continue learning!
Learn to Age Gracefully
Join the 98k+ American women taking control of their health & aging with our 100% free (and fun!) daily emails:
-
Blood-Brain Barrier Breach Blamed For Brain-Fog
10almonds is reader-supported. We may, at no cost to you, receive a portion of sales if you purchase a product through a link in this article.
Move Over, Leaky Gut. Now It’s A Leaky Brain.
…which is not a headline that promises good news, and indeed, the only good news about this currently is “now we know another thing that’s happening, and thus can work towards a treatment for it”.
Back in February (most popular media outlets did not rush to publish this, as it rather goes against the narrative of “remember when COVID was a thing?” as though the numbers haven’t risen since the state of emergency was declared over), a team of Irish researchers made a discovery:
❝For the first time, we have been able to show that leaky blood vessels in the human brain, in tandem with a hyperactive immune system may be the key drivers of brain fog associated with long covid❞
~ Dr. Matthew Campbell (one of the researchers)
Let’s break that down a little, borrowing some context from the paper itself:
- the leaky blood vessels are breaching the blood-brain-barrier
- that’s a big deal, because that barrier is our only filter between our brain and Things That Definitely Should Not Go In The Brain™
- a hyperactive immune system can also be described as chronic inflammation
- in this case, that includes chronic neuroinflammation which, yes, is also a major driver of dementia
You may be wondering what COVID has to do with this, and well:
- these blood-brain-barrier breaches were very significantly associated (in lay terms: correlated, but correlated is only really used as an absolute in write-ups) with either acute COVID infection, or Long Covid.
- checking this in vitro, exposure of brain endothelial cells to serum from patients with Long Covid induced the same expression of inflammatory markers.
How important is this?
As another researcher (not to mention: professor of neurology and head of the school of medicine at Trinity) put it:
❝The findings will now likely change the landscape of how we understand and treat post-viral neurological conditions.
It also confirms that the neurological symptoms of long covid are measurable with real and demonstrable metabolic and vascular changes in the brain.❞
~ Dr. Colin Doherty (see mini-bio above)
You can read a pop-science article about this here:
Irish researchers discover underlying cause of “brain fog” linked with long covid
…and you can read the paper in full here:
Want to stay safe?
Beyond the obvious “get protected when offered boosters/updates” (see also: The Truth About Vaccines), other good practices include the same things most people were doing when the pandemic was big news, especially avoiding enclosed densely-populated places, washing hands frequently, and looking after your immune system. For that latter, see also:
Beyond Supplements: The Real Immune-Boosters!
Take care!
Don’t Forget…
Did you arrive here from our newsletter? Don’t forget to return to the email to continue learning!
Learn to Age Gracefully
Join the 98k+ American women taking control of their health & aging with our 100% free (and fun!) daily emails:
- the leaky blood vessels are breaching the blood-brain-barrier
-
Sweet Spot for Brain Health – by Dr. Sui Wong
10almonds is reader-supported. We may, at no cost to you, receive a portion of sales if you purchase a product through a link in this article.
At 10almonds we often mention that “what’s good for the heart is good for the brain”, but at least in part, it’s because (as this book makes very clear), “what’s good for the blood is good for the brain”. After all, our brain uses about 25% of our energy, and that energy is delivered there by the blood. And if it doesn’t get enough nutrients, oxygen, etc, and detritus isn’t taken away, then problems happen.
Dr. Wong discusses Alzheimer’s as heavily driven by metabolic problems such as diabetes and even pre-diabetes, and sets out to put in our hands the guidebook to not only not doing that, but also, actually making sure our brain gets proper nourishment without delivering that as intermittent sugar spikes because we opted for a something with very fast-acting carbs to perk us up energetically.
More than most books on the topic, she talks a lot about the neurobiology of glucose metabolism, so that’s something that really sets this book apart from many of its genre.
The style is narrative, explaining the body’s processes in a clear fashion, without skimping on science. There are definitely words that your average layperson might not know, but they’re explained as we go, and there are frequent recaps of what we learned previously, making for ultimately easy reading.
After all the information is given, there’s also a guided “12-week challenge” with a theme-of-the-week for each week, to integrate a new lifestyle adjustment each week in a progressive fashion so that without needing to drastically change many things at once, we get where we need to be in terms of healthy habits.
Bottom line: if you’d like to do right by your brain and while you’re at it say goodbye to blood sugar highs and lows, then this book is an excellent guide for that.
Click here to check out Sweet Spot For Brain Health, and enjoy a consistently-energized brain!
Don’t Forget…
Did you arrive here from our newsletter? Don’t forget to return to the email to continue learning!
Learn to Age Gracefully
Join the 98k+ American women taking control of their health & aging with our 100% free (and fun!) daily emails: