What is reformer pilates? And is it worth the cost?
10almonds is reader-supported. We may, at no cost to you, receive a portion of sales if you purchase a product through a link in this article.
Reformer pilates is steadily growing in popularity, with new studios opening regularly in major cities all over the world.
But what exactly is reformer pilates? And how does it compare with regular pilates and other types of exercise?
Classes aren’t cheap so let’s look at the potential benefits and drawbacks to help you decide if it’s right for you.
Pilates with special equipment
Pilates is a mode of exercise that focuses on core stability and flexibility, while also addressing muscular strength and endurance, balance and general fitness. At first glance, it might look a bit like yoga, with some more traditional weight training components thrown in.
Reformer pilates uses a piece of equipment called a “reformer”. This looks like a narrow bed that slides along a carriage, has straps to hold onto, and has adjustable springs that add resistance to movement. You perform pilates on the reformer to target specific muscle groups and movement patterns.
The reformer was first designed to help people recover from injuries. However, it has now become common for general fitness and even sports performance.
Unlike normal pilates, also known as “mat pilates”, which only uses your body weight, the reformer adds resistance, meaning you can change the difficulty according to your current level of fitness.
This not only provides a way to overload your muscles, but can make the exercise session more aerobically demanding, which has been proposed to improve cardiovascular fitness.
What are the benefits of reformer pilates?
Despite being around for decades, there is surprisingly little research looking at the benefits of reformer pilates. However, what we have seen so far suggests it has a similar effect to other modes of exercise.
Reformer pilates has been shown to help with weight loss, cause some small increases in muscle mass, and enhance cognitive function. All of these benefits are commonly seen when combining weight training and cardio into the same routine.
Similarly, among older adults, it has been shown to improve strength, enhance flexibility and may even reduce the risk of falling.
From a rehabilitation perspective, there is some evidence indicating reformer pilates can improve shoulder health and function, reduce lower back pain and increase flexibility.
Finally, there is some evidence suggesting a single session of reformer pilates can improve two key markers of cardiovascular health, being flow-mediated dilation and pulse wave velocity, while also improving cholesterol and insulin levels. This suggests reformer pilates could lead to long-term improvements in heart and metabolic health, although more research is needed to confirm this.
However, there are some key things to consider when discussing these benefits. Most of this research is quite exploratory and comes from a very small number of studies. So we do not know whether these findings will apply to everyone.
Very few studies compared reformer pilates to other types of exercise. Therefore, while it can improve most aspects of health and function, it’s unlikely reformer pilates provides the optimal mode of exercise for each individual component of physical fitness.
Traditional weight training, for example, will likely cause larger improvements in strength than reformer pilates. Similarly, stretching will probably make you more flexible. And running or cycling will make you fitter.
However, if you want a type of exercise that gives you broad overall health benefits, it could be a good option.
What are the downsides of reformer pilates
Reformer pilates is not for everyone.
First and foremost, classes can be expensive compared to other fitness options. You need to be doing at least two to three sessions per week of any type of exercise to maximise the benefits. So even if you can find a class for A$20 or $30, paying for two or three classes a week (or buying a weekly or monthly subscription) is a significant outlay.
Second, it’s not as accessible as other exercise. Even if you can afford it, not every town or suburb has a reformer pilates studio.
Third, the effectiveness of your workout is likely to be impacted by how competent your instructor is. There are a host of different pilates qualifications you can get in Australia, and some take much less time than others. With this in mind, it might be best to look for accredited pilates instructors, although this will further reduce the number of options you have available.
Finally, there is a learning curve. While you will get better over time, the exercise will likely be less effective during those first few weeks (or months) when you are getting used to the machine and the movements.
Is it right for you?
Reformer pilates can be a great addition to your fitness routine, especially if you’re looking for a low-impact way to build strength and flexibility.
But if you have more specific goals, you might need a more specific mode of exercise. For example, if you need to get stronger to improve your ability to manage your daily life, then strength training is probably your best bet. Likewise, if your goal is to run a marathon, you will get more specific benefits from running.
The cost and availability of reformer pilates make it less accessible for some people. With this in mind, if you are after similar benefits at a lower price point, mat pilates might be a better option. Not only does it have evidence suggesting it can improve strength and fitness, but it is something you can do at home if you find a good resource (YouTube could be a good starting point here).
Hunter Bennett, Lecturer in Exercise Science, University of South Australia; Jacinta Brinsley, Exercise Physiologist and Postdoctoral Researcher in the Alliance for Research in Nutrition, Exercise and Activity, University of South Australia, and Lewis Ingram, Lecturer in Physiotherapy, University of South Australia
This article is republished from The Conversation under a Creative Commons license. Read the original article.
Don’t Forget…
Did you arrive here from our newsletter? Don’t forget to return to the email to continue learning!
Learn to Age Gracefully
Join the 98k+ American women taking control of their health & aging with our 100% free (and fun!) daily emails:
-
Health Care AI, Intended To Save Money, Turns Out To Require a Lot of Expensive Humans
10almonds is reader-supported. We may, at no cost to you, receive a portion of sales if you purchase a product through a link in this article.
Preparing cancer patients for difficult decisions is an oncologist’s job. They don’t always remember to do it, however. At the University of Pennsylvania Health System, doctors are nudged to talk about a patient’s treatment and end-of-life preferences by an artificially intelligent algorithm that predicts the chances of death.
But it’s far from being a set-it-and-forget-it tool. A routine tech checkup revealed the algorithm decayed during the covid-19 pandemic, getting 7 percentage points worse at predicting who would die, according to a 2022 study.
There were likely real-life impacts. Ravi Parikh, an Emory University oncologist who was the study’s lead author, told KFF Health News the tool failed hundreds of times to prompt doctors to initiate that important discussion — possibly heading off unnecessary chemotherapy — with patients who needed it.
He believes several algorithms designed to enhance medical care weakened during the pandemic, not just the one at Penn Medicine. “Many institutions are not routinely monitoring the performance” of their products, Parikh said.
Algorithm glitches are one facet of a dilemma that computer scientists and doctors have long acknowledged but that is starting to puzzle hospital executives and researchers: Artificial intelligence systems require consistent monitoring and staffing to put in place and to keep them working well.
In essence: You need people, and more machines, to make sure the new tools don’t mess up.
“Everybody thinks that AI will help us with our access and capacity and improve care and so on,” said Nigam Shah, chief data scientist at Stanford Health Care. “All of that is nice and good, but if it increases the cost of care by 20%, is that viable?”
Government officials worry hospitals lack the resources to put these technologies through their paces. “I have looked far and wide,” FDA Commissioner Robert Califf said at a recent agency panel on AI. “I do not believe there’s a single health system, in the United States, that’s capable of validating an AI algorithm that’s put into place in a clinical care system.”
AI is already widespread in health care. Algorithms are used to predict patients’ risk of death or deterioration, to suggest diagnoses or triage patients, to record and summarize visits to save doctors work, and to approve insurance claims.
If tech evangelists are right, the technology will become ubiquitous — and profitable. The investment firm Bessemer Venture Partners has identified some 20 health-focused AI startups on track to make $10 million in revenue each in a year. The FDA has approved nearly a thousand artificially intelligent products.
Evaluating whether these products work is challenging. Evaluating whether they continue to work — or have developed the software equivalent of a blown gasket or leaky engine — is even trickier.
Take a recent study at Yale Medicine evaluating six “early warning systems,” which alert clinicians when patients are likely to deteriorate rapidly. A supercomputer ran the data for several days, said Dana Edelson, a doctor at the University of Chicago and co-founder of a company that provided one algorithm for the study. The process was fruitful, showing huge differences in performance among the six products.
It’s not easy for hospitals and providers to select the best algorithms for their needs. The average doctor doesn’t have a supercomputer sitting around, and there is no Consumer Reports for AI.
“We have no standards,” said Jesse Ehrenfeld, immediate past president of the American Medical Association. “There is nothing I can point you to today that is a standard around how you evaluate, monitor, look at the performance of a model of an algorithm, AI-enabled or not, when it’s deployed.”
Perhaps the most common AI product in doctors’ offices is called ambient documentation, a tech-enabled assistant that listens to and summarizes patient visits. Last year, investors at Rock Health tracked $353 million flowing into these documentation companies. But, Ehrenfeld said, “There is no standard right now for comparing the output of these tools.”
And that’s a problem, when even small errors can be devastating. A team at Stanford University tried using large language models — the technology underlying popular AI tools like ChatGPT — to summarize patients’ medical history. They compared the results with what a physician would write.
“Even in the best case, the models had a 35% error rate,” said Stanford’s Shah. In medicine, “when you’re writing a summary and you forget one word, like ‘fever’ — I mean, that’s a problem, right?”
Sometimes the reasons algorithms fail are fairly logical. For example, changes to underlying data can erode their effectiveness, like when hospitals switch lab providers.
Sometimes, however, the pitfalls yawn open for no apparent reason.
Sandy Aronson, a tech executive at Mass General Brigham’s personalized medicine program in Boston, said that when his team tested one application meant to help genetic counselors locate relevant literature about DNA variants, the product suffered “nondeterminism” — that is, when asked the same question multiple times in a short period, it gave different results.
Aronson is excited about the potential for large language models to summarize knowledge for overburdened genetic counselors, but “the technology needs to improve.”
If metrics and standards are sparse and errors can crop up for strange reasons, what are institutions to do? Invest lots of resources. At Stanford, Shah said, it took eight to 10 months and 115 man-hours just to audit two models for fairness and reliability.
Experts interviewed by KFF Health News floated the idea of artificial intelligence monitoring artificial intelligence, with some (human) data whiz monitoring both. All acknowledged that would require organizations to spend even more money — a tough ask given the realities of hospital budgets and the limited supply of AI tech specialists.
“It’s great to have a vision where we’re melting icebergs in order to have a model monitoring their model,” Shah said. “But is that really what I wanted? How many more people are we going to need?”
KFF Health News is a national newsroom that produces in-depth journalism about health issues and is one of the core operating programs at KFF—an independent source of health policy research, polling, and journalism. Learn more about KFF.
Subscribe to KFF Health News’ free Morning Briefing.
This article first appeared on KFF Health News and is republished here under a Creative Commons license.
Share This Post
-
Fix Tight Hamstrings In Just 3 Steps
10almonds is reader-supported. We may, at no cost to you, receive a portion of sales if you purchase a product through a link in this article.
There’s a better way to increase your flexibility than just stretching and stretching and hoping for the best. Here’s a 3-step method that will transform your flexibility:
As easy as 1-2-3
Only one part actually involves stretching:
Step 1: reciprocal inhibition
- Concept: when one muscle contracts, the opposing muscle relaxes—which is what we need.
- Goal: engage hip flexors to encourage hamstring relaxation.
- Method:
- Kneeling hamstring stretch position with one leg forward.
- Support with yoga blocks or a chair; use a cushion for comfort.
- Maintain a slight arch in the lower back and hinge forward slightly.
- Attempt to lift the foot off the floor, even if it doesn’t move.
- Hold for around 10 seconds.
Step 2: engaging more muscle fibers
- Concept: our muscles contain a lot of fibers, and often not all of them come along for the ride when we do something (exercising, stretching, etc), and those fibers that weren’t engaged will hold back the whole process.
- Goal: activate more fibers in the hamstring for a deeper stretch.
- Method:
- Same kneeling position, slight back arch, and forward hinge.
- Drive the heel into the floor as if trying to dent it.
- Apply significant effort but hold for only 10 seconds.
- A small bend in the knee is acceptable.
Step 3: manipulating the nervous system
- Concept: the nervous system often limits flexibility due to safety signals (causing sensations of discomfort to tell us to stop a lot sooner than we really need to).
- Goal: passive stretching to reduce nervous system resistance.
- Method:
- Avoid muscle engagement or movement—stay completely relaxed.
- Focus on calmness, with slow, steady breaths.
- Avoid signs of tension (e.g. clenched fists, short/sharp breathing). While your nervous system is trying to communicate to you that you are in danger, you need to communicate to your nervous system that this is fine actually, so in order to reassure your nervous system you need to avoid signs that will tip it off that you’re worried too.
- Don’t overstretch; prioritize a relaxed, safe feeling.
For more on all of this, plus visual demonstrations, enjoy:
Click Here If The Embedded Video Doesn’t Load Automatically!
Want to learn more?
You might also like:
Tight Hamstrings? Here’s A Test To Know If It’s Actually Your Sciatic Nerve
Take care!
Share This Post
-
What’s Missing from Medicine – by Dr. Saray Stancic
10almonds is reader-supported. We may, at no cost to you, receive a portion of sales if you purchase a product through a link in this article.
Another from the ranks of “doctors who got a serious illness and it completely changed how they view the treatment of serious illness”, Dr. Stancic was diagnosed with multiple sclerosis, and wasn’t impressed with the treatments presented.
Taking an evidence-based lifestyle medicine approach, she was able to not only manage her illness sufficiently to resume her normal activities, but even when so far as to run a marathon, and today boasts a symptom-free, active life.
The subtitular six lifestyle changes are not too shocking, and include a plants-centric diet, good exercise, good sleep, stress management, avoidance of substance abuses, and a fostering of social connections, but the value here is in what she has to say about each, especially the ones that aren’t so self-explanatory and/or can even cause harm if done incorrectly (such as exercise, for example).
The style is on the academic end of pop-science, of the kind that has frequent data tables, lots of statistics, and an extensive bibliography, but is still very readable.
Bottom line: if you are faced with a chronic disease, or even just an increased risk of some chronic disease, or simply like to not take chances, then this is a high-value book for you.
Click here to check out What’s Missing From Medicine, and enjoy chronic good health!
Share This Post
-
4 ways to cut down on meat when dining out – and still make healthy choices
10almonds is reader-supported. We may, at no cost to you, receive a portion of sales if you purchase a product through a link in this article.
Many of us are looking for ways to eat a healthier and more sustainable diet. And one way to do this is by reducing the amount of meat we eat.
That doesn’t mean you need to become a vegan or vegetarian. Our recent research shows even small changes to cut down on meat consumption could help improve health and wellbeing.
But not all plant-based options are created equal and some are ultra-processed. Navigating what’s available when eating out – including options like tofu and fake meats – can be a challenge.
So what are your best options at a cafe or restaurant? Here are some guiding principles to keep in mind when cutting down on meat.
Mikhaylovskiy/Shutterstock Health benefits to cutting down
Small amounts of lean meat can be part of a healthy, balanced diet. But the majority of Australians still eat more meat than recommended.
Only a small percentage of Australians (10%) are vegetarian or vegan. But an increasing number opt for a flexitarian diet. Flexitarians eat a diet rich in fruits and vegetables, while still enjoying small amounts of meat, dairy, eggs and fish.
Our recent research looked at whether the average Australian diet would improve if we swapped meat and dairy for plant-based alternatives, and the results were promising.
The study found health benefits when people halved the amount of meat and dairy they ate and replaced them with healthy plant-based foods, like tofu or legumes. On average, their dietary fibre intake – which helps with feeling fuller for longer and digestive health – went up. Saturated fats – which increase our blood cholesterol levels, a risk factor for heart disease – went down.
Including more fibre and less saturated fat helps reduce the risk of heart disease.
Achieving these health benefits may be as simple as swapping ham for baked beans in a toastie for lunch, or substituting half of the mince in your bolognese for lentils at dinner.
Filling your plate with fibre-rich foods can help lower cholesterol. Wally Pruss/Shutterstock How it’s made matters
For a long time we’ve known processed meats – such as ham, bacon and sausages – are bad for your health. Eating high amounts of these foods is associated with poor heart health and some forms of cancer.
But the same can be true of many processed meat alternatives.
Plant-based alternatives designed to mimic meat, such as sausages and burgers, have become readily available in supermarkets, cafes and restaurants. These products are ultra-processed and can be high in salt and saturated fat.
Our study found when people replaced meat and dairy with ultra-processed meat alternatives – such as plant-based burgers or sausages – they ate more salt and less calcium, compared to eating meat or healthy plant-based options.
So if you’re cutting down on meat for health reasons, it’s important to think about what you’re replacing it with. The Australian Dietary Guidelines recommend eggs, legumes/beans, tofu, nuts and seeds.
Tofu can be a great option. But we recommend flavouring plain tofu with herbs and spices yourself, as pre-marinated products are often ultra-processed and can be high in salt.
What about when dining out?
When you’re making your own food, it’s easier to adapt recipes or reduce the amount of meat. But when faced with a menu, it can be difficult to work out what is the best option.
Eating a range of colours is one way to ensure variety. Mikhail Nilov/Pexels Here are our four ways to make healthy choices when you eat out:
1. Fill half your plate with vegetables
When cutting down on meat, aim for half your plate to be vegetables. Try to also eat a variety of colours, such as leafy green spinach, red capsicum and pumpkin.
When you’re out, this might look like choosing a vegetable-based entree, a stir-fry or ordering a side salad to have with your meal.
2. Avoid the deep fryer
The Australian Dietary Guidelines recommend limiting deep fried foods to once a week or less. When dining out, choose plant-based options that are sautéed, grilled, baked, steamed, boiled or poached – instead of those that are crumbed or battered before deep frying.
This could mean choosing vegetarian dumplings that are steamed not fried, or poached eggs at brunch instead of fried. Ordering a side of roast vegetables instead of hot chips is also a great option.
3. Pick wholegrains
Scan the menu for wholegrain options such as brown rice, wholemeal pizza or pasta, barley, quinoa or wholemeal burger buns. Not only are they good sources of protein, but they also provide more dietary fibre than refined grains, which help keep you fuller for longer.
4. If you do pick meat – choose less processed kinds
You may not always want, or be able, to make a vegetarian choice when eating out and with other people. If you do opt for meat, it’s better to steer clear of processed options like bacon or sausages.
If sharing dishes with other people, you could try adding unprocessed plant-based options into the mix. For example, a curry with lentils or chickpeas, or a vegetable-based pizza instead of one with ham or salami. If that’s not an option, try choose meat that’s a lean cut, such as chicken breast, or options which are grilled rather than fried.
Laura Marchese, PhD candidate at the Institute for Physical Activity and Nutrition, Deakin University and Katherine Livingstone, NHMRC Emerging Leadership Fellow and Senior Research Fellow at the Institute for Physical Activity and Nutrition, Deakin University
This article is republished from The Conversation under a Creative Commons license. Read the original article.
Don’t Forget…
Did you arrive here from our newsletter? Don’t forget to return to the email to continue learning!
Learn to Age Gracefully
Join the 98k+ American women taking control of their health & aging with our 100% free (and fun!) daily emails:
-
Mythbusting The Big O
10almonds is reader-supported. We may, at no cost to you, receive a portion of sales if you purchase a product through a link in this article.
“Early To Bed…”
In yesterday’s newsletter, we asked you for your (health-related) views on orgasms.
But what does the science say?
Orgasms are essential to good health: True or False?
False, in the most literal sense. One certainly won’t die without them. Anorgasmia (the inability to orgasm) is a condition that affects many postmenopausal women, some younger women, and some men. And importantly, it isn’t fatal—just generally considered unfortunate:
Anorgasmia Might Explain Why You’re Not Orgasming When You Want To
That article focuses on women; here’s a paper focusing on men:
Orgasms are good for the health, but marginally: True or False?
True! They have a wide array of benefits, depending on various factors (including, of course, one’s own sex). That said, the benefits are so marginal that we don’t have a flock of studies to cite, and are reduced to pop-science sources that verbally cite studies that are, alas, nowhere to be found, for example:
- For women: 9 Orgasm Benefits That Might Surprise You
- For men: 9 Ways Orgasms May Benefit Your Health
Doubtlessly the studies do exist, but are sparse enough that finding them is a nightmare as the keywords for them will bring up a lot of studies about orgasms and health that aren’t answering the above question (usually: health’s affect on orgasms, rather than the other way around).
There is some good science for post-menopausal women, though! Here it is:
Misconceptions About Sexual Health in Older Women
(if you have the time to read this, this also covers many very avoidable things that can disrupt sexual function, in ways that people will errantly chalk up to old age, not knowing that they are missing out needlessly)
Orgasms are good or bad, depending on being male or female: True or False
False, broadly. The health benefits are extant and marginal for almost everyone, as indicated above.
What’s that “almost” about, then?
There are a very few* people (usually men) for whom it doesn’t go well. In such cases, they have a chronic and lifelong problem whereby orgasm is followed by 2–7 days of flu-like and allergic symptoms. Little is known about it, but it appears to be some sort of autoimmune disorder.
Read more: Post-orgasmic illness syndrome: history and current perspectives
*It’s hard to say for sure how few though, as it is surely under-reported and thus under-diagnosed; likely even misdiagnosed if the patient doesn’t realize that orgasms are the trigger for such episodes, and the doctor doesn’t think to ask. Instead, they will be busy trying to eliminate foods from the diet, things like that, while missing this cause.
Orgasms are better avoided for optimal health: True or False?
Aside from the above, False. There is a common myth for men of health benefits of “semen retention”, but it is not based in science, just tradition. You can read a little about it here:
The short version is: do it if you want; don’t if you don’t; the body will compensate either way so it won’t make a meaningful difference to anything for most people, healthwise.
Small counterpoint: while withholding orgasm (and ejaculation) is not harmful to health, what does physiologically need draining sometimes is prostate fluid. But that can also be achieved mechanically through prostate milking, or left to fend for itself (as it will in nocturnal emissions, popularly called wet dreams). However, if you have problems with an enlarged prostate, it may not be a bad idea to take matters into your own hands, so to speak. As ever, do check with your doctor if you have (or think you may have) a condition that might affect this.
One final word…
We’re done with mythbusting for today, but we wanted to share this study that we came across (so to speak) while researching, as it’s very interesting:
On which note: if you haven’t already, consider getting a “magic wand” style vibe; you can thank us later (this writer’s opinion: everyone should have one!).
Top tip: do get the kind that plugs into the wall, not rechargeable. The plug-into-the-wall kind are more powerful, and last much longer (both “in the moment”, and in terms of how long the device itself lasts).
Enjoy!
Don’t Forget…
Did you arrive here from our newsletter? Don’t forget to return to the email to continue learning!
Learn to Age Gracefully
Join the 98k+ American women taking control of their health & aging with our 100% free (and fun!) daily emails:
-
Health Care AI, Intended To Save Money, Turns Out To Require a Lot of Expensive Humans
10almonds is reader-supported. We may, at no cost to you, receive a portion of sales if you purchase a product through a link in this article.
Preparing cancer patients for difficult decisions is an oncologist’s job. They don’t always remember to do it, however. At the University of Pennsylvania Health System, doctors are nudged to talk about a patient’s treatment and end-of-life preferences by an artificially intelligent algorithm that predicts the chances of death.
But it’s far from being a set-it-and-forget-it tool. A routine tech checkup revealed the algorithm decayed during the covid-19 pandemic, getting 7 percentage points worse at predicting who would die, according to a 2022 study.
There were likely real-life impacts. Ravi Parikh, an Emory University oncologist who was the study’s lead author, told KFF Health News the tool failed hundreds of times to prompt doctors to initiate that important discussion — possibly heading off unnecessary chemotherapy — with patients who needed it.
He believes several algorithms designed to enhance medical care weakened during the pandemic, not just the one at Penn Medicine. “Many institutions are not routinely monitoring the performance” of their products, Parikh said.
Algorithm glitches are one facet of a dilemma that computer scientists and doctors have long acknowledged but that is starting to puzzle hospital executives and researchers: Artificial intelligence systems require consistent monitoring and staffing to put in place and to keep them working well.
In essence: You need people, and more machines, to make sure the new tools don’t mess up.
“Everybody thinks that AI will help us with our access and capacity and improve care and so on,” said Nigam Shah, chief data scientist at Stanford Health Care. “All of that is nice and good, but if it increases the cost of care by 20%, is that viable?”
Government officials worry hospitals lack the resources to put these technologies through their paces. “I have looked far and wide,” FDA Commissioner Robert Califf said at a recent agency panel on AI. “I do not believe there’s a single health system, in the United States, that’s capable of validating an AI algorithm that’s put into place in a clinical care system.”
AI is already widespread in health care. Algorithms are used to predict patients’ risk of death or deterioration, to suggest diagnoses or triage patients, to record and summarize visits to save doctors work, and to approve insurance claims.
If tech evangelists are right, the technology will become ubiquitous — and profitable. The investment firm Bessemer Venture Partners has identified some 20 health-focused AI startups on track to make $10 million in revenue each in a year. The FDA has approved nearly a thousand artificially intelligent products.
Evaluating whether these products work is challenging. Evaluating whether they continue to work — or have developed the software equivalent of a blown gasket or leaky engine — is even trickier.
Take a recent study at Yale Medicine evaluating six “early warning systems,” which alert clinicians when patients are likely to deteriorate rapidly. A supercomputer ran the data for several days, said Dana Edelson, a doctor at the University of Chicago and co-founder of a company that provided one algorithm for the study. The process was fruitful, showing huge differences in performance among the six products.
It’s not easy for hospitals and providers to select the best algorithms for their needs. The average doctor doesn’t have a supercomputer sitting around, and there is no Consumer Reports for AI.
“We have no standards,” said Jesse Ehrenfeld, immediate past president of the American Medical Association. “There is nothing I can point you to today that is a standard around how you evaluate, monitor, look at the performance of a model of an algorithm, AI-enabled or not, when it’s deployed.”
Perhaps the most common AI product in doctors’ offices is called ambient documentation, a tech-enabled assistant that listens to and summarizes patient visits. Last year, investors at Rock Health tracked $353 million flowing into these documentation companies. But, Ehrenfeld said, “There is no standard right now for comparing the output of these tools.”
And that’s a problem, when even small errors can be devastating. A team at Stanford University tried using large language models — the technology underlying popular AI tools like ChatGPT — to summarize patients’ medical history. They compared the results with what a physician would write.
“Even in the best case, the models had a 35% error rate,” said Stanford’s Shah. In medicine, “when you’re writing a summary and you forget one word, like ‘fever’ — I mean, that’s a problem, right?”
Sometimes the reasons algorithms fail are fairly logical. For example, changes to underlying data can erode their effectiveness, like when hospitals switch lab providers.
Sometimes, however, the pitfalls yawn open for no apparent reason.
Sandy Aronson, a tech executive at Mass General Brigham’s personalized medicine program in Boston, said that when his team tested one application meant to help genetic counselors locate relevant literature about DNA variants, the product suffered “nondeterminism” — that is, when asked the same question multiple times in a short period, it gave different results.
Aronson is excited about the potential for large language models to summarize knowledge for overburdened genetic counselors, but “the technology needs to improve.”
If metrics and standards are sparse and errors can crop up for strange reasons, what are institutions to do? Invest lots of resources. At Stanford, Shah said, it took eight to 10 months and 115 man-hours just to audit two models for fairness and reliability.
Experts interviewed by KFF Health News floated the idea of artificial intelligence monitoring artificial intelligence, with some (human) data whiz monitoring both. All acknowledged that would require organizations to spend even more money — a tough ask given the realities of hospital budgets and the limited supply of AI tech specialists.
“It’s great to have a vision where we’re melting icebergs in order to have a model monitoring their model,” Shah said. “But is that really what I wanted? How many more people are we going to need?”
KFF Health News is a national newsroom that produces in-depth journalism about health issues and is one of the core operating programs at KFF—an independent source of health policy research, polling, and journalism. Learn more about KFF.
Subscribe to KFF Health News’ free Morning Briefing.
This article first appeared on KFF Health News and is republished here under a Creative Commons license.
Don’t Forget…
Did you arrive here from our newsletter? Don’t forget to return to the email to continue learning!
Learn to Age Gracefully
Join the 98k+ American women taking control of their health & aging with our 100% free (and fun!) daily emails:
