Researchers at New York University’s Langone Health academic medical center have developed a large linguistic model, now deployed at three of their hospitals, that predicts a 30-day risk of re-hospitalization. patients and other clinical outcomes.
WHY IT IMPORTANT
Coinciding with research published on Nature This week, the release of the NYUTron model’s codebase in GitHub allows other healthcare organizations to train their own LLMs and provide doctors with insights that can help them identify patients. who may need intervention to reduce readmissions.
This model was used to evaluate 50,000 patients discharged from the NYU health care system. NYUTron shares projections of re-hospitalization risk with physicians via email.
NYU has partnered with NVIDIA to develop and run its LLM on several of the company’s artificial intelligence platforms — its stack, libraries, and software.
“Not all hospitals have the resources to train a large language model in the first place, but they can apply a pre-trained model like NYUTron and then refine it,” said Dr. tune it with a small local data sample using the cloud GPU.” Eric Oermann, assistant professor of neurosurgery, radiology, and data science at NYU Langone Health, said in a blog post on NVIDIA’s website.
“That’s within reach of most people in healthcare,” he said.
“Oermann’s team found that after pre-training their LLM, refining it in situ with a specific hospital’s data significantly enhanced accuracy,” NVIDIA said in the announcement. mine.
NYUTron received 10 years of pre-trained health records from NYU Langone Health, including more than four billion words of clinical notes representing nearly 400,000 patients. Oermann said the team is using highly refined data-driven, mid-size models to complete healthcare-specific tasks.
The team then developed four other algorithms that predicted a patient’s length of stay, the likelihood of in-hospital death, and the likelihood that a patient’s insurance claim might be denied.
During a webinar last year, the researchers said their approach was to treat readmission prediction as a natural language processing task that involved generating a pre-trained LLM on the repositories. Medical system-scale clinical text on high-end multi-node GPU servers.
When NYUTron was developed, they said their goal was to address the following questions:
- How to handle long string lengths?
- How to solve label imbalance?
- How to evaluate the impact of noisy labels on model evaluation?
First, they “collected a large set of unlabeled clinical notes and five task-specific labeled clinical notes from the NYU Langone EHR,” before training the model, fine-tuning, deployed and tested it in a real-world environment, the researchers said in the report.
“On small samples, NYUTron competed with a small group of physicians in predicting re-hospitalization after 30 days,” they said.
Testing a group of six physicians of varying levels of seniority with NYUTron in a head-to-head comparison, they said they established a baseline difficulty in predicting all-cause readmissions at time point. discharge from hospital.
“Average physician performance is worse than that of NYUTron,” the researchers said.
“For physicians and NYUTron, the average false-positive rate was 11.11%, while the median true-positive rate was 50% for physicians compared with 81.82% for NYUTron. Doctors. average F1 score [a machine learning evaluation metric that measures a model’s accuracy] was 62.8% and significant variance was 22.2% compared with NYUTron, which had a mean F1 score of 77.8%.
At an NVIDIA advance press conference on Tuesday, Oermann also said that NYU Langone Health is considering licensing its models to organizations that don’t have the resources to build from scratch.
The next phase for Oermann’s group is a planned clinical trial to test whether interventions based on NYUTron analyzes reduce readmission rates.
TREND TO BIGGER WOMAN
Research posted on Multidisciplinary healthcare journal Last year, nearly 15% of all hospitalized patients were readmitted within 30 days of their initial discharge.
Rehospitalization rates are influenced by a multitude of variables – including five common treatments in emergency departments – that not only affect a patient’s overall care, but can also redirect beds and resources from patients who may have more specialized health care needs.
“Industry research as well as our own experience shows that up to 20 percent of readmissions can be prevented,” said Teresa Radford, clinical program coordinator at the University of Virginia Health.
She told Healthcare IT News in December, after finding that UVA Health’s 30-day readmission rate for patients with complex and costly medical conditions was as high as 17% to 18% per year, the provider Healthcare has created a home hospital program that reduces hospitalizations and readmissions by 46%.
A multidisciplinary team of doctors, nurses, and mental health professionals has created a special care plan for every individual with at least one chronic illness or behavioral condition that results in frequent have to go to a UVA Health hospital and use a variety of hospital services.
ON PROFILE
“Although there have been computational models to predict patient readmissions since the 1980s, we are treating this as a [NLP] The task requires a clinical text at the health system scale,” Oermann said in the announcement.
“We trained our LLM on the unstructured data of the electronic health record to see if it could capture insights that people hadn’t considered before.”
Andrea Fox is the senior editor of Healthcare IT News.
Email: [email protected]
Healthcare IT News is a publication of HIMSS Media.
