Starting with electrocardiogram algorithms used in healthcare settings, Dandelion Health will run the algorithms on its dataset – including 10 million patients – to measure mental performance and bias. artificial intelligence across major racial, ethnic, and geographic subgroups.
WHY IT IMPORTANT
Dandelion is aggregating and de-identifying clinical data through a consortium of three major health systems — Sharp HealthCare in San Diego, Sanford Health in Sioux Falls, South Dakota, and Texas Health Resources in Arlington, Texas — for authentication. algorithms, according to today’s announcement.
The early-stage, New York-based startup, also known as the Dandelion Health Data Consortium, says it has developed this free program that can provide overall performance metrics and metrics data to pre-assigned groups of interest according to patient privacy, publicity. – access model.
Dandelion says the first service of its kind addresses the fundamental challenge of validating the performance of algorithms against intended populations – ensuring high-quality clinical data sets that include subgroups different populations and putting patient privacy first causes high costs and difficulty to find.
The company is committed to making it accessible to as many algorithm developers working on predictive analytics for cardiology using electrocardiogram waveforms as possible.
Any researcher or company with the legal right to do so can upload their ECG algorithm to the HIPAA and SOC-2 compliant platform and specify the performance metrics they want to query. .
The user will contract to protect their IP address, pre-process the algorithm on the provided sample data, write about 10 lines of wrapper code with the template, submit the container, and confirm that they are ready accuracy. Dandelion then runs the algorithm on ~500,000 records, collects and computes the results, and sends the report.
“When an algorithm is trained on datasets from Cambridge or Palo Alto, it is then applied to other populations – a different geography, a different care setting, or a group of patients of different race and ethnicity. different races – the performance of the algorithm can drop significantly,” said Elliott Green, CEO and co-founder of Dandelion Health.
“It’s important to know how algorithms work, both overall and for the most vulnerable groups of patients, who are often underrepresented in the algorithm training dataset. our testers answer the question, is your algorithm doing what it’s supposed to? and does it do it fairly, for everyone?”
Dandelion said that the names of the researchers and participating companies will never be made public and that the results will only be made available to the algorithm’s developers unless they license it to others.
It also said Sendhil Mullainathan, co-founder and advisor and Roman Family University Professor of Computational and Behavioral Sciences at the Chicago Pavilion, and Dr. Ziad Obermeyer, co-founder and chief science officer, and is an associate professor and Blue Cross of California Distinguished Professor at Berkeley Public Health, who will oversee the free AI validation program.
Dandelion will begin accepting ECG waveform algorithms designed to represent the US population for validation from developers worldwide starting July 15 for an initial period of three month. Future plans beyond the pilot include the addition of clinical data modalities such as clinical notes and radiographic images.
TREND TO BIGGER
With the rapid growth, development, and integration of AI tools in healthcare, the U.S. Food and Drug Administration, the Health AI Alliance, and other organizations have called for ways to method to rigorously measure and validate the safety and effectiveness of algorithms.
At HIMSS23, Sonya Makhni, medical director of the Mayo Clinic Platform, discussed the importance of platform-based authentication and collaboration in developing and implementing effective healthcare AI. On the clinic’s development platform, collaborators agree to data standards and retain ownership of their data as they work together to train and validate the ecosystem, she explains.
With access to 10.4 million de-identified vertical patient records, Mayo Clinic facilitates external innovation at a much faster pace, she said. That platform is implementing validated algorithms – such as EKG algorithms – and integrating them into clinic workflows.
Sharp Healthcare and Sanford Health, the largest rural health system in the United States, joined the Dandelion Health Data Consortium in August to build diversity in data science.
“People with great ideas can’t access research-quality, AI-ready data in a safe, principled way. This means fundamentally transformable AI products work hard. clinical squirrel will not be created,” Obermeyer said last year.
ON PROFILE
“There is particular interest in algorithms that input an electrocardiogram’s waveform — which tests the heart’s function by measuring its electrical activity — and make predictions about arrhythmias, values,” says Obermeyer. lab or the results of other, more expensive tests like echocardiography.” statement.
“Questions about data quality and patient privacy have become a global conversation among the scientific and healthcare communities, governments, and the public alike. The goal of the trial program Our experience is to make it easier for developers to better understand the overall performance of an algorithm – and for specific interest groups – using scientifically rigorous validation to protect privacy patient’s privacy and in accordance with the common interest.”
Andrea Fox is the senior editor of Healthcare IT News.
Email: [email protected]
Healthcare IT News is a publication of HIMSS Media.
