Aaron Patzer

As the discourse shifts from models to compound AI systems / agents, we need better AI benchmarks to evaluate multi-modal and multi-step task performance, especially in healthcare and life sciences. When we wrote the first paper demonstrating ChatGPT's performance on the USMLE, we chose the US Medical Licensing Exam as a benchmark for accessibility, speed, and ease. This benchmark was never intended to represent AI model performance on real-world clinical tasks. Today, I still see so many research teams and startups using benchmarks (like the USMLE) that are ill-fitted for assessing the true clinical or scientific performance and utility of the models they are developing for real-world contexts. Benchmark development may be seen as a "less sexy" area of research, but it is of paramount importance. Years after the rise of the transformer, we still lack adequate benchmarks for so many single-step tasks in biomedicine. With compound AI systems (i.e., architectures that integrate multiple AI models to perform complex tasks) emerging, we need new benchmarks for agentic behaviors. I'd argue that developing an agent with novel capabilities without at least proposing a companion benchmark (if an industry standard does not yet exist) may hinder the adoption of said agent, especially for high-stakes workflows. Designing more benchmarks that capture/simulate real-world clinical and scientific workflows will help us mitigate the major discrepancies observed between in silico and in vivo performance and better support safe + effective deployment of AI in biomedicine. There are already brilliant people focused here, and we need more. DMs are open if you're researching or working in this area of multi-step/multi-modal benchmarking in healthcare and life sciences! #healthcare #ai #artificialintelligence #generativeai

100

18 Comments

With healthcare AI deals growing twice as fast as in other tech areas, as Rebecca Torrence highlights here, so does the need for high-quality, updated training data. Our team is proud to be at the forefront of solutions that enable AI companies to compliantly and quickly leverage sensitive data. #HealthcareAI #HealthcareData #AI

21

1 Comment

Here’s your recap of last week’s health IT news 🗞️ 👇 🤖 AI Products • Emergency Services, Inc. + Augmedix Go ED: The emergency medicine group adopted the new ED-specific AI ambient scribe solution.    • Color + OpenAI: The cancer care tech company partnered with the GenAI giant to build a AI clinical decision support copilot.    • AKASA Medical Coding: The AI RCM vendor known for its AI prior auth tool AKASA Authorization Advisor released a new GenAI medical coding solution.    • Humata Health: The AI prior auth for providers vendor closed a $25M funding round led by healthcare org-backed funds The Blue Venture Fund and LRVHealth.    • Wellsheet + Concord Hospital Health System: The AI-enabled Smart EHR UI solution announced a client-reported 16.3% reduction in length of stay and 40% decrease in time spent in the EHR via its partnership with Concord Health, as well as the release of its new LLM-generated handoff summary feature. 🍎 VBC • BSIM Healthcare Services + Innovaccer: The healthcare provider—embarking on its first VBC contract—adopted the pop health data platform for analytics. 🩼 Durable Medical Equipment • Parachute Health: The DME e-prescribing and ordering solution released an update that integrates prior authorization. ⚕️Clinical Staffing • Matchwell + Indiana Hospital Association: The clinical staffing service partnered with the hospital member org to create the first state-wide resource pool for temp staffing. 🧑‍💻 Virtual and At-Home Care • Deaconess Health System + KeyCare: The Indiana-based health system adopted the Epic-based virtual care provider for around-the-clock virtual urgent care.    • UC Davis Health + Current Health: The health system launched a new RPM program for high blood pressure in partnership with Best Buy Health’s care-at-home platform. 🧠 Food for Thought: We came across a couple of good resources this past week on the current state of healthcare data information exchange: • This article in Health Affairs from Deven McGraw and Tina Grande provides a comprehensive overview of some of the patient health data abuses we've seen come to light in recent months, as well as a number of policy recommendations to improve the health information exchange landscape for all participants. https://lnkd.in/e_GEQ-JJ  (h/t Alya Sulaiman)    • This episode of Health Gorilla's InteropTalk gives a helpful update on the current state of affairs, with a particular focus on the state of trust across the health data exchange community and how the definition of "Treatment" may evolve. https://lnkd.in/e96AJG-5 (h/t Brendan Keeler) --- Reminder: you can sign up for the Elion Briefing to get the latest healthcare tech news delivered to your inbox each week 📬 👉 https://lnkd.in/e2UJ3ckP

35

2 Comments

The defining factor isn't the size of the company or startup, but the vision behind it. Take OpenAI, for instance. Until last year, unless you were deeply embedded in the ML world as a computer scientist/researcher, or enthusiast, you might not have heard of OpenAI. Meanwhile, a giant like Google was dominating the scene. However, a few years back, OpenAI was just another small startup. Now, it's rapidly evolving into a major player in big tech. Why and how? It comes down to having a long-term vision, a clear roadmap, patience, and solid execution (i.e., talents). Even in the context of healthcare AI, where (small) startups face many challenges (including the ones mentioned), the principles of vision and strategy apply. While merging with big tech might seem like an option for some, if a startup truly believes in its vision and has a well-defined path to achieve it, it should forge ahead independently. The landscape is evolving, and like OpenAI, today’s small startups with clear, visionary goals could be tomorrow's industry leaders. #starup #founders #openai #google #healthcare

3

Big healthcare AI news yesterday as Tempus AI received the first-ever CPT code for an algorithm-only analysis of previously-sequenced, laboratory developed test transcriptomic data. TLDR: AI molecular phenotyping has a new business model. The PurISTSM test uses RNA profiling to identify molecular subtypes in patients with advanced pancreatic ductal adenocarcinoma, which may inform first-line therapy selection and management. While I'm skeptical that payer balance sheets are going to solve the "who pays for healthcare AI" problem, the reimbursement pipes for clinical AI are maturing. I am surprised to see more progress for usage-based vs. performance-based models given the many challenges with FFS economics. It will be interesting to see how payers assess the "medical necessity" and therefore reimbursability of clinical AI as these models come to market. https://t.co/eLe18SmihV

396

9 Comments

Every time a brand name investor posts a rallying cry “time to fix the $4T US healthcare”, many entrepreneurs will pay attention. There will be a rush into healthcare, primarily in the HIT field, and, lately, AI. What is important to understand, is that US healthcare is not a $4T market in terms of TAM/SAM calculations. It is a thousand small markets, none of which would be attractive to VC capital by itself, if thoroughly understood and well-defined. There are hundreds of incumbents in this highly fractured environment and some of them had monopolised their niches. While this is true, the Healthcare overall expense of 20% of GDP is unsustainable. How do you look at this spend empirically, sans the hype? Look for horizontal underlying patterns and market-spanning issues and yes, one has to be on top of regulatory space and learn it as well as their technology. All these are huge barriers to entry, but absolutely required for developing long term healthcare solutions. One of these horizontal and paramount issues is amount of healthcare data produced, collected and exchanged. Of total amount of data produced in the world now, 30% is generated by healthcare. 97% was of that data is unused. That should be a rallying cry.

26

1 Comment

Grab a megapack of the hottest HLTH takeaways — By top healthcare innovators: (see more in the presentation ↓ ) Personally,  I was impressed by the HLTH scale. The agenda, networking, and expo were a blast. And connecting with healthcare forward-thinkers? The greatest pleasure of all. 👇 Here’re some worthy ideas to dig into: Christian Hein →  Healthtech marking positive trends in care. Carolina Correa →  We need to rethink the balance of tech/human roles. Jeremy Dähn →  Data & transparency are key drivers of healthcare change. Shez Partovi →  GenAI should be used for patient empowerment. Arnaud WILMET → The risks of not adopting AI outweigh adoption risks. Blas Diaz → AI needs a patient-centric focus for impact. Christopher Nial → UK healthcare is ready to embrace AI benefits. Ben Richardson →  Change management in AI adoption is the key. Eve Cunningham MD MBA →  Virtual care requires digital infrastructure transformation. Natalia Karbasova → Tech-driven prevention is key to wellness. Dr. Sven Jansen → The new healthtech era lies in holistic health. Dr. Felicitas Muth → Value-based care is impossible without patient focus. Tanja Tervonen →  To succeed, nail the 4 R’s for US commercialization. Christele Berthelot-Logsdon → A neglected strategy: incentivize payors to help startups. Roxanne King →  Focus on AI, chronic disease, health equity. Marty Gordon →  Tackling sustainable healthcare is crucial. Stef Nimmegeers →  Prevention is underfunded and underprioritized. Charlotte Pierron-Perlès → Connected health solves clinical trial inefficiencies. Sophie Burton-Jones →  Incentivization is way harder than it seems. Olaf Schneider → Health data drives next-gen diagnostics accuracy. Anna Codognotto → Data protection enhances AI reliability. Clara Landahl →  Administration complexity hinders efficiency. Moz Siddiqui → Women's health needs prioritization and support. Shana Hudes →  The transition from curative to predictive care is near. Hagar Anna Spits → The UK is ready to drive healthcare innovation. Kim Leung → Underfunded women's health needs change. Adam Brown, MD MBA →  The risks of healthtech divide are real. Samuel Scheer → Healthcare innovations will drive cardiometabolic care. P.S. Which one’s your favorite? Share in the comments and let’s kick off the discussion! HLTH Europe #hlth #hltheurope #hlth2024 #healthtech #innovation

36

7 Comments

One key reason why healthcare technologies like AI, telemedicine, and software haven't structurally lowered costs or broadened access for the entire system after decades of experimentation (excluding pure financial engineering of payer contracts) is the limited reimagining of care delivery itself, with asynchronous telemedicine as a notable exception. In the same way that digitizing physical paper failed to exploit the capabilities of a new computing medium for word processing, hosting physician-patient conversations over video does not transform the actual delivery of care. In telemedicine, many of the inputs (e.g., clinician time) and therefore cost structures of in-person care persist, which is why we often see fee-for-service telemedicine companies struggle to achieve economies of scale (i.e., growing revenues non-linearly while growing costs linearly). Until we deconstruct the components of a medical appointment – from intake and triage, to chart review, history gathering, physical examination, diagnosis, treatment, and follow-up – we will continue to be underwhelmed by the disconnect between the transformative potential of these technologies and real-world patient/system impact. I’m optimistic that the emergence of reliable and performant medical AI agents will urge us to perform this deconstruction and redesign the fundamental building blocks of a “medical appointment.”

215

39 Comments

Harnessing AI in Healthcare: EliteCoders and Dr. Larry Goldstein Revolutionizing Patient Care: https://lttr.ai/AUbUL #ImprovedPatientOutcomes #PredictiveAnalytics #ImpactfulInnovations #AI #SolveRealProblems #HitCrisisLevel #RealWorldMedicalChallenges #LevenbergMarquardtBackpropagationAlgorithm #CustomizeTreatmentPlans #BuildingStrongerBonds

7

About 75% of all deaths in the U.S. are due to chronic conditions. And, evolving RPM is proving to be an effective way of delivering care to patients with chronic conditions. But, why is it so promising? Remote monitoring reduces the provider and patient gap. If AI-enabled, it can process data more accurately. A strip on your wrist, tracking your vitals all day long. So effortless, right? Just imagine the impact on people with chronic conditions. The future of chronic care with RPM looks bright as I see it in the coming years- 👉 Highly sensitive and advanced wearable 👉 AI-empowered health predictions 👉 Enhanced patient engagement 👉 Improved reimbursement model 👉 Tech powered hospital-at-home Model 👉 Closing health access disparities gaps 👉 Tailored care plans Agree? I'm looking forward to your thoughts and visions. Tara English Michael Giles Kelley Bowker, MBA Joy Bhosai, MD MPH Emilie Ansel, MBA #RemotePatientMonitoring #ChronicCareManagement #RPMforChronicConditions #TelehealthMonitoring #RemoteHealthManagement

11

10 Comments

Excited to share my recent article on Patient Safety and Quality Healthcare! Discover how innovative digital solutions are reshaping patient care and safety. From predictive analytics to real-time monitoring, explore the future of healthcare and how we can achieve excellence in patient outcomes. Let’s innovate together! #DigitalTransformation #HealthTech #FutureOfHealth

32

🔷 The Language of Medicine Is Changing 🔷 Optimizing clinical dialogues with LLMs: enhancing care, not complexity. 🔵 LLMs have the potential to transform health care, but effective prompting is key to unlocking their utility. 🔵 Context-rich inputs and iterative refinement enhance the accuracy and relevance of LLM responses. 🔵 Seamless integration of LLMs into clinical workflows is crucial to avoid overburdening clinicians. https://lnkd.in/eDE-nZpQ #AI #LLMs #medicine #digitalhealth

10

The debate about who controls one's own healthcare data is fundamental to how we evolve our health systems. - It enables us to manage our #wellness on an ongoing basis rather than any illness that may occur - It enables us to contribute our unique #realworlddata as we see relevant for #clinicaltrials or #populationhealth studies - It enables us to optimise #healthsystems to increase efficiency and improve #valuebasedoutcomes - It enables us to address #diversity and #bias through synthetic data generation where gaps exist - It gives us the potential of #portability of service for our healthcare needs The article from Artur Olesch shows how badly broken current systems are around the world. Currently Governments; Health Care Providers; Insurers; Tech Companies and Health Care Professionals all manage our healthcare data. Innovation in this space to enable citizens and patients to manage their own data assets and benefit from #genai tools to analyse will provide us with the next great leap forward in terms of health outcomes and system efficiencies.

11

2 Comments

Manually entering patient data into your EHR? Duncan McGillivary explores how AI is revolutionizing healthcare, offering solutions that break free from clunky EHRs and deliver more efficient workflows. Check out our latest #AnsaOpenSpaces. #healthcare #AI #EHR #clinicianempowerment

8

1 Comment

Will Generative AI Transform Healthcare? Uncover the Opportunities and Challenges. Generative AI is poised to transform healthcare, offering unprecedented capabilities in data management and patient interaction. As investments from both tech giants and innovative startups flood this emerging sector, the potential for groundbreaking developments is significant. However, this enthusiasm is tempered by considerable concerns. The readiness of AI systems, coupled with issues of bias and privacy, has prompted calls for a cautious approach. Healthcare professionals and industry advocates stress the importance of establishing stringent regulatory frameworks and conducting thorough testing to ensure that AI technologies are both safe and effective. For investors, the integration of AI into healthcare presents a unique dual opportunity: to lead the charge in technological innovation while addressing the ethical and practical challenges it brings. This careful balance of advancement and accountability is crucial in navigating the future landscape of healthcare technology. https://hubs.la/Q02t4NwV0 What are your views on AI’s role in modern healthcare? Join the conversation below and share your insights.

19

2 Comments

Reboot Communications Ltd. We invite you to participate in a groundbreaking event at the forefront of healthcare innovation. The 24th Annual Healthcare Summit: Medicine 3.0 will explore the intersection of Longevity, Artificial Intelligence, and Genomics. Esteemed healthcare leaders from across Canada and other parts of the world will convene to foster connections, impart knowledge, and explore pioneering strategies poised to revolutionize healthcare delivery. Be part of the conversation shaping the future of transformative healthcare implementation. The agenda will feature over 50 subject matter experts in personalized medicine, longevity, artificial intelligence and genomics. These thought leaders will share case studies, cutting-edge insights, and the latest advancements in health policy, programs, research and technologies. The agenda will include investors and companies presenting their business case to promote the value and opportunity that longevity and healthy living will offer to the future of healthcare delivery. Molecular You Rob Fraser Jen Mielguj Payal Sondhi John Jacobson Dianne Balon, CEC, ACC Shaun McCumber Diane T. Finegood Pieter Cullis Jim Slater Alan Low Howard Waldner Mary Ackenhusen Aaron Wudrick Barry Burk Stephen Samis Catalina Lopez-Correa Tim Murphy Barry Rivelis, ICD.D Chen Wan Wendy Hurlburt Malcolm Crow Rachael Ritchie Sue Paish Soyean Kim Robin Gould-Soil Larry Kozak Aleina Spigelman, MSc Laura Fitzgerald Skylar Newton Kathryn Seeley Steven Tam Veronica Mustoe #hcs24 #hcs #healthcare #healthcareinnovation #healthcaretransformation #healthcaretrends #pointofcare #homecare #digital #digitalhealth #wholepersonhealth #elderlycare #disruption #genetics #longevity #healthyliving #healthdata #hcs #hcs24#POG #BreasCancer #BreastCancerResearch #PrecisionOncology #Oncology #Genomics #CancerJourney  #Equity #Access #WholeGenome #WGS #NGS #EquitableAccess #NeverGiveUp #GenomicsForAll #biohacking #ao

18

2 Comments

🏥 "Agent Hospital" simulates hospital operations with autonomous agents representing patients, nurses, and doctors. Powered by LLMs, they manage the full treatment process from illness onset to follow-up. 🧠 Doctor agents self-evolve without manually labeled data. They improve through simulated patient interactions and reading medical documents, leading to continuous performance enhancement. 🔬 Experiments show evolved doctor agents excel in diagnosing and treating respiratory diseases. Interactions with generated patients demonstrate improved performance over time. 👩‍⚕️ The simulation includes medical professional and resident agents. They interact with generated patient profiles and evolving clinical situations for dynamic, personalized treatment sequences. 📚 Doctor agents learn from medical records, news, and textbooks. They generate and solve multiple-choice questions to enhance their knowledge, improving diagnosis and treatment recommendations. #AI #Healthcare #MedicalSimulation #MachineLearning #Innovation #Tech #LLM

1

AI #HealthTech Aims to Fix #Health Problems You Didn't Know You Had #fintech #healthcare #AI #ArtificialIntelligence #MachineLearning #GenerativeAI @jessica_rendall @CNET

6