Validating LLM-derived RWD across borders: Extending the VALID framework to UK prostate cancer data
As large language models (LLMs) are increasingly used to extract clinical information from electronic health records at scale, demonstrating the quality and fitness for purpose of the resulting real-world data (RWD) is becoming increasingly important. For researchers, HTA bodies and regulators, the challenge is not simply whether an LLM performs accurately, but whether the resulting dataset can be evaluated transparently and consistently enough to support evidence generation and decision-making.
In this interview, Arun Sujenthiran, Senior Medical Director and Clinical Lead, International at Flatiron Health, discusses new research being presented at ISPOR Europe 2026. Building on earlier work in the US, the study applies Flatiron’s VALID framework (Validation of Accuracy for LLM/Machine Learning-Extracted Information and Data) to a UK prostate cancer dataset of 13,905 patients from an NHS Trust network, benchmarking LLM-derived data against expert human abstraction and national registry data.
The interview explores:
- Why validating a single variable against human abstraction does not provide a complete picture of data quality, and how VALID combines variable-level metrics, verification checks and replication analyses
- What researchers learned when applying the framework in a different healthcare system, and which elements proved transferable across settings
- How LLM-derived survival estimates compared with human-abstracted data, including median overall survival estimates for ARPIs and taxanes
- What benchmarking against NHS England and National Prostate Cancer Audit data can add to the validation process, as well as the limitations of registry comparisons
- What this next phase of the research could mean for developing more consistent approaches to evaluating LLM-extracted RWD across countries
For teams generating, assessing or commissioning LLM-derived RWD, this is essential reading on how transparent, repeatable validation could help decision-makers understand the quality, limitations and fitness for purpose of AI-enabled real-world datasets.
Read the full interview to discover what the UK findings reveal about extending VALID beyond the US.
About the interviewee
Arun Sujenthiran
Senior Medical Director and Clinical Lead, International, Flatiron Health
Arun Sujenthiran is Senior Medical Director and Clinical Lead at Flatiron Health, overseeing teams who curate oncology data products across the UK, Germany, and Japan. He works with cross-functional partners to ensure the clinical relevance and scientific rigor of Flatiron’s real-world data offerings.
A former full-time uro-oncology surgeon and health outcomes researcher, Arun continues to practice at a London NHS hospital. He earned his medical degree with distinction from Imperial College London and completed an MD(Res) at the Royal College of Surgeons Clinical Effectiveness Unit. His research focuses on using real-world data to improve cancer care and patient outcomes.
Acknowledgments
Arun would like to acknowledge the contribution of his co-authors:
- Victor Lhoste, PhD, Associate Research Scientist, Flatiron Health
- Kathi Seidl-Rathkopf, PhD, Director, Data Science and Machine Learning, Flatiron Health
- Natalia Viani, PhD, Staff Machine Learning Engineer, Flatiron Health
- Marina Kushnir, MS, Oncology Clinical Data Specialist, Flatiron Health
- Nikola Dolezalova, PhD, Staff Data Science Manager, Flatiron Health
- Anna Schwarz, MS, Senior Machine Learning Engineer, Flatiron Health
- Amin Hashemian, MS, Machine Learning Engineer, Flatiron Health
- Conor Walsh, MS, Machine Learning Engineer, Flatiron Health
- Alex Enrique, BA, Senior Data Scientist, Flatiron Health
- Golnessa Mojtahedi, MS, Director, International Product, Flatiron Health
- Cornelius Thaiss, MD, Product Manager, Flatiron Health
- Patrycja Pluta, DVM, Associate Research Scientist, International, Flatiron Health
- Qianyi Zhang, MS, Director, Research Sciences, Flatiron Health
Disclaimers
The opinions expressed in this feature are those of the author and do not necessarily reflect the views of The Evidence Base® or Becaris Publishing Ltd.
Sponsorship for this Interview was provided by Flatiron Health.