Hybrid programmatic-AI workflows: Combining automation and human judgment in HTA-grade time-to-event reporting

As health technology assessment (HTA) bodies face growing demand for timely and transparent evidence, interest is increasing in tools that can improve the efficiency and consistency of analytical reporting without compromising scientific rigor.
Generative AI (GenAI) is being widely explored across HTA, with potential applications spanning evidence identification and synthesis, data analysis, interpretation and reporting. However, questions remain about where these tools can add most value and how their use should be balanced with appropriate scientific oversight.
At ISPOR Europe 2026, Simone Rivolo, George Bungey, Jack Ishak and colleagues from the PPD™ Evidera™ Health Economics and Market Access team at Thermo Fisher Scientific presented a case study evaluating a hybrid workflow that combines programmatic automation with GenAI to support the interpretation and reporting of time-to-event analyses. In this interview, they discuss how the workflow was developed and evaluated, its potential efficiencies and the continuing role of human judgment in HTA-grade reporting.
Thanks for speaking with us today about this research. What prompted you and your colleagues to explore a hybrid programmatic-AI workflow for time-to-event analysis interpretation and reporting? Which parts of the conventional process were you particularly looking to improve?
Reporting and interpretation of time-to-event (TTE) analysis needs to follow best practices and published HTA guidance, such as NICE Technical Support Documents 14 and 21. This means that it is a process with well-defined steps, including evaluation of the shape of the hazard function, proportionality hazard assessment and fit statistics, which is therefore amenable to automation.
Compared with the conventional process involving full human drafting, we were particularly looking to:
- Improve the consistency and comprehensiveness of what is reported for TTE analyses, aligned with HTA guidance.
- Allow subject matter experts (SMEs) to focus their time on results interpretation and implications of findings, rather than full drafting and copy-pasting tables and figures manually.
- Through AI drafting, reduce SME effort in the interpretation of each TTE analysis review step with write up including clear technical language.
- Reduce the risk of errors arising from manual copy-paste errors.
You describe the objective as producing “HTA-grade” interpretation and reporting. What does HTA-grade mean in this context, and which requirements did the workflow need to meet?
For us, the workflow needed to include, for each TTE outcome, all the analytical components that HTA agencies require to report, including those outlined in the previously mentioned NICE DSU TSD14 guidance. Furthermore, given the ongoing recent best practice developments on AI use in HEOR (e.g., ISPOR Working Group Report and NICE Position Statement), the workflow needed to meet the following requirements:
- Augment rather than replace human involvement, with well-defined stages in the workflow for human review.
- Provide transparency and traceability around where AI is used, incorporating the inputs provided to AI, the outputs produced by AI and the prompts on which those outputs are based.
Can you take us through the TTERep-AI workflow and explain how responsibilities are divided between the programmatic layer, the generative AI layer and the SME? Why was this hybrid structure important?
The hybrid structure was important mostly because we wanted to reduce the risk of the AI layer introducing errors or hallucinations when transferring the TTE statistical analysis results, such as figures, tables, AIC/BIC values, predicted milestones, into the templated report. Therefore, anything that is essentially done the same way every time, including the structure of the report, standard text with quoted results and placement of output in the appropriate section, was therefore implemented programmatically.
This approach is less suitable for tasks such as description and interpretation of results as patterns may differ between analyses. We believed this was where AI would add most value and provide the user with a first draft to revise and refine.
Briefly, the programmatic layer is responsible for copying all the TTE analysis results needed into the templated report, filling in some specifics of the analyses (e.g., trial name, treatment names and outcome names) and providing description and interpretation of the fit statistics section. The AI layer then works through each section requiring description and interpretation, and drafts the required paragraph solely based on the specific statistical analysis results provided, following the section-specific templated prompt.
The SME receives the full draft, together with the TTE results, and finalizes the report by ensuring that the descriptions and interpretations are correct and integrating additional context not yet included in the TTERep-AI workflow, such as clinical expert feedback on the long-term survival expectations). SME review is facilitated by clearly distinguishing between templated text, content produced by the programmatic layer and AI-generated draft, alongside warnings on what to watch for by section and analysis type.
The section-specific prompts were developed using two datasets before the workflow was evaluated across a further six. How did you design and refine these prompts, and how did you assess whether the resulting reports were accurate, consistent and applicable across different datasets?
The AI persona and the section-specific prompt were designed by two SMEs with extensive experience in TTE analysis interpretation. For each prompt, the SMEs defined the following:
- Task objective
- Evidence provided to the AI (such as a Kaplan–Meier [KM] figure)
- Instructions on how to perform the task
- Content to cover
- Expected output
These prompts were iteratively refined until the SMEs were satisfied with the performance achieved by the AI layer on the two test datasets. Since the AI-layer produces multiple paragraphs for each section, the key focus of the prompt development was on correctness of the description, interpretation provided and general appropriateness of the technical language used for describing survival analysis, rather than minor language edits needed for finalization.
We then evaluated the TTERep-AI workflow on the six TTE datasets not used during development. These were selected to represent a range of scenarios the AI could encounter including different outcomes, such as progression-free survival, overall survival and time to treatment discontinuation; single-arm and two-arm KM curves; mature and immature data; immediate and delayed separation of the curves; and variation in the plausibility of the proportional hazards assumption.
The draft reports created across these datasets were directly shared with SMEs who were not involved in developing the workflow. They rated the draft reports in terms of quality and time savings needed for finalization of the AI written sections (as compared with full human de-novo drafting).
Performance varied across the different reporting tasks. What made this aspect more challenging for the AI, and what does this tell us about the limits of current models?
The variability of performance across the different reporting tasks was expected. This is mostly due to the stochastic nature of LLMs, the fact that often in TTE analyses there are interpretation nuances which are likely not fully captured in the current prompt, such as whether the delayed separation in the KM curves is noteworthy, and our decision to keep the workflow relatively simple. For example, we did not use a different LLM to review and improve the AI-drafted paragraph.
“What did surprise us was the limited performance achieved in the AI-drafting of the visual-fit assessment, which is a relatively straightforward task for SMEs.”
With further experimentation conducted after this study, we were able to improve the AI drafting of visual-fit assessment by simplifying the figures fed to the AI for visual assessment and asking it to tabulate scoring of each parametric distribution visual fit assessment before drafting the required paragraph. Further performance gains might also derive from the continuous improvements of foundational AI models.
Estimated time savings reached 80% for most sections but were below 30% for visual-fit interpretation, and every report still required human editing. Where do you see the most meaningful efficiency gains in practice, and how should organizations balance these against the time needed for review and quality assurance?
We designed the workflow to augment human involvement, so we do see SME editing and finalization as the key final step of the workflow. Based on accumulating data from broader use of TTERep-AI in our workflows, SMEs are reporting overall time savings of 50–60% to create the first draft and 15–20% for final SME review.
We believe the most meaningful efficiency gains come from avoiding SMEs spending time copy and pasting results from analyses into the report, reducing the effort needed by SMEs to write-up the interpretation of each TTE step, and reducing the chance of copy–paste errors or missingness of required sections that subsequently require reworking.
“The use of AI in HEOR is still in the early adoption phase. Based on our experience with TTERep-AI, we currently see meaningful efficiency gains in redesigning current workflows around AI use for desk research, content summarization and other forms of report drafting.”
We do not currently believe that the time SMEs spend on review and quality assurance is something that can be significantly reduced. Nevertheless, designing workflows to facilitate the process and using AI to assist SME review, as well as help improve overall consistency and quality assurance, can definitely be further integrated. However, this will likely result in higher research and reporting quality rather than further efficiencies.
In your opinion, what safeguards are needed to make hybrid AI workflows sufficiently transparent, reproducible and trustworthy for HTA applications?
We believe that redesigning workflows by thinking upfront on which overall tasks and sub-tasks the AI should conduct is key for reproducible HTA applications. This allows transparent reporting of the inputs provided to the AI, the outputs the AI produced and how AI-generated content was verified across workflow stages, tasks and sub-tasks. The TTERep-AI workflow presented is a relatively simple and well-defined task, thus facilitating transparent reporting of the workflow and trustworthiness, through SME review and finalization. However, the challenge with safeguarding increases significantly when designing workflows for broader HEOR tasks, requiring agentic design and multiple interactions across agents and between AI and SMEs.
Finally, looking ahead, where else in evidence generation could this approach be applied, and what further validation would be needed before routine use?
TTERep-AI is now moving to routine use, with the SME finalization step ensuring that AI-generated content is carefully reviewed and adjusted. We believe that this approach could be applied in other evidence generation tasks where the reporting flow and content can be standardized and largely templated, such as the reporting of indirect treatment comparisons and mapping/utility analyses.
More complex hybrid workflows could also be designed, depending on the specific task of interest. For TTE reporting, for instance, an AI agent can start by running the TTERep-AI workflow, then conduct research to identify available literature for evaluating long-term extrapolation, and then expand the draft report summarizing the reviewed evidence and the implications for parametric model selection.
About the interviewees
Simone Rivolo, PhD
Senior Director, PPD Evidera Health Economics & Market Access, Thermo Fisher Scientific

Dr Rivolo has extensive experience developing pharmacoeconomic models and value demonstration tools across multiple disease areas including cardiovascular, oncology, neurology, infectious disease, inflammatory bowel disease and rare diseases. His work spans multiple countries, including the US, Canada, Spain, France, England, Ireland, Scotland and Switzerland. He brings expertise in several modeling approaches including decision trees, Markov cohort models, sequencing models, microsimulations and patient-level simulations, and software using Excel, VBA and R.
George Bungey, MSc
Research Scientist, PPD Evidera Health Economics & Market Access, Thermo Fisher Scientific

Mr Bungey has extensive experience developing, adapting and reviewing cost-effectiveness, cost-consequence and budget impact analyses, as well as preparing technical reports across a range of therapeutic areas. His prior work has spanned a range of indications including oncology, COVID-19, NASH/MASH, auto-immune conditions, growth hormone deficiency and acute coronary syndrome. He has conducted parametric time-to-event analyses in oncology, Duchenne muscular dystrophy and psoriasis, and developed statistical analysis plans for mapping and utility studies. Mr Bungey has contributed to multiple submissions to HTA bodies including NICE, SMC, AWMSG and NoMA.
K. Jack Ishak (Sahakian), PhD
Vice President, Statistical Methodology & Strategy, PPD Evidera Health Economics & Market Access, Thermo Fisher Scientific

Dr Sahakian leads Methods and Innovation for the PPD Evidera Health Economics & Market Access Team and the company’s AI transformation workstream. He has more than 20 years experience advancing statistical science in HEOR and RWE, with a focus on development and application of novel statistical methods. His expertise includes advanced time-to-event modeling and projection, population-adjusted indirect treatment comparisons, modeling effects of changing treatment patterns (e.g., crossover), disease progression and forecasting, and simulation-based approaches to disease modelling and study design. His recent work also spans multilevel network meta-regression, transportability, AI and automation. He also co-developed a Bayesian framework that integrates evidence from multiple sources to quantify the benefits of new treatments for rare diseases and the certainty of those estimates.
Acknowledgments
We would like to acknowledge the other co-authors of the research presented: Tereza Lanitis, Michalis Galanakis and Ivan Houisse in the PPD Evidera Health Economics & Market Access team at Thermo Fisher Scientific.
Disclaimers
All authors are employees of PPD Evidera Health Economics & Market Access, Thermo Fisher Scientific. This poster was funded by Thermo Fisher Scientific.
The opinions expressed in this feature are those of the author and do not necessarily reflect the views of The Evidence Base® or Becaris Publishing Ltd.
Sponsorship for this Interview was provided by Thermo Fisher Scientific.
