Evaluation of large language models for clinical decision support in penile and testicular cancer: a multidisciplinary Delphi study

In this study, three selected lightweight LLM variants showed high concordance with a multidisciplinary panel of experts when responding to guideline-based questions after being provided with guideline materials comprising the original 2025 EAU guideline text and plain-language textual descriptions of figures and tables for penile and testicular cancer. More than 90% of the responses reached consensus in agreement, with no statistically significant differences among models, indicating broadly comparable performance in standard scenarios. This finding is particularly relevant given the rarity of these malignancies and the limited representation of such tumor types in previous research on AI applications in uro-oncology. Importantly, only a small number of responses required a second Delphi round or failed to reach consensus, and these were typically the most complex or controversial scenarios. Most disagreements were concentrated in aspects related to staging and prognostic assessment. These findings suggest that, in a controlled, guideline-fed setting, selected lightweight LLM variants can consistently interpret guideline-derived information across most standard scenarios, although limitations remain in areas of greater clinical ambiguity, where decision-making requires nuanced interpretation and accumulated expertise.

LLMs are increasingly being explored as tools to support access to evidence-based medical information in medicine [5,14–16]. These advanced AI systems are trained on vast biomedical literature and clinical guidelines, enabling them to generate detailed, contextually relevant responses to complex clinical queries [17,18]. In oncology and other specialties, LLMs have the potential to bridge the gap between rapidly evolving guidelines and clinical practice, offering clinicians timely access to synthesized knowledge and supporting access to diagnostic and therapeutic information [14,18]. However, the reliability and safety of LLM-generated responses must be rigorously validated, particularly in high-stakes fields like cancer care, to ensure effective and responsible future integration into clinical workflows [15,17].

A detailed analysis of our results indicates that the degree of agreement among the panelists was similar for testicular cancer and penile cancer responses, with no statistically significant differences observed. However, it is noteworthy that only in testicular cancer did two responses result in consensus disagreement with some of the LLMs, suggesting that these models occasionally provided responses that did not align with expert panel expectations. Across both tumor types, responses related to standard management generally reached high agreement. By comparison, responses related to disease staging and prognosis, such as the application of staging systems and the interpretation of prognostic factors, were more likely to generate disagreement or require a second Delphi round. Common panelist criticisms in these domains included omission or misinterpretation of key staging criteria and a lack of nuance in the assessment of prognostic factors. These findings highlight that, while LLMs can consistently interpret guideline-derived information in most routine scenarios, their performance appears less robust when faced with the complexity of cancer staging and prognosis, reinforcing the need for expert oversight in these critical aspects of patient management. Moreover, the apparent simplicity of such tools carries the inherent risk that even general practitioners or non-subspecialized specialists may underestimate the importance of involving dedicated experts when interpreting clinical information or considering therapeutic decisions. The two responses reaching consensus in disagreement are particularly relevant from a safety perspective. In both cases, the models generated plausible responses despite clinically relevant errors or omissions. Such inaccuracies may be difficult for non-specialist users to identify, reinforcing the need for specialist validation of LLM-generated responses.

Despite these limitations, overall acceptance of the LLM-generated responses was remarkably high, with median scores of approximately 8 out of 9 and 94–98% of LLM responses achieving a median score ≥7 across the three models. These high panel scores support the consistency of the LLM-generated responses with the guidelines, although validation remains necessary before clinical implementation.

Several recent studies provide important context for interpreting our findings, both in general oncology and specifically within urological oncology. In the field of medical oncology, Longwell et al. [18] conducted a cross-sectional evaluation of LLMs using standardized examination-style questions from the American Society of Clinical Oncology (ASCO) and the European Society for Medical Oncology (ESMO). They found that the best-performing proprietary LLM, which was not named in the publication due to legal restrictions, correctly answered 85% of questions, with most explanations containing no or only minor errors. However, the study also highlighted that a significant proportion of incorrect answers could potentially lead to moderate or severe clinical harm if acted upon, emphasizing the need for careful oversight even when overall accuracy is high.

Focusing on urological oncology, Eckrich et al. [16] compared the performance of several LLMs with that of urology consultants using case-based clinical questions. Although LLMs consistently generated coherent and comprehensible answers, their medical adequacy was rated lower than that of consultants. Of note, the study identified potential misinformation hazards in 2.8–18.9% of LLM-generated answers, compared with less than 1% for consultant responses. The authors concluded that, while LLMs can produce coherent medical advice in urology, their adequacy remains inferior, and their answers may contain potentially hazardous misinformation. Therefore, they should not be used in clinical practice without expert supervision. In our study, consensus in disagreement occurred in 2 of 150 model responses (1.3%); however, this outcome is not directly comparable to the potential misinformation hazard assessed by Eckrich et al. due to differences in the evaluation frameworks. A similar need for expert oversight was observed in pediatric urology. Hoen et al. assessed LLMs’ ability to update guideline content and found that, despite producing scientifically accurate responses in many cases, the models demonstrated insufficient consistency and reliability to replace expert panel review, especially when integrating new evidence [19].

A significant challenge in the evaluation of LLMs in oncology is the absence of a standardized methodology. As highlighted by Carl et al. [17], studies use diverse approaches, such as multiple-choice questions, open-ended scenarios or comparisons with expert decisions, making direct comparison difficult. Our use of a Delphi process with a multidisciplinary panel offers advantages, like structured consensus and detailed feedback, but alternative methods also provide valuable perspectives. This diversity underscores the need for greater methodological harmonization in future research [17].

Our results, along with recent studies, suggest that LLMs may support access to guideline-based information, especially in standard scenarios or resource-limited settings. However, our findings should be interpreted as evidence of performance in a controlled guideline-fed task, rather than as evidence of clinical decision-making capability [17,18]. Therefore, their integration into clinical practice should be gradual and always under expert supervision, beginning with low-risk tasks or educational support and expanding as more evidence emerges regarding their safety and effectiveness [14,18]. Transparent reporting of model version, training data and update criteria, as well as independent evaluation of biases and limitations, is essential [14,17]. Multidisciplinary collaboration among clinicians, AI experts, and patients is also crucial to ensure these tools address real clinical needs.

Regarding practical implementation, the planned urooncoguias.aeu.es chatbot is intended exclusively for healthcare professionals and will require professional authentication. It is designed to support access to and interpretation of guideline-based information rather than replace specialist judgment. The errors identified in the present study, particularly in staging and prognosis, reinforce the need for expert oversight when interpreting chatbot-generated responses.

Finally, in our view, to maximize benefit and minimize risk, clinical guidelines – particularly those related to complex areas such as cancer staging – should increasingly adopt structured formats, with clear decision algorithms, explicit definitions and standardized tables, ensuring interpretability by both clinicians and AI systems.

This study has several limitations. First, it focused on a limited number of clinical settings and tumor types, which may affect the generalizability of the findings. In addition, the relatively small number of questions evaluated per cancer type may have limited the ability to detect differences between models; therefore, the absence of statistically significant differences should not be interpreted as evidence of equivalence. Second, the evaluation relied on expert panel consensus, which, while robust, may introduce subjective bias. In some questions, especially those with greater specialty specificity, only a subset of the panel voted, and consensus was calculated exclusively among those who responded. While this increases content expertise for each item, it may introduce potential bias due to variable sample sizes across questions, which should be considered when interpreting the results. The study did not evaluate open-ended, real-world clinical use of LLMs. Instead, the models were provided with guideline materials comprising the original text and plain-language textual descriptions of figures and tables, so the study primarily assessed their ability to interpret and reproduce the supplied guideline-derived information. Moreover, the rapidly evolving nature of LLM technology means that model performance may differ in future versions. Furthermore, only a single response was generated per question and model, and repeated runs were not performed; therefore, output stability and reproducibility across runs could not be assessed. Nevertheless, this work has important strengths. We converted EAU guideline figures and tables into clear, structured plain-language textual descriptions, ensuring that both experts and AI models had access to the same information and reducing ambiguity in interpretation. The use of a multidisciplinary Delphi process further enhanced the rigor and relevance of the evaluation.

Overall, our results indicate that selected lightweight variants from widely used LLM families, when provided with clinical guideline content, can generate responses that align with expert consensus in most standard guideline-based scenarios involving penile and testicular cancer. These findings should be interpreted within the context of a controlled, guideline-fed evaluation and require further validation in more realistic clinical settings before any integration into oncology practice.

A Borque-Fernando led the project and contributed to the study’s conceptualization, methodology, project administration and supervision. All authors contributed jointly to the investigation, formal analysis, data curation, validation, resources, visualization, writing of the original draft and review and editing of the manuscript, and approved the final version for publication.

The authors thank Pablo Rivas for his assistance with medical writing on behalf of Content Ed Net.

This project was funded by the Spanish Association of Urology. The authors received support from Instituto de Investigación Sanitaria Aragón (Spain) for the project “Uro-Onco GuÍAs: Desarrollo de un Chatbot basado en IA Generativa para la Interpretación de Guías Clínicas en Uro-Oncología”.

The authors have no competing interests or relevant affiliations with any organization or entity with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.

Medical writing of the manuscript through Content Ed Net was funded by the Spanish Association of Urology.

This study involved only expert panelists who participated voluntarily in the Delphi process. No patient data were used. Participation implied informed consent, and formal ethical approval was not required.

Participation by expert panelists was voluntary and implied informed consent.

The results of the Delphi process are presented within the manuscript. The complete set of responses generated by each large language model to all 25 questions are available for consultation at https://www.contentednet.com/proyecto-urooncoguias-aeu/

This work is licensed under the Attribution-NonCommercial-NoDerivatives 4.0 Unported License. To view a copy of this license, visit https://creativecommons.org/licenses/by-nc-nd/4.0/