Evaluation of large language models for clinical decision support in prostate cancer: a multidisciplinary Delphi study

A panel of 11 experts evaluated the responses from the three LLMs (Claude, Gemini, and ChatGPT) to 25 clinical questions on PC management. Since experts could abstain if they felt unqualified to assess a given item, the number of experts who rated each item ranged from 8 to 11.

Median scores assigned by panelists for each question and LLM are shown in Figure 1. The questions, details of the consensus results, number of panelists rating each item, and the summarized panelist comments are shown in Supplementary Table 1. The distribution of scores (1–3 = disagreement, 4–6 = no consensus, 7–9 = agreement) per clinical question and LLM is shown in Supplementary Figure 1.

Consensus in agreement was reached in the first round for 52% (Claude), 80% (Gemini) and 76% (ChatGPT) of the questions. Additional consensus in agreement was achieved in the second round for 4% (Claude), 12% (Gemini) and 4% (ChatGPT). Explicit disagreement was noted in three questions (12%) for Claude after the second round. No consensus was reached for 32% (Claude), 8% (Gemini) and 20% (ChatGPT) (Supplementary Table 1).

After two rounds of assessment, Gemini reached consensus in agreement on 23 of 25 responses (92%), ChatGPT in 20 of 25 responses (80%) and Claude in 14 of 25 responses (56%) (Supplementary Table 1). The analysis of consensus in agreement versus no consensus or consensus in disagreement revealed statistically significant differences among the three models (p = 0.0033). Regarding the analysis of median scores ≥7 versus <7, Gemini obtained scores ≥7 in 24 of 25 responses (96%), ChatGPT in 22 of 25 responses (88%) and Claude in 14 of 25 responses (56%) (p = 0.0002) (Figure 2).

In the pairwise comparison of models, the number of items reaching consensus in agreement was significantly higher for Gemini compared with Claude (p = 0.0117). No significant differences were observed between Claude and ChatGPT (p = 0.0703) or Gemini and ChatGPT (p = 0.2500) after Bonferroni correction (Figure 2).

When analyzing items with a median score ≥7 versus <7, both Gemini and ChatGPT outperformed Claude (p = 0.0020 and p = 0.0078, respectively), with no significant difference between Gemini and ChatGPT (p = 0.5000) after Bonferroni correction (Figure 2).

Consensus difficulties, requiring additional rounds, disagreement or no agreement, mainly involved managing complex patients, advanced imaging and genetic testing. In four questions, some panelists felt that the question lacked enough data to provide a comprehensive answer (questions 16, 17 and 18) or addressed a controversial topic with no consensus (question 13) (Supplementary Table 1).

This study evaluated, under controlled conditions, the performance of three leading LLMs (Claude, Gemini and ChatGPT) in generating responses to structured, guideline-derived clinical questions related to PC. Using a Delphi methodology, a multidisciplinary expert panel systematically assessed their agreement with each model's output. For most questions, responses achieved strong panel support, with Gemini and ChatGPT performing best numerically, followed by Claude. The differences in performance among the models were statistically significant. However, several questions, particularly those involving management of complex clinical cases, advanced imaging, genetic testing or those with insufficient clinical data, failed to achieve strong panel endorsement. In some of these cases, performance could potentially be improved by providing more comprehensive clinical information. These findings highlight that although LLMs often deliver guideline-concordant recommendations in routine PC scenarios, their outputs can be variable in more challenging clinical contexts and should be interpreted with expert oversight. Importantly, this study did not assess LLMs as autonomous clinical decision-makers in real-world clinical practice.

LLMs are increasingly being explored as decision-support tools in oncology [15,16]. These AI systems are trained on vast corpora of biomedical literature and clinical guidelines, equipping them to generate detailed and context-specific responses to challenging clinical questions [15,16]. In PC care, LLMs and similar AI-driven platforms may help synthesize complex clinical information, support treatment recommendations, and facilitate the practical application of contemporary guidelines in everyday clinical settings [17–20]. Although these technologies show great potential to improve evidence-based decision-making, their clinical accuracy and reliability remain variable, especially in complex or evolving scenarios, highlighting the need for caution before implementation in patient care [15,21].

A closer look at our findings shows variability in median panel scores across the 25 questions. Standard management items generally received high scores, while questions on complex patients, advanced imaging, genetic testing, or those lacking sufficient clinical details were more likely to score below 7 for one or more models. Claude, in particular, recorded consensus in disagreement for three items (questions 15, 23 and 24), highlighting specific scenarios where its recommendations were considered inappropriate by most panelists. Qualitative feedback noted occasional omissions, limited clinical nuance and some factual inaccuracies, including clinically relevant issues in treatment sequencing, imaging interpretation, follow-up strategies and systemic treatment selection. These results indicate that, although LLMs can support clinical reasoning in PC, their use should remain subject to expert validation, especially in areas with limited or ambiguous guidance.

Recent evidence in PC supports a cautious yet optimistic perspective on integrating LLMs into clinical and educational practice. Riaz et al. described how AI, including LLMs, is expanding its role across the PC continuum, from early detection and risk stratification to individualized treatment selection and evidence synthesis for clinical guidelines [17]. They noted that these tools can integrate multimodal data and automate complex tasks, but highlighted challenges related to data quality, model interpretability, and the necessity of human oversight [17]. In a comparative analysis, Geantă et al. evaluated three LLMs (ChatGPT 3.5, CoPilot and Gemini) against the official Patient’s Guide using 25 expert-validated questions, with performance assessed for accuracy, timeliness, completeness and understandability through a Likert scale [18]. ChatGPT 3.5 achieved the highest overall ratings across all criteria, followed by CoPilot, while Gemini showed less consistent results. The authors observed that model performance consistently varied according to the type and complexity of the clinical context, indicating strengths in certain question types and weaker performance in others. They stressed that the safe implementation of these technologies in PC care will require robust ethical and regulatory frameworks, ongoing refinement and expert oversight. Importantly, the authors emphasized that future research should examine potential biases in AI-generated responses and their impact on patient outcomes, a recommendation with which we fully agree to ensure the equitable, safe and effective use of these technologies in PC care [18]. Extending this evidence, Trapp et al. assessed the responses of ChatGPT-4, Gemini, Copilot and Claude to common questions about radiotherapy for localized PC, incorporating evaluations from both clinicians and patients. While clinicians generally rated the responses as relevant and accurate, noting differences in completeness, clinical nuance and accuracy, patients found ChatGPT-4’s answers clear, relevant and trustworthy, reporting increased confidence and willingness to use such tools for future medical queries [19]. Collectively, these studies indicate that although LLMs hold promise for enhancing decision-making and improving health literacy in PC, their integration into practice should proceed with continuous model refinement, external validation and expert supervision, particularly in complex or guideline-limited scenarios.

A notable challenge in evaluating LLMs in PC is the absence of a standardized methodology across studies. As noted by Carl et al., current research applies a wide variety of evaluation strategies, including multiple-choice assessments, open-ended clinical prompts and comparisons with expert recommendations, making direct comparison across studies difficult [15]. In our study, we employed a Delphi process with a multidisciplinary expert panel from urology, medical oncology, radiation oncology, nuclear medicine, pathology, radiology and hospital pharmacy, providing the advantages of structured consensus-building, iterative review and rich qualitative feedback. Nevertheless, alternative evaluation frameworks may also yield valuable insights, and the existing heterogeneity in study designs highlights the importance of moving toward greater methodological harmonization in future research on the medical applications of LLMs in oncology.

This issue is particularly relevant when interpreting the differences observed between Claude, Gemini and ChatGPT in our current study. In previous research across major urological cancers, we used a direct expert rating approach with nine to eleven multidisciplinary uro-oncology specialists independently scoring each response on a standardized 5-point Likert scale, which suggested Claude’s superior performance [8]. However, the present study focused exclusively on PC and applied a Delphi methodology involving iterative rounds of expert feedback and consensus building. Under this design, Gemini and ChatGPT, both based on versions launched in 2025, demonstrated higher levels of agreement among panelists, whereas Claude’s performance, from a version launched in late 2024, was notably lower, with several items showing disagreement. Although methodological differences between studies contribute to these variances, a more likely cause is the rapid evolution and frequent updating of LLM architectures and training datasets, resulting in changes to model behavior over relatively short time frames. These observations highlight the necessity of ongoing validation and cautious interpretation of LLM performance, as results may not remain stable or comparable across different versions or models, even within the same model family.

These findings collectively emphasize the potential of LLMs as supportive tools in PC, especially in standard or resource-limited settings, where they can enable quick access to guideline-based recommendations and help reduce care disparities [15,22]. However, their adoption into practice should occur gradually, starting with low-risk tasks such as education, documentation or triage, and expanding only when strong evidence supports their safety and clinical usefulness [22,23]. Transparent reporting of model characteristics and independent validation remains crucial, along with close cooperation among medical experts, AI developers, and patients to ensure the model meets real-world needs [15,23]. To promote safe implementation, clinical guidelines, especially in complex fields, should use clear and structured language that can be understood by both humans and AI systems.

This study presents several limitations that should be considered when interpreting its findings. It focused exclusively on PC and included a limited set of 25 predefined clinical scenarios, which may not capture the full complexity and variability of real-world PC management and could limit the generalizability of the results to other tumor types or more diverse clinical settings. Additionally, the evaluation relied on a Delphi process involving a multidisciplinary panel of experts. While this method provides a structured approach for consensus building and the collection of qualitative insights, it remains inherently subjective and may be influenced by the panel composition and individual interpretations. In some questions, especially those with greater specialty specificity, only a subset of the panel voted, and consensus was calculated exclusively among those who responded. While this increases content expertise for each item, it may introduce potential bias due to variable sample sizes across questions, which should be taken into account when interpreting the results. In addition, some lower-scoring items involved incomplete clinical scenarios or topics with limited or controversial guideline support, which may have made direct comparisons between models more difficult. Moreover, the rapid evolution and frequent updates of LLMs mean that their performance is dynamic and may differ significantly after the evaluation period. Despite these limitations, the study has important strengths. Specifically, guideline figures and tables were carefully reformulated into clear, structured language to reduce ambiguity for the AI models, improving the comparability of responses. Furthermore, employing a multidisciplinary Delphi methodology enriched the clinical relevance and validity of the evaluation by combining quantitative consensus with qualitative analysis of model outputs.

In conclusion, our results indicate that advanced LLMs, when provided with PC guidelines with plain-language adaptations of figures and tables, can generate responses that frequently align with expert consensus in routine clinical scenarios. Nonetheless, notable limitations persist in more complex or nuanced situations, where clinical expertise and judgment remain indispensable. While LLMs hold promise as decision-support tools in PC care, their outputs should be interpreted with caution and always under expert supervision. To facilitate their safe and effective integration into clinical workflows, we recommend that urological oncology guidelines adopt clear, standardized and structured formats that are readily interpretable by clinicians and AI systems, and that future research continues to refine evaluation methodologies, monitor for potential bias and ensure consistent performance as LLMs evolve.

A Borque-Fernando led the project and contributed to the study's conceptualization, methodology, project administration and supervision. All authors contributed jointly to the investigation, formal analysis, data curation, validation, resources, visualization, writing of the original draft and review and editing of the manuscript, and approved the final version for publication.

The authors thank Pablo Rivas for his assistance with medical writing on behalf of Content Ed Net.

This project was funded by the Spanish Association of Urology. The authors received support from Instituto de Investigación Sanitaria Aragón (Spain) for the project “Uro-Onco GuÍAs: Desarrollo de un Chatbot basado en IA Generativa para la Interpretación de Guías Clínicas en Uro-Oncología”.

The authors have no financial and/or nonfinancial competing interests or relevant affiliations with any organization/entity to declare that are relevant to the subject matter or materials discussed in this manuscript. This includes employment, grants or research funding, consultancies, membership on scientific or other advisory boards, honoraria, stock ownership or options, paid expert testimony, patents received or pending or royalties.

Medical writing of the manuscript through Content Ed Net was funded by the Spanish Association of Urology.

This study involved only expert panelists who participated voluntarily in the Delphi process. No patient data were used. Participation implied informed consent, and formal ethical approval was not required.

Participation by expert panelists was voluntary and implied informed consent.

The results of the Delphi process are presented within the manuscript. The complete set of responses generated by each large language model to all 25 questions are available for consultation at https://www.contentednet.com/proyecto-urooncoguias-aeu/

This work is licensed under the Attribution-NonCommercial-NoDerivatives 4.0 Unported License. To view a copy of this license, visit https://creativecommons.org/licenses/by-nc-nd/4.0/