Generative Pretrained Transformers for Medical Communication Training and Automated Performance Assessment

Effective clinician-patient communication is a cornerstone of safe, high-quality healthcare. The General Medical Council (GMC) 2018 curriculum update reinforced the need for structured, high-quality communication training in medical education. Traditional methods—relying on human actors, manual evaluation, and subjective feedback—remain resource-intensive and difficult to standardise at scale. To address this, we developed an AI-powered Communication training system that leverages fine-tuned Generative Pretrained Transformer (GPT) models for both patient simulation and automated assessment.
Using the MedDialog-EN dataset—comprising over 250,000 clinician-patient dialogues—GPT-3.5 Turbo was fine-tuned to generate realistic, contextually appropriate, and empathetic patient interactions. This model was integrated into a web-based platform where medical students could engage in structured, case-based scenarios featuring AI-driven patient personas. To ensure objective and consistent evaluation, GPT-4o-mini was employed via the Assistant Application Programming Interface (API) to analyse student responses against predefined assessment rubrics. These rubrics measured key communication competencies, including eliciting medical history, addressing lifestyle factors, and demonstrating empathy. Fine-tuning and prompt optimisation enhanced rubric adherence, achieving approximately 90% accuracy, as validated by medical expert review.
Our preliminary findings suggest that AI-driven patient-clinician interactions provide naturalistic and engaging training experiences. At the same time, our GPT-based assessment tool deliver structured, unbiased and accurate feedback on history-taking dialogs. By augmenting or even replacing resource-intensive human simulations with scalable, data-driven AI tools, this approach enhances accessibility and standardisation in communication skills training. Future developments will focus on integrating voice-based AI interactions, refining model personalisation, and expanding assessment capabilities to further improve medical education. This research highlights the transformative potential of AI in bridging gaps in communication training, fostering a more efficient, equitable, and scalable learning environment for future healthcare professionals.