CT-Insight VLM: Multimodel Vision-Language Framework for Accurate, Explainable, and Label-Efficient Lung CT Analysis

The rise in chronic respiratory diseases has created a growing need for AI-driven tools that can analyze multimodal lung CT data for early and accurate diagnosis. Recent multimodal large language models (LLMs) show promise for automated lung CT diagnosis, but they often rely on large-scale annotated data and struggle to generate structured reports with quantitative disease metrics. To address this, we propose CT-Insight VLM, a novel multimodel vision-language framework for accurate, explainable, and “label-efficient” lung CT analysis. Our method integrates a three-stage pipeline: normality assessment, lesion analysis, and knowledge-augmented retrieval, to extract key visual features and generate structured prompts. These prompts guide a lightweight LLM to produce expert-level diagnostic reports that include both textual explanations and quantitative lesion measurements, without requiring extensive multimodal annotations. CT-Insight VLM achieves 98.54% accuracy in normality classification and a 67.06% DICE score in lesion segmentation across five public datasets. Expert evaluation yields a mean radiological assessment score of 3.48/5 (approximately 70% normalized agreement), underscoring the clinical relevance and practical reliability of the diagnostic reports generated by the proposed framework. The source code and related resources for the proposed framework are available at GitHub.