ADAPTIVE GRAPH AND MULTIMODAL RETRIEVAL FOR EVIDENCE-BASED MEDICAL QUESTION ANSWERING

Abstract

Medical question-answering systems must retrieve relevant information, reconcile heterogeneous evidence and make the basis of their answers traceable. Text-centred retrieval-augmented generation (RAG) systems can overlook relationships encoded in medical knowledge graphs and information presented in guideline figures. We propose Adaptive Graph and Multimodal Retrieval (AGMR-Med), a conversational framework that integrates unstructured text, structured medical knowledge and visual evidence. The framework combines a domain-aware query encoder, an adaptive network that weights retrieval scores across modalities, and a multi-level evidence aggregator with claim-level citation verification. Evaluation using MedQA-USMLE and a curated multimodal guideline corpus yielded 91.7% answer accuracy, 88.4% citation fidelity and a mean response time of 7.6 s. These values exceeded the accuracy and citation fidelity, and reduced the latency, of the evaluated Vitruvius/MedSearch and GraphRAG baselines. Ablation experiments indicated that adaptive weighting and multimodal retrieval contributed to performance. Additional evaluations included MedMCQA, pre-operative guidance questions, demographic perturbations and a study involving 20 clinicians. In the clinician study, AGMR-Med achieved a mean answer-validity score of 2.9 out of 3 and a mean response time of 5.8 s, compared with 9.7 s for conventional web search. The results support further investigation of adaptive evidence integration for medical question answering. Broader external validation remains necessary to establish reproducibility, clinical applicability and performance across underrepresented specialties.