ClinicalTrACE: a self-correcting agent with interpretable uncertainty for clinical question answering

Abstract

Answering precise questions about patient records requires retrieving events that satisfy type, temporal, and content constraints simultaneously, a multi-constraint satisfaction problem that embedding-based systems cannot solve. We introduce ClinicalTrACE, a self-correcting agent that retrieves through explicit structured queries with no fine-tuning and no task-specific training data, achieving 94.1% accuracy versus 76.5% for RAG and 55.3% for a fine-tuned baseline trained on 400K examples. Ablation shows that retrieval design drives this gain: categorical constraints alone contribute +12.6%, while scaling from 3B to 14B parameters adds only +5.1%. We also develop TrACE+, an uncertainty framework that predicts errors from ClinicalTrACE’s observable execution trace. A domain-agnostic variant achieves 0.83 AUROC with stable calibration (ECE = 0.060) and transfers across hospital systems; a domain-aware variant reaches 0.86 AUROC and 97.9% accuracy at 75% coverage at the cost of portability, revealing a clear generalization-discrimination tradeoff.