In 2021, Andrew Wong and colleagues published an external validation of the Epic Sepsis Model in JAMA Internal Medicine. Across 38,455 hospitalizations, the model's area under the curve (AUC) was 0.63, against the 0.76 to 0.83 its developers had reported. It generated alerts on 18 percent of all hospitalized patients and still missed 67 percent of the patients who had sepsis.
The model had a published performance figure. Few buyers had checked that figure on their own patients.
Most health system evaluations of AI vendors weight model performance metrics heavily: accuracy, sensitivity, specificity, AUC. They give less time to the factors that decide whether an implementation delivers the value in its business case: local performance, integration, workflow, adoption, and monitoring. The sections below take those factors one at a time and end with the questions to put to every vendor.
The Model Accuracy Trap
The most common evaluation error is ranking model accuracy above everything else. Selection committees spend most of their time on sensitivity and specificity tables, head-to-head benchmark comparisons, and peer-reviewed validation studies. They spend little on the factors that decide whether the system improves outcomes in their own environment.
Model accuracy matters. A clinical AI system with poor discriminative performance will not deliver value however well it is implemented. But accuracy is a necessary condition, not a sufficient one. It is also the factor vendors can most easily tune for a controlled evaluation, which makes a reported figure a weak predictor of performance on your wards.
The specific accuracy-related evaluation errors:
Evaluating on published benchmarks rather than local data. An illustrative example, with made-up numbers: a model that reaches an AUC of 0.95 on the dataset in its validation study could reach 0.80 on your patient population. Your population may have a different demographic profile, a different disease burden, a different documentation culture in the EHR, or a different clinical protocol that changes the input data the model receives. The Epic Sepsis Model study above shows a real gap of that kind. Require vendors to validate model performance on retrospective data from your own EHR before deployment, and treat that step as non-negotiable.
Ignoring alert burden. A model with high sensitivity but mediocre specificity generates large numbers of false positive alerts. In clinical environments, high alert burden produces alert fatigue: clinicians begin dismissing alerts without reviewing them, which removes the clinical benefit of the system. Alert burden does not show up in a headline accuracy metric. In the Wong study, alerts on 18 percent of patients came with a positive predictive value of 12 percent, so most alerts did not point to a patient with sepsis. Ask every vendor for the expected alert rate per 100 admissions and the positive predictive value at the threshold you would use.
Conflating research performance with deployment performance. AI systems can perform better in research settings, where data is clean, cases are selected, and context is controlled, than in production, where data quality varies, cases are unselected, and the system runs alongside dozens of other alerts and clinical demands. Ask vendors for performance data from production deployments at comparable health systems, not only from validation studies.
The Integration Underestimation Problem
The second evaluation error is underestimating integration complexity and cost. Most AI vendors say their products integrate "with your existing EHR." That is technically accurate and operationally incomplete.
EHR integration for clinical AI is not a simple API connection. It involves:
Data pipeline engineering. AI systems require specific data inputs (specific fields, in specific formats, from specific clinical events) extracted from the EHR in real time or near real time. Building a pipeline that delivers these inputs reliably takes clinical informatics work on the health system side, not only vendor integration work.
EHR workflow integration. A sepsis alert that fires in a separate dashboard clinicians must remember to check is a different product from one that appears in the EHR workflow where the clinician is already working. Alert delivery inside Epic or Oracle Health workflows, AI documentation that lands in the note template, and recommendations surfaced in the ordering workflow all need configuration work inside the EHR that is separate from the API integration.
Build and certification requirements. Epic and Oracle Health each have their own requirements for third-party applications. The time from contract signature to production go-live includes app review, EHR build, and integration testing. Ask each vendor for the actual go-live dates at its three most recent sites on your EHR, and plan against those dates rather than the demo estimate.
Ongoing maintenance. EHR systems are updated frequently, and updates can break AI integrations. Monitoring, break-fix, and upgrade compatibility testing are recurring costs. Include them in the total cost of ownership.
The practical step: before selecting a vendor, require a written technical integration assessment that states what the integration requires, who is responsible for each component, and the realistic timeline to go-live in your EHR environment.
The Workflow Redesign Gap
The third evaluation error is treating AI implementation as a technology deployment rather than a workflow redesign project.
AI systems change how clinical work is done. A sepsis prediction system changes the workflow for responding to deteriorating patients. An ambient documentation system changes how physicians structure an encounter. An AI coding assistant changes how coders review and finalize documentation. These changes need design, testing, training, and change management. In a typical AI contract, none of that is the vendor's responsibility.
If the workflow has not been redesigned so that using the AI system is the path of least resistance, clinicians will default to the workflows they already know, even when the technology is installed and accessible. Budget for the redesign before go-live, and ask vendors what they will contribute to it.
The evaluation questions that identify vendors who will help with workflow design:
- What is your implementation methodology, and what clinical workflow design support do you provide?
- At reference sites, what workflow changes were required to achieve the adoption rates you are reporting?
- What is the typical ratio of workflow redesign effort to technology configuration effort in your implementations?
- Can we speak with the chief medical information officer and the clinical informatics lead at a reference site about their workflow experience?
The Change Management Underinvestment Pattern
Related to workflow redesign but distinct: change management is the organized effort to get clinicians to adopt and use the AI system as intended, and it is easy to underfund.
Clinical change management is hard. Physicians are trained to be skeptical of new interventions. Alert systems have a long history of producing noise. New documentation tools have often added burden rather than removing it. Clinical AI starts with low credibility among many clinicians, and earning that credibility takes sustained engagement, feedback loops, and evidence that the system improves their work.
Three change management investments to fund explicitly:
Clinical champion development. Respected clinical peers who have used the system, trust it, and can explain its value in clinical terms are the people most able to persuade skeptical colleagues. Identifying and supporting frontline champions, not only informatics staff, takes deliberate effort and protected time from those champions.
Feedback and iteration mechanisms. A clinical AI system with no way for clinicians to flag incorrect alerts, report problematic recommendations, or suggest workflow changes will lose their trust. Build a feedback loop in which clinician input visibly changes system behavior.
Performance transparency. Show clinicians real data on system performance: how many sepsis alerts were actionable, how many coding suggestions were accepted, how documentation time has changed. That evidence is what converts skeptics and sustains use.
The Ongoing Performance Monitoring Blind Spot
A final and often neglected dimension: how will you know, six months after deployment, whether the system is performing as expected?
Model performance can degrade as patient populations shift, documentation patterns change, and clinical protocols evolve. Without continuous tracking of model metrics, alert acceptance rates, outcome measures, and user satisfaction, a health system may not learn that its system is underperforming until it is well into a multi-year contract.
The evaluation questions that test a vendor's monitoring capability:
- What model monitoring infrastructure do you provide, and which specific metrics do you track?
- How are health systems notified when model performance drops below an agreed threshold?
- What is your process for retraining and revalidation when degradation is found?
- What performance guarantees are in your contract, and what remedies apply if performance falls below contracted thresholds?
If a vendor cannot give specific answers to these questions, assume you will be doing the monitoring yourself, and price that work into the decision.
A Better Evaluation Framework
Weight these five dimensions roughly equally when you score vendors:
1. Local validation. Performance on retrospective data from your EHR, your patient population, and your documentation patterns, rather than benchmark performance.
2. Reference site depth. Conversations with clinical informatics leaders and frontline clinicians at comparable sites about implementation problems and real adoption rates, rather than a list of logos.
3. Integration specifics. A technical specification of integration requirements, the realistic go-live timeline in your environment, and who owns each integration component, rather than "compatible with Epic."
4. Workflow design support. Evidence of the vendor's method for redesigning clinical workflows, with examples from other sites, rather than product demos.
5. Performance monitoring commitment. Contractual monitoring commitments, the specific metrics tracked, and remedies if performance falls below thresholds, rather than projected outcomes.
Start with local validation. Ask your shortlisted vendors to run their model on a retrospective extract from your EHR and report AUC, alert rate, and positive predictive value at the threshold you would use. A vendor that declines has answered the first question for you.
GradeCircle builds and runs six products of its own: AfroSynergy, CallerBridge, HealthSquire, Heyzl, Lexfora, and LukeAPI, two of them in beta. The assumptions in our models come from operating those systems, not from a framework. See what we run.
If you have a specific decision in front of you, scope it in five questions and see the engagement, price band, and timeline before sharing contact details.
