Benefits
- Benefit
-
The no-show rate was 33% with calls and 36% without
Awaiting review
- Outcome identifiers
-
Unresolved
Not reviewedUnresolved
- Category
-
Improved Accessibility
Awaiting review
- Status
-
Reported benefit
Awaiting review
- Beneficiary groups
-
People. Organizations.
Awaiting review
- Beneficiaries
-
Unresolved
Not reviewedUnresolved
- Reach
-
Unresolved
Not reviewedUnresolved
- Distribution and access
-
Unresolved
Not reviewedUnresolved
- Duration
-
Unresolved
Not reviewedUnresolved
- Expected?
-
Yes
Awaiting review
- Costs and harms
-
Unresolved
Not reviewedUnresolved
AI system
- System
-
Epic no-show prediction model (random forest), localized at MetroHealth
Awaiting review
- Model
-
This is an explicit unresolved assignment: the sources on record do not settle this question.
Not reviewedUnresolved
- Release
-
Unresolved
Not reviewedUnresolved
- Provider
-
Unresolved
Not reviewedUnresolved
- Role
-
Unresolved
Not reviewedUnresolved
- Configuration
-
Unresolved
Not reviewedUnresolved
- Inputs and outputs
-
Unresolved
Not reviewedUnresolved
- Autonomy
-
Unresolved
Not reviewedUnresolved
- Workflow
-
The Epic model was retrained on local appointment data and checked for calibration by race before use. Schedulers called patients at 15% or higher predicted risk in addition to automated reminders.
Awaiting review
- Human oversight
-
Unresolved
Not reviewedUnresolved
- Relation to AI
-
High-risk appointments were randomized to model-directed calls or standard reminders. The trial measures calls targeted by the model; it does not test the calls without the model or the model without the calls.
Awaiting review
- AI contribution
-
High-risk appointments were randomized to model-directed calls or standard reminders. The trial measures calls targeted by the model; it does not test the calls without the model or the model without the calls.
Awaiting review
Measurement
- Studies
-
Randomized. [2]
Fully reviewed: supporting details for Studies
Review date: 29 October 2026.
Sources: [2]
- Study design
-
Randomized. [2]
Fully reviewed: supporting details for Study design
Review date: 29 October 2026.
Sources: [2]
- Measurement
-
Scheduling records of attended and missed appointments. [2]
Fully reviewed: supporting details for Measurement
Review date: 29 October 2026.
Sources: [2]
- Setting and sample
-
5,840 appointments with predicted no-show risk of 15% or higher at a MetroHealth internal medicine clinic. [2]
Fully reviewed: supporting details for Setting and sample
Review date: 29 October 2026.
Sources: [2]
- Measures
-
No-show rate overall and by race and ethnicity. [2]
Fully reviewed: supporting details for Measures
Review date: 29 October 2026.
Sources: [2]
- Comparison
-
Standard automated reminders. [2]
Fully reviewed: supporting details for Comparison
Review date: 29 October 2026.
Sources: [2]
- Results
-
33% versus 36% overall; 36% versus 42% among Black patients. [2]
Fully reviewed: supporting details for Results
Review date: 29 October 2026.
Sources: [2]
- Additional value from AI
-
3 percentage points fewer no-shows overall. [2]
Fully reviewed: supporting details for Additional value from AI
Review date: 29 October 2026.
Sources: [2]
- Limitations
-
One clinic, quality improvement design, two process changes during the initiative. [2]
Fully reviewed: supporting details for Limitations
Review date: 29 October 2026.
Sources: [2]
Money
- Kind of value
-
The kind of monetary value is unclear.
Not reviewedUnresolved
- Amount
-
The monetary value is not established.
Not reviewedUnresolved
- Currency and prices
-
The currency and price basis are unclear.
Not reviewedUnresolved
- Period and unit
-
The period and unit of the monetary estimate are unclear.
Not reviewedUnresolved
- Realization
-
Whether the monetary value was realized is unclear.
Not reviewedUnresolved
- Recipients and payers
-
The recipients and payers are unclear.
Not reviewedUnresolved
- Gross or net
-
Whether the monetary figure includes costs is unclear.
Not reviewedUnresolved
- Included costs
-
The costs included in the estimate are unclear.
Not reviewedUnresolved
- Comparison
-
The financial comparison is not established.
Not reviewedUnresolved
- Method
-
The method used to calculate monetary value is unclear.
Not reviewedUnresolved
- Return and payback
-
The return on investment or payback period is not established.
Not reviewedUnresolved
People and places
- Participants
-
Developer: Epic Systems Corporation. Operator: MetroHealth.
Awaiting review
- Jurisdictions
-
Benefit: MetroHealth internal medicine primary care clinic, Cleveland, Ohio,
United States. Developer: United States. Operator: United States.Awaiting review
- Ownership and control
-
Unresolved
Not reviewedUnresolved
- Dependencies
-
Unresolved
Not reviewedUnresolved
- Cross-border connections
-
Unresolved
Not reviewedUnresolved
- Geopolitical implications
-
The geopolitical implications are unclear.
Not reviewedUnresolved
Follow-up
- Conditions for replication
-
Staff time is available for live calls. The model remains calibrated across patient groups.
Awaiting review
- Replication attempts
-
1 follow-up entry. Benefit recurred: 1.
Awaiting review
- Later findings
-
The cited sources do not report later use at MetroHealth.
Awaiting review
- Availability
-
Unresolved
Not reviewedUnresolved
- Institutional decision
-
A relevant institutional decision has not been established.
Not reviewedUnresolved
- Grounds
-
The grounds for an institutional decision are unclear.
Not reviewedUnresolved
- Action and result
-
The result of an institutional decision is not established.
Not reviewedUnresolved
- Success criterion
-
The criterion for a successful institutional decision is unclear.
Not reviewedUnresolved
Evidence
- Source numbers
-
AIBD-0612-source-0001–0003
Awaiting review
- Citations
-
Numbered references below.
Awaiting review
- Language
-
Unresolved
Not reviewedUnresolved
- Source relationships
-
Independent, Operator
Awaiting review
- Original evidence
-
Unresolved
Not reviewedUnresolved
- Access and versions
-
Unresolved
Not reviewedUnresolved
- Supported findings
-
Unresolved
Not reviewedUnresolved
- Evidence links
-
Unresolved
Not reviewedUnresolved
- Evidence strength
-
Independently studied. Sources: one independent peer-reviewed study and the deployer's peer-reviewed study. The cited evidence includes no finding by a regulator or court. [1] [2]
- Missing evidence
-
Unresolved
Not reviewedUnresolved
Grades
- Assessed benefits
-
People. Organizations.
Awaiting review
- Depth
-
People: 2. Organizations: 1.
Awaiting review
- Reasons
-
People: More high-risk patients attended their visits: the no-show rate fell from 36% to 33%, and from 42% to 36% for Black patients. Organizations: Schedulers focused limited call time on appointments that the model ranked as high risk.
Awaiting review
- Observed or intended
-
Unresolved
Not reviewedUnresolved
- Rubric
-
Unresolved
Not reviewedUnresolved
- Assessor and date
-
Unresolved
Not reviewedUnresolved
- Alternative assessments
-
Unresolved
Not reviewedUnresolved
- Fill
-
Independently studied
Awaiting review
Entry details
- Kind
-
The case describes a program or activity. A randomized quality improvement initiative run by the health system. [1]
Fully reviewed: supporting details for Kind
Review date: 29 October 2026.
Sources: [1]
- Related entries
-
No incident in the AI Incident Database snapshot of 26 October 2026 concerns the same kind of system harming the same kind of stakeholder.
Partly reviewedUnresolved: supporting details for Related entries
Compared with the incidents in the AI Incident Database snapshot of 26 October 2026.
- Field
-
Healthcare.
Awaiting review
- Purpose
-
The model was built to predict appointment no-shows so that staff can target outreach. Fewer missed visits is that purpose. The initiative aimed to cut no-shows without widening racial and ethnic gaps in attendance.
Awaiting review
- Dates
-
Observed: 24 August 2022. Updated: 29 October 2026.
Awaiting review
- Stage
-
The system was used in a pilot. An eight-month randomized quality improvement initiative in one clinic. [1]
Fully reviewed: supporting details for Stage
Review date: 29 October 2026.
Sources: [1]
- Version
-
SHA-256: 58bb6e0243ed…
Awaiting review: supporting details for Version
SHA-256 of the public case, its assessments, and its displayed answers and notes, excluding its own identifier and the derived coverage summary.
References
- Other identifiers
-
Unresolved
Not reviewedUnresolved
- Category mappings
-
Unresolved
Not reviewedUnresolved
- Citable summary
-
Unresolved
Not reviewedUnresolved
- Revisions
-
1 revision. Latest: 29 October 2026.
Awaiting review
Sources
-
Peer-reviewed study Independent author
Flagel D. Prediction without intervention: why fifteen years of no-show models have not closed the loop on patient adherence. Frontiers in Digital Health 2026;8:1949255. doi:10.3389/fdgth.2026.1949255. Peer-reviewed Perspective article with an evidence table, by an author at Purdue University; no funding and no conflicts declared.
a model checked for calibration in Black and white patient strata before deployment drove live reminder calls in a safety-net system, reducing no-shows predominantly among Black patients and narrowing the disparity
-
Peer-reviewed study Deployer
Tarabichi Y, Higginbotham J, Riley N, Kaelber DC, Watts B. Reducing Disparities in No Show Rates Using Predictive Model-Driven Live Appointment Reminders for At-Risk Patients: a Randomized Controlled Quality Improvement Initiative. Journal of General Internal Medicine 2023;38(13):2921-2927. doi:10.1007/s11606-023-08209-0. Written by MetroHealth staff.
The augmented care group had a significantly lower no show rate than the standard care group (33% vs 36%, p < 0.01).
The no show model is a random forest algorithm developed by the Epic Corporation (Verona, WI).