{"id":"AIBD-0612","title":"Epic no-show model guided reminder calls that lowered missed primary care visits at MetroHealth, with the largest drop for Black patients","summary":"At MetroHealth in Cleveland, reminder calls to patients flagged by an Epic no-show model cut no-shows from 36% to 33%, and from 42% to 36% for Black patients, in a 2022 randomized initiative.","date_observed":"2022-08-24","date_basis":"event","date_precision":"day","benefit_status":"reported","source_collection":null,"source_entry":null,"account":"MetroHealth, a safety-net health system in Cleveland, Ohio, ran a randomized quality improvement initiative in an internal medicine primary care clinic from 1 January to 24 August 2022. An Epic random forest model, retrained on local data, predicted the risk that a patient would miss an in-person appointment. For appointments with a predicted risk of 15% or higher, schedulers placed live reminder calls in the augmented arm on top of standard automated reminders. Among 5,840 high-risk appointments, the no-show rate was 33% with calls and 36% without. For Black patients the rate fell from 42% to 36%, while white non-Hispanic patients showed no reduction. The team checked the model's calibration in Black and white patients before use and left race out of its inputs. The initiative covered one clinic, and the calls themselves were the intervention that the model directed.","category":"general benefit","unexpected":false,"purpose":"The model was built to predict appointment no-shows so that staff can target outreach. Fewer missed visits is that purpose. The initiative aimed to cut no-shows without widening racial and ethnic gaps in attendance.","system":"Epic no-show prediction model (random forest), localized at MetroHealth","model_version":null,"developer":"Epic Systems Corporation","deployer":"MetroHealth","location":"MetroHealth internal medicine primary care clinic, Cleveland, Ohio, United States","domain":"Healthcare","segment":null,"benefit_type":"Improved Accessibility","taxonomy_note":null,"registry_ids":{},"ratings":[{"group":"personal","depth":2,"reason":"More high-risk patients attended their visits: the no-show rate fell from 36% to 33%, and from 42% to 36% for Black patients."},{"group":"organizational","depth":1,"reason":"Schedulers focused limited call time on appointments that the model ranked as high risk."}],"ai_role":{"status":"assessed","statement":"High-risk appointments were randomized to model-directed calls or standard reminders. The trial measures calls targeted by the model; it does not test the calls without the model or the model without the calls."},"evidence":[{"kind":"peer-reviewed study","citation":"Flagel D. Prediction without intervention: why fifteen years of no-show models have not closed the loop on patient adherence. Frontiers in Digital Health 2026;8:1949255. doi:10.3389/fdgth.2026.1949255. Peer-reviewed Perspective article with an evidence table, by an author at Purdue University; no funding and no conflicts declared.","title":"Prediction without intervention: why fifteen years of no-show models have not closed the loop on patient adherence","url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC13683762/","date":"2026-10-07","written_by":"independent","quote":"a model checked for calibration in Black and white patient strata before deployment drove live reminder calls in a safety-net system, reducing no-shows predominantly among Black patients and narrowing the disparity"},{"kind":"peer-reviewed study","citation":"Tarabichi Y, Higginbotham J, Riley N, Kaelber DC, Watts B. Reducing Disparities in No Show Rates Using Predictive Model-Driven Live Appointment Reminders for At-Risk Patients: a Randomized Controlled Quality Improvement Initiative. Journal of General Internal Medicine 2023;38(13):2921-2927. doi:10.1007/s11606-023-08209-0. Written by MetroHealth staff.","title":"Reducing Disparities in No Show Rates Using Predictive Model-Driven Live Appointment Reminders for At-Risk Patients: a Randomized Controlled Quality Improvement Initiative","url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC10150669/","date":"2023-05-01","written_by":"deployer","quote":"The augmented care group had a significantly lower no show rate than the standard care group (33% vs 36%, p < 0.01)."},{"kind":"peer-reviewed study","citation":"Same study, model description.","title":"Reducing Disparities in No Show Rates Using Predictive Model-Driven Live Appointment Reminders for At-Risk Patients","url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC10150669/","date":"2023-05-01","written_by":"deployer","quote":"The no show model is a random forest algorithm developed by the Epic Corporation (Verona, WI)."}],"replication":{"produced_by":["The Epic model was retrained on local appointment data and checked for calibration by race before use.","Schedulers called patients at 15% or higher predicted risk in addition to automated reminders."],"must_hold":["Staff time is available for live calls.","The model remains calibrated across patient groups."],"echoes":[{"date":null,"setting":"Academic primary care in the United States (Shah et al., randomized trial of targeted reminder calls to patients at high predicted risk of no-show, as summarized in a 2026 review)","model_version":null,"outcome":"recurred","source":"https://pmc.ncbi.nlm.nih.gov/articles/PMC13683762/"}]},"paired_incident":null,"headline":"No-show predictions steered reminder calls that cut missed visits","metric":"the no-show rate was 33% with calls and 36% without","comparator":{"against":"High-risk appointments randomized to standard automated reminders only","before":"36% no-show rate","after":"33% no-show rate","basis":"measured","quote":"There was a significant reduction in no show rates for Black patients (36% vs 42% respectively, p < 0.001) not reflected in white, non-Hispanic patients.","url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC10150669/"},"later":{"status":"unknown","date":null,"note":"The cited sources do not report later use at MetroHealth.","quote":null,"url":null},"value":null,"developer_country":"United States","deployer_country":"United States","unrated_because":null,"brief_notes":null,"brief":null,"revisions":[{"date":"2026-10-29","change":"Added the case.","source":"https://pmc.ncbi.nlm.nih.gov/articles/PMC13683762/","editor":"AIBD"}],"confidence":"studied","shade":"high","claim_status":"Independently studied","claim_basis":"Sources: one independent peer-reviewed study and the deployer's peer-reviewed study. The cited evidence includes no finding by a regulator or court.","case_jurisdictions":["United States"],"profile":{"case_id":"AIBD-0612","catalog_version":"1.0.0","source_fingerprint":"d4eda8269c6bf837a4d9ef566e4c1bc44d8236136e9be73d7c1256cfccf9b093","collection_refs":[{"collection_id":"aibd","version":"2026-10-29"}],"source_ids":["AIBD-0612-source-0001","AIBD-0612-source-0002","AIBD-0612-source-0003"],"families":{"strategic assessments":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"evaluations":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-0612-evaluation-1"],"role":"Evaluations of the benefit"},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":[],"source_refs":["AIBD-0612-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"partial","record_ids":["AIBD-0612-evaluation-1"]},"monetary observations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"institutional decisions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]}},"fields":{"actors[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"developer":"Epic Systems Corporation","deployer":"MetroHealth"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["developer","deployer"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"actors[].ownership_control":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_contribution":{"state":"value_present","value":{"kind":"record_fields","fields":{"ai_role":{"status":"assessed","statement":"High-risk appointments were randomized to model-directed calls or standard reminders. The trial measures calls targeted by the model; it does not test the calls without the model or the model without the calls."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"ai_relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"category":"general benefit","ai_role":{"status":"assessed","statement":"High-risk appointments were randomized to model-directed calls or standard reminders. The trial measures calls targeted by the model; it does not test the calls without the model or the model without the calls."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role","category"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].alternatives":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].assessor_date":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].depth":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":2,"reason":"More high-risk patients attended their visits: the no-show rate fell from 36% to 33%, and from 42% to 36% for Black patients."},{"group":"organizational","depth":1,"reason":"Schedulers focused limited call time on appointments that the model ranked as high risk."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].display_confidence":{"state":"value_present","value":{"kind":"structured","data":{"confidence":"high","evidence_word":"studied","claim_status":"Independently studied","basis":"Sources: one independent peer-reviewed study and the deployer's peer-reviewed study. The cited evidence includes no finding by a regulator or court.","interpretation":"Existing display rule preserved; the richer outcome-specific confidence question still needs review."}},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rationale":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":2,"reason":"More high-risk patients attended their visits: the no-show rate fell from 36% to 33%, and from 42% to 36% for Black patients."},{"group":"organizational","depth":1,"reason":"Schedulers focused limited call time on appointments that the model ranked as high risk."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rubric_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].status_basis":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].target":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":2,"reason":"More high-risk patients attended their visits: the no-show rate fell from 36% to 33%, and from 42% to 36% for Black patients."},{"group":"organizational","depth":1,"reason":"Schedulers focused limited call time on appointments that the model ranked as high risk."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"availability":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"brief":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_kind":{"state":"value_present","value":{"kind":"structured","data":{"classification":"program or activity","answer":"The case describes a program or activity. A randomized quality improvement initiative run by the health system.","scope":"The period and activity described in the case."}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["account","evidence"],"source_refs":["AIBD-0612-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The case describes a program or activity. A randomized quality improvement initiative run by the health system.","inferred":false}},"case_links[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Compared with the incidents in the AI Incident Database snapshot of 26 October 2026.","record_fields":["paired_incident"],"source_refs":[]},"reason":"No incident in the AI Incident Database snapshot of 26 October 2026 concerns the same kind of system harming the same kind of stakeholder.","review_state":"reviewed","coverage":"partial"},"claims[].evidence_links":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_strength":{"state":"value_present","value":{"kind":"structured","data":{"answer":"Independently studied. Sources: one independent peer-reviewed study and the deployer's peer-reviewed study. The cited evidence includes no finding by a regulator or court.","evidence_word":"studied","claim_status":"Independently studied"}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0612-source-0001","AIBD-0612-source-0002","AIBD-0612-source-0003"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Independently studied. Sources: one independent peer-reviewed study and the deployer's peer-reviewed study. The cited evidence includes no finding by a regulator or court.","inferred":false}},"claims[].missingness":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].proposition":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"cross_border_flows[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"decisions[].action_outcome":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].grounds":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].institution_task":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].success_criterion":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"deployment_stage":{"state":"value_present","value":{"kind":"structured","data":{"classification":"pilot","answer":"The system was used in a pilot. An eight-month randomized quality improvement initiative in one clinic.","scope":"The period and activity described in the case."}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["account","evidence"],"source_refs":["AIBD-0612-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The system was used in a pilot. An eight-month randomized quality improvement initiative in one clinic.","inferred":false}},"economics[].amount":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].costs[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].counterfactual":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].currency_price_basis":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].gross_net":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].measure_type":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].period_unit":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].realization":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].recipient_payer":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].return_payback":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].valuation_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"evaluations[].ai_uplift":{"state":"value_present","value":{"kind":"text","text":"3 percentage points fewer no-shows overall."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0612-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"3 percentage points fewer no-shows overall.","inferred":false}},"evaluations[].comparator":{"state":"value_present","value":{"kind":"text","text":"Standard automated reminders."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0612-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Standard automated reminders.","inferred":false}},"evaluations[].design":{"state":"value_present","value":{"kind":"text","text":"Randomized."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0612-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Randomized.","inferred":false}},"evaluations[].effect":{"state":"value_present","value":{"kind":"text","text":"33% versus 36% overall; 36% versus 42% among Black patients."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0612-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"33% versus 36% overall; 36% versus 42% among Black patients.","inferred":false}},"evaluations[].exposure":{"state":"value_present","value":{"kind":"text","text":"2,858 appointments in the augmented arm."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0612-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"2,858 appointments in the augmented arm.","inferred":false}},"evaluations[].id":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-0612-evaluation-1"],"role":"Evaluations of the benefit"},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0612-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Randomized.","inferred":false}},"evaluations[].measurement_method":{"state":"value_present","value":{"kind":"text","text":"Scheduling records of attended and missed appointments."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0612-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Scheduling records of attended and missed appointments.","inferred":false}},"evaluations[].metric":{"state":"value_present","value":{"kind":"text","text":"No-show rate overall and by race and ethnicity."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0612-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"No-show rate overall and by race and ethnicity.","inferred":false}},"evaluations[].reproduction_materials":{"state":"value_present","value":{"kind":"text","text":"Supplementary material; data on request."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0612-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Supplementary material; data on request.","inferred":false}},"evaluations[].setting_sample":{"state":"value_present","value":{"kind":"text","text":"5,840 appointments with predicted no-show risk of 15% or higher at a MetroHealth internal medicine clinic."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0612-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"5,840 appointments with predicted no-show risk of 15% or higher at a MetroHealth internal medicine clinic.","inferred":false}},"evaluations[].uncertainty_limitations":{"state":"value_present","value":{"kind":"text","text":"One clinic, quality improvement design, two process changes during the initiative."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0612-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"One clinic, quality improvement design, two process changes during the initiative.","inferred":false}},"external_references[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"follow_up[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"later":{"status":"unknown","note":"The cited sources do not report later use at MetroHealth."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["later"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"human_oversight":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"id":{"state":"value_present","value":{"kind":"record_fields","fields":{"id":"AIBD-0612"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["id"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"infrastructure_dependencies[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiaries":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiary_count":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].benefit_type":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_type":"Improved Accessibility"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].claim":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"MetroHealth, a safety-net health system in Cleveland, Ohio, ran a randomized quality improvement initiative in an internal medicine primary care clinic from 1 January to 24 August 2022. An Epic random forest model, retrained on local data, predicted the risk that a patient would miss an in-person appointment. For appointments with a predicted risk of 15% or higher, schedulers placed live reminder calls in the augmented arm on top of standard automated reminders. Among 5,840 high-risk appointments, the no-show rate was 33% with calls and 36% without. For Black patients the rate fell from 42% to 36%, while white non-Hispanic patients showed no reduction. The team checked the model's calibration in Black and white patients before use and left race out of its inputs. The initiative covered one clinic, and the calls themselves were the intervention that the model directed.","benefit_type":"Improved Accessibility"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].distribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].duration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].id":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].stakeholder_group":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":2,"reason":"More high-risk patients attended their visits: the no-show rate fell from 36% to 33%, and from 42% to 36% for Black patients."},{"group":"organizational","depth":1,"reason":"Schedulers focused limited call time on appointments that the model ranked as high risk."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].status":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_status":"reported"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_status"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].tradeoffs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].unexpected":{"state":"value_present","value":{"kind":"record_fields","fields":{"unexpected":false,"purpose":"The model was built to predict appointment no-shows so that staff can target outreach. Fewer missed visits is that purpose. The initiative aimed to cut no-shows without widening racial and ethnic gaps in attendance."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["unexpected","purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"places[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"location":"MetroHealth internal medicine primary care clinic, Cleveland, Ohio, United States","developer_country":"United States","deployer_country":"United States"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["location","developer_country","deployer_country"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"purpose":{"state":"value_present","value":{"kind":"record_fields","fields":{"purpose":"The model was built to predict appointment no-shows so that staff can target outreach. Fewer missed visits is that purpose. The initiative aimed to cut no-shows without widening racial and ethnic gaps in attendance."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"record_dates":{"state":"value_present","value":{"kind":"record_fields","fields":{"date_observed":"2022-08-24","date_basis":"event","date_precision":"day","revisions":[{"date":"2026-10-29","change":"Added the case.","source":"https://pmc.ncbi.nlm.nih.gov/articles/PMC13683762/","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["date_observed","date_basis","date_precision","revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"replication.attempts[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"replication":{"produced_by":["The Epic model was retrained on local appointment data and checked for calibration by race before use.","Schedulers called patients at 15% or higher predicted risk in addition to automated reminders."],"must_hold":["Staff time is available for live calls.","The model remains calibrated across patient groups."],"echoes":[{"setting":"Academic primary care in the United States (Shah et al., randomized trial of targeted reminder calls to patients at high predicted risk of no-show, as summarized in a 2026 review)","outcome":"recurred","source":"https://pmc.ncbi.nlm.nih.gov/articles/PMC13683762/"}]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"replication.conditions":{"state":"value_present","value":{"kind":"record_fields","fields":{"replication":{"produced_by":["The Epic model was retrained on local appointment data and checked for calibration by race before use.","Schedulers called patients at 15% or higher predicted risk in addition to automated reminders."],"must_hold":["Staff time is available for live calls.","The model remains calibrated across patient groups."],"echoes":[{"setting":"Academic primary care in the United States (Shah et al., randomized trial of targeted reminder calls to patients at high predicted risk of no-show, as summarized in a 2026 review)","outcome":"recurred","source":"https://pmc.ncbi.nlm.nih.gov/articles/PMC13683762/"}]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"revision_history[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"revisions":[{"date":"2026-10-29","change":"Added the case.","source":"https://pmc.ncbi.nlm.nih.gov/articles/PMC13683762/","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sector":{"state":"value_present","value":{"kind":"record_fields","fields":{"domain":"Healthcare"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["domain"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].access_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].citation_locator":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"peer-reviewed study","citation":"Flagel D. Prediction without intervention: why fifteen years of no-show models have not closed the loop on patient adherence. Frontiers in Digital Health 2026;8:1949255. doi:10.3389/fdgth.2026.1949255. Peer-reviewed Perspective article with an evidence table, by an author at Purdue University; no funding and no conflicts declared.","title":"Prediction without intervention: why fifteen years of no-show models have not closed the loop on patient adherence","url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC13683762/","date":"2026-10-07","written_by":"independent","quote":"a model checked for calibration in Black and white patient strata before deployment drove live reminder calls in a safety-net system, reducing no-shows predominantly among Black patients and narrowing the disparity"},{"kind":"peer-reviewed study","citation":"Tarabichi Y, Higginbotham J, Riley N, Kaelber DC, Watts B. Reducing Disparities in No Show Rates Using Predictive Model-Driven Live Appointment Reminders for At-Risk Patients: a Randomized Controlled Quality Improvement Initiative. Journal of General Internal Medicine 2023;38(13):2921-2927. doi:10.1007/s11606-023-08209-0. Written by MetroHealth staff.","title":"Reducing Disparities in No Show Rates Using Predictive Model-Driven Live Appointment Reminders for At-Risk Patients: a Randomized Controlled Quality Improvement Initiative","url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC10150669/","date":"2023-05-01","written_by":"deployer","quote":"The augmented care group had a significantly lower no show rate than the standard care group (33% vs 36%, p < 0.01)."},{"kind":"peer-reviewed study","citation":"Same study, model description.","title":"Reducing Disparities in No Show Rates Using Predictive Model-Driven Live Appointment Reminders for At-Risk Patients","url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC10150669/","date":"2023-05-01","written_by":"deployer","quote":"The no show model is a random forest algorithm developed by the Epic Corporation (Verona, WI)."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].id":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-0612-source-0001","AIBD-0612-source-0002","AIBD-0612-source-0003"],"role":"Existing evidence items, in their original order"},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"sources[].language":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"peer-reviewed study","citation":"Flagel D. Prediction without intervention: why fifteen years of no-show models have not closed the loop on patient adherence. Frontiers in Digital Health 2026;8:1949255. doi:10.3389/fdgth.2026.1949255. Peer-reviewed Perspective article with an evidence table, by an author at Purdue University; no funding and no conflicts declared.","title":"Prediction without intervention: why fifteen years of no-show models have not closed the loop on patient adherence","url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC13683762/","date":"2026-10-07","written_by":"independent","quote":"a model checked for calibration in Black and white patient strata before deployment drove live reminder calls in a safety-net system, reducing no-shows predominantly among Black patients and narrowing the disparity"},{"kind":"peer-reviewed study","citation":"Tarabichi Y, Higginbotham J, Riley N, Kaelber DC, Watts B. Reducing Disparities in No Show Rates Using Predictive Model-Driven Live Appointment Reminders for At-Risk Patients: a Randomized Controlled Quality Improvement Initiative. Journal of General Internal Medicine 2023;38(13):2921-2927. doi:10.1007/s11606-023-08209-0. Written by MetroHealth staff.","title":"Reducing Disparities in No Show Rates Using Predictive Model-Driven Live Appointment Reminders for At-Risk Patients: a Randomized Controlled Quality Improvement Initiative","url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC10150669/","date":"2023-05-01","written_by":"deployer","quote":"The augmented care group had a significantly lower no show rate than the standard care group (33% vs 36%, p < 0.01)."},{"kind":"peer-reviewed study","citation":"Same study, model description.","title":"Reducing Disparities in No Show Rates Using Predictive Model-Driven Live Appointment Reminders for At-Risk Patients","url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC10150669/","date":"2023-05-01","written_by":"deployer","quote":"The no show model is a random forest algorithm developed by the Epic Corporation (Verona, WI)."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].upstream_sources":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"strategic_assessments[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"strategic assessments"},"summary":{"state":"value_present","value":{"kind":"record_fields","fields":{"summary":"At MetroHealth in Cleveland, reminder calls to patients flagged by an Epic no-show model cut no-shows from 36% to 33%, and from 42% to 36% for Black patients, in a 2022 randomized initiative.","account":"MetroHealth, a safety-net health system in Cleveland, Ohio, ran a randomized quality improvement initiative in an internal medicine primary care clinic from 1 January to 24 August 2022. An Epic random forest model, retrained on local data, predicted the risk that a patient would miss an in-person appointment. For appointments with a predicted risk of 15% or higher, schedulers placed live reminder calls in the augmented arm on top of standard automated reminders. Among 5,840 high-risk appointments, the no-show rate was 33% with calls and 36% without. For Black patients the rate fell from 42% to 36%, while white non-Hispanic patients showed no reduction. The team checked the model's calibration in Black and white patients before use and left race out of its inputs. The initiative covered one clinic, and the calls themselves were the intervention that the model directed."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["summary","account"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].autonomy_permissions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].component_role":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].deployment_configuration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].inputs_outputs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_name":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].product_name":{"state":"value_present","value":{"kind":"record_fields","fields":{"system":"Epic no-show prediction model (random forest), localized at MetroHealth"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["system"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].provider":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"taxonomy_crosswalks[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"title":{"state":"value_present","value":{"kind":"record_fields","fields":{"title":"Epic no-show model guided reminder calls that lowered missed primary care visits at MetroHealth, with the largest drop for Black patients","headline":"No-show predictions steered reminder calls that cut missed visits"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["title","headline"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"version":{"state":"value_present","value":{"kind":"identifiers","identifiers":["58bb6e0243ed7f06441704837d9a6cdaafcb90295b13233b9f54971f4ee3c5d7"],"role":"SHA-256 of the complete public case and its assessments"},"provenance":{"kind":"curatorial","basis":"SHA-256 of the public case, its assessments, and its displayed answers and notes, excluding its own identifier and the derived coverage summary.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"workflow":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"MetroHealth, a safety-net health system in Cleveland, Ohio, ran a randomized quality improvement initiative in an internal medicine primary care clinic from 1 January to 24 August 2022. An Epic random forest model, retrained on local data, predicted the risk that a patient would miss an in-person appointment. For appointments with a predicted risk of 15% or higher, schedulers placed live reminder calls in the augmented arm on top of standard automated reminders. Among 5,840 high-risk appointments, the no-show rate was 33% with calls and 36% without. For Black patients the rate fell from 42% to 36%, while white non-Hispanic patients showed no reduction. The team checked the model's calibration in Black and white patients before use and left race out of its inputs. The initiative covered one clinic, and the calls themselves were the intervention that the model directed.","replication":{"produced_by":["The Epic model was retrained on local appointment data and checked for calibration by race before use.","Schedulers called patients at 15% or higher predicted risk in addition to automated reminders."],"must_hold":["Staff time is available for live calls.","The model remains calibrated across patient groups."],"echoes":[{"setting":"Academic primary care in the United States (Shah et al., randomized trial of targeted reminder calls to patients at high predicted risk of no-show, as summarized in a 2026 review)","outcome":"recurred","source":"https://pmc.ncbi.nlm.nih.gov/articles/PMC13683762/"}]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"}},"summary":{"total":91,"addressed":91,"known":43,"reviewed":15,"reviewed_complete":14,"reviewed_partial":1,"record_mapped":29,"review_not_assessed":47,"not_assessed":0,"complete":false,"by_state":{"value_present":43,"not_reported":0,"unresolved":48,"not_assessed":0,"not_applicable":0,"withheld":0,"source_unavailable":0}}},"presentation_version":"182222d067228fa770a04a9e8838192222672c2744dc927d06b410ca5da8e28b"}