{"id":"AIBD-0727","title":"Stanford evaluators find that the Allegheny Family Screening Tool made screen-in decisions more accurate, with a possible small loss for screen-outs","summary":"Stanford evaluators found that the Allegheny Family Screening Tool and related policies made screen-ins more accurate, with roughly 24 more children a month screened in accurately.","date_observed":"2017-01-01","date_basis":"event","date_precision":"year","benefit_status":"mixed","source_collection":null,"source_entry":null,"account":"Allegheny County's Department of Human Services began giving hotline call screeners the Allegheny Family Screening Tool in 2016. The tool is a predictive risk model that scores each referral from county data on the people involved. Jeremy Goldhaber-Fiebert and Lea Prince of Stanford University evaluated it in a report dated 20 March 2019, which the county published. They compared about 31,000 children in general protective service referrals before the tool with about 34,000 after full implementation in December 2016. They used interrupted time series and regression analyses because randomization was not feasible. They found that the tool and associated policies increased accuracy for children screened in for investigation. Accuracy may have slightly decreased for children screened out. Roughly 24 more children each month were screened in accurately, and the early gain faded somewhat over time. Gains were largest for white children and children under four. Changes to field screening and to screeners' roles came at the same time, so the evaluators could not credit the effect to the tool alone.","category":"aggregate result","unexpected":false,"purpose":"Developers working for Allegheny County's Department of Human Services built the Allegheny Family Screening Tool to help call screeners decide which child welfare referrals to investigate.","system":"Allegheny Family Screening Tool (AFST), a predictive risk model that scores child welfare hotline referrals","model_version":null,"developer":"Not named in the evaluation (built for the Allegheny County Department of Human Services)","deployer":"Allegheny County Department of Human Services, Office of Children, Youth and Families (hotline call screeners)","location":"Allegheny County, Pennsylvania, United States","domain":"Child welfare","segment":null,"benefit_type":"Improved Task Accuracy","work_activities":["4.A.2.a.1","4.A.2.b.1"],"taxonomy_note":null,"registry_ids":{},"ratings":[{"group":"organizational","depth":2,"reason":"Roughly 24 more children each month were screened in accurately after the tool was introduced."}],"ai_role":{"status":"unassessed","statement":"The evaluators compared periods before and after the tool, which arrived with changes to field screening and to screeners' roles. They could not fully separate the score's effect from those changes. The report does not describe how the model was built."},"evidence":[{"kind":"evaluation report","citation":"Goldhaber-Fiebert JD, Prince L. Impact Evaluation of a Predictive Risk Modeling Tool for Allegheny County's Child Welfare Office. Stanford University, 20 March 2019. Published by the Allegheny County Department of Human Services.","title":"Impact Evaluation of a Predictive Risk Modeling Tool for Allegheny County's Child Welfare Office","url":"https://analytics.alleghenycounty.us/wp-content/uploads/2019/05/Impact-Evaluation-from-16-ACDHS-26_PredictiveRisk_Package_050119_FINAL-6.pdf","date":"2019-03-20","written_by":"evaluator","quote":"Overall, our analyses showed that the AFST and associated policies increased accuracy for children screened-in for investigation and may have slightly decreased accuracy for children screened-out."}],"replication":{"produced_by":["Call screeners saw the score, or a mandatory screen-in flag, for every general protective service referral.","Screeners made recommendations to supervisors, who made the final decision."],"must_hold":["County data on the people in each referral are available when the referral arrives.","Investigative capacity grows if more children are screened in."],"echoes":[]},"paired_incident":null,"headline":"*Allegheny Family Screening Tool* makes screen-in calls more accurate","metric":"Roughly 24 more children each month were screened in accurately","comparator":{"against":"General protective service referrals before the tool, January 2015 to July 2016","before":null,"after":"Roughly 24 more children each month screened in accurately","basis":"measured","quote":"Roughly 24 more children each month screen-in accurately after the AFST, with over half of these children in the 7 to 12 year old age range and almost all of these children in the white race group.","url":"https://analytics.alleghenycounty.us/wp-content/uploads/2019/05/Impact-Evaluation-from-16-ACDHS-26_PredictiveRisk_Package_050119_FINAL-6.pdf"},"later":null,"value":null,"developer_country":null,"deployer_country":"United States","unrated_because":null,"brief_notes":null,"brief":null,"revisions":[{"date":"2026-10-31","change":"Added the record from the report of the Stanford evaluators.","source":"https://analytics.alleghenycounty.us/wp-content/uploads/2019/05/Impact-Evaluation-from-16-ACDHS-26_PredictiveRisk_Package_050119_FINAL-6.pdf","editor":"AIBD"}],"confidence":"claimed","shade":"low","claim_status":"The evaluator's account","claim_basis":"Sources: the evaluators' evaluation report. The cited evidence includes no independent source and no finding by an external body.","case_jurisdictions":["United States"],"profile":{"case_id":"AIBD-0727","catalog_version":"1.0.0","source_fingerprint":"479de47898f62f7c4aeccb9f773fd6a1d5426efaf9952aebf22d30101418811d","collection_refs":[{"collection_id":"aibd","version":"2026-10-31"},{"collection_id":"querist","version":"2026-10-25"}],"source_ids":["AIBD-0727-source-0001"],"families":{"strategic assessments":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"evaluations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"monetary observations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"institutional decisions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]}},"fields":{"actors[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"developer":"Not named in the evaluation (built for the Allegheny County Department of Human Services)","deployer":"Allegheny County Department of Human Services, Office of Children, Youth and Families (hotline call screeners)"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["developer","deployer"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"actors[].ownership_control":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_contribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"category":"aggregate result","ai_role":{"status":"unassessed","statement":"The evaluators compared periods before and after the tool, which arrived with changes to field screening and to screeners' roles. They could not fully separate the score's effect from those changes. The report does not describe how the model was built."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role","category"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].alternatives":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].assessor_date":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].depth":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"Roughly 24 more children each month were screened in accurately after the tool was introduced."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].display_confidence":{"state":"value_present","value":{"kind":"structured","data":{"confidence":"low","evidence_word":"claimed","claim_status":"The evaluator's account","basis":"Sources: the evaluators' evaluation report. The cited evidence includes no independent source and no finding by an external body.","interpretation":"Existing display rule preserved; the richer outcome-specific confidence question still needs review."}},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rationale":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"Roughly 24 more children each month were screened in accurately after the tool was introduced."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rubric_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].status_basis":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].target":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"Roughly 24 more children each month were screened in accurately after the tool was introduced."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"availability":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"brief":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_kind":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_links[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_links":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_strength":{"state":"value_present","value":{"kind":"structured","data":{"answer":"The evaluator's account. Sources: the evaluators' evaluation report. The cited evidence includes no independent source and no finding by an external body.","evidence_word":"claimed","claim_status":"The evaluator's account"}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0727-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The evaluator's account. Sources: the evaluators' evaluation report. The cited evidence includes no independent source and no finding by an external body.","inferred":false}},"claims[].missingness":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].proposition":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"cross_border_flows[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"decisions[].action_outcome":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].grounds":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].institution_task":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].success_criterion":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"deployment_stage":{"state":"value_present","value":{"kind":"structured","data":{"classification":"routine use","answer":"The system was in routine use. Call screeners used the score on general protective service referrals from 2016.","scope":"The period and activity described in the case."}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["account","evidence"],"source_refs":["AIBD-0727-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The system was in routine use. Call screeners used the score on general protective service referrals from 2016.","inferred":false}},"economics[].amount":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].costs[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].counterfactual":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].currency_price_basis":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].gross_net":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].measure_type":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].period_unit":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].realization":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].recipient_payer":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].return_payback":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].valuation_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"evaluations[].ai_uplift":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].comparator":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].design":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].effect":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].exposure":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].id":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].measurement_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].metric":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].reproduction_materials":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].setting_sample":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].uncertainty_limitations":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"external_references[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"follow_up[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"human_oversight":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"id":{"state":"value_present","value":{"kind":"record_fields","fields":{"id":"AIBD-0727"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["id"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"infrastructure_dependencies[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiaries":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiary_count":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].benefit_type":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_type":"Improved Task Accuracy"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].claim":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"Allegheny County's Department of Human Services began giving hotline call screeners the Allegheny Family Screening Tool in 2016. The tool is a predictive risk model that scores each referral from county data on the people involved. Jeremy Goldhaber-Fiebert and Lea Prince of Stanford University evaluated it in a report dated 20 March 2019, which the county published. They compared about 31,000 children in general protective service referrals before the tool with about 34,000 after full implementation in December 2016. They used interrupted time series and regression analyses because randomization was not feasible. They found that the tool and associated policies increased accuracy for children screened in for investigation. Accuracy may have slightly decreased for children screened out. Roughly 24 more children each month were screened in accurately, and the early gain faded somewhat over time. Gains were largest for white children and children under four. Changes to field screening and to screeners' roles came at the same time, so the evaluators could not credit the effect to the tool alone.","benefit_type":"Improved Task Accuracy"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].distribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].duration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].id":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].stakeholder_group":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"Roughly 24 more children each month were screened in accurately after the tool was introduced."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].status":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_status":"mixed"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_status"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].tradeoffs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].unexpected":{"state":"value_present","value":{"kind":"record_fields","fields":{"unexpected":false,"purpose":"Developers working for Allegheny County's Department of Human Services built the Allegheny Family Screening Tool to help call screeners decide which child welfare referrals to investigate."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["unexpected","purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"places[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"location":"Allegheny County, Pennsylvania, United States","deployer_country":"United States"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["location","deployer_country"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"purpose":{"state":"value_present","value":{"kind":"record_fields","fields":{"purpose":"Developers working for Allegheny County's Department of Human Services built the Allegheny Family Screening Tool to help call screeners decide which child welfare referrals to investigate."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"record_dates":{"state":"value_present","value":{"kind":"record_fields","fields":{"date_observed":"2017-01-01","date_basis":"event","date_precision":"year","revisions":[{"date":"2026-10-31","change":"Added the record from the report of the Stanford evaluators.","source":"https://analytics.alleghenycounty.us/wp-content/uploads/2019/05/Impact-Evaluation-from-16-ACDHS-26_PredictiveRisk_Package_050119_FINAL-6.pdf","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["date_observed","date_basis","date_precision","revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"replication.attempts[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"replication.conditions":{"state":"value_present","value":{"kind":"record_fields","fields":{"replication":{"produced_by":["Call screeners saw the score, or a mandatory screen-in flag, for every general protective service referral.","Screeners made recommendations to supervisors, who made the final decision."],"must_hold":["County data on the people in each referral are available when the referral arrives.","Investigative capacity grows if more children are screened in."],"echoes":[]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"revision_history[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"revisions":[{"date":"2026-10-31","change":"Added the record from the report of the Stanford evaluators.","source":"https://analytics.alleghenycounty.us/wp-content/uploads/2019/05/Impact-Evaluation-from-16-ACDHS-26_PredictiveRisk_Package_050119_FINAL-6.pdf","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sector":{"state":"value_present","value":{"kind":"record_fields","fields":{"domain":"Child welfare"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["domain"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].access_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].citation_locator":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"evaluation report","citation":"Goldhaber-Fiebert JD, Prince L. Impact Evaluation of a Predictive Risk Modeling Tool for Allegheny County's Child Welfare Office. Stanford University, 20 March 2019. Published by the Allegheny County Department of Human Services.","title":"Impact Evaluation of a Predictive Risk Modeling Tool for Allegheny County's Child Welfare Office","url":"https://analytics.alleghenycounty.us/wp-content/uploads/2019/05/Impact-Evaluation-from-16-ACDHS-26_PredictiveRisk_Package_050119_FINAL-6.pdf","date":"2019-03-20","written_by":"evaluator","quote":"Overall, our analyses showed that the AFST and associated policies increased accuracy for children screened-in for investigation and may have slightly decreased accuracy for children screened-out."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].id":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-0727-source-0001"],"role":"Existing evidence items, in their original order"},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"sources[].language":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"evaluation report","citation":"Goldhaber-Fiebert JD, Prince L. Impact Evaluation of a Predictive Risk Modeling Tool for Allegheny County's Child Welfare Office. Stanford University, 20 March 2019. Published by the Allegheny County Department of Human Services.","title":"Impact Evaluation of a Predictive Risk Modeling Tool for Allegheny County's Child Welfare Office","url":"https://analytics.alleghenycounty.us/wp-content/uploads/2019/05/Impact-Evaluation-from-16-ACDHS-26_PredictiveRisk_Package_050119_FINAL-6.pdf","date":"2019-03-20","written_by":"evaluator","quote":"Overall, our analyses showed that the AFST and associated policies increased accuracy for children screened-in for investigation and may have slightly decreased accuracy for children screened-out."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].upstream_sources":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"strategic_assessments[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"strategic assessments"},"summary":{"state":"value_present","value":{"kind":"record_fields","fields":{"summary":"Stanford evaluators found that the Allegheny Family Screening Tool and related policies made screen-ins more accurate, with roughly 24 more children a month screened in accurately.","account":"Allegheny County's Department of Human Services began giving hotline call screeners the Allegheny Family Screening Tool in 2016. The tool is a predictive risk model that scores each referral from county data on the people involved. Jeremy Goldhaber-Fiebert and Lea Prince of Stanford University evaluated it in a report dated 20 March 2019, which the county published. They compared about 31,000 children in general protective service referrals before the tool with about 34,000 after full implementation in December 2016. They used interrupted time series and regression analyses because randomization was not feasible. They found that the tool and associated policies increased accuracy for children screened in for investigation. Accuracy may have slightly decreased for children screened out. Roughly 24 more children each month were screened in accurately, and the early gain faded somewhat over time. Gains were largest for white children and children under four. Changes to field screening and to screeners' roles came at the same time, so the evaluators could not credit the effect to the tool alone."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["summary","account"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].autonomy_permissions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].component_role":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].deployment_configuration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].inputs_outputs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_name":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].product_name":{"state":"value_present","value":{"kind":"record_fields","fields":{"system":"Allegheny Family Screening Tool (AFST), a predictive risk model that scores child welfare hotline referrals"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["system"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].provider":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"taxonomy_crosswalks[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"title":{"state":"value_present","value":{"kind":"record_fields","fields":{"title":"Stanford evaluators find that the Allegheny Family Screening Tool made screen-in decisions more accurate, with a possible small loss for screen-outs","headline":"*Allegheny Family Screening Tool* makes screen-in calls more accurate"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["title","headline"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"version":{"state":"value_present","value":{"kind":"identifiers","identifiers":["2b40d5c7ee49b0f6a867a1c0828dcb5b88ea95b0cd08de78c9b94c070126d518"],"role":"SHA-256 of the complete public record and its assessments"},"provenance":{"kind":"curatorial","basis":"SHA-256 of the public record, its assessments, and its displayed answers and notes, excluding its own identifier and the derived coverage summary.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"workflow":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"Allegheny County's Department of Human Services began giving hotline call screeners the Allegheny Family Screening Tool in 2016. The tool is a predictive risk model that scores each referral from county data on the people involved. Jeremy Goldhaber-Fiebert and Lea Prince of Stanford University evaluated it in a report dated 20 March 2019, which the county published. They compared about 31,000 children in general protective service referrals before the tool with about 34,000 after full implementation in December 2016. They used interrupted time series and regression analyses because randomization was not feasible. They found that the tool and associated policies increased accuracy for children screened in for investigation. Accuracy may have slightly decreased for children screened out. Roughly 24 more children each month were screened in accurately, and the early gain faded somewhat over time. Gains were largest for white children and children under four. Changes to field screening and to screeners' roles came at the same time, so the evaluators could not credit the effect to the tool alone.","replication":{"produced_by":["Call screeners saw the score, or a mandatory screen-in flag, for every general protective service referral.","Screeners made recommendations to supervisors, who made the final decision."],"must_hold":["County data on the people in each referral are available when the referral arrives.","Investigative capacity grows if more children are screened in."],"echoes":[]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"}},"summary":{"total":91,"addressed":91,"known":28,"reviewed":2,"reviewed_complete":2,"reviewed_partial":0,"record_mapped":26,"review_not_assessed":63,"not_assessed":0,"complete":false,"by_state":{"value_present":28,"not_reported":0,"unresolved":63,"not_assessed":0,"not_applicable":0,"withheld":0,"source_unavailable":0}}},"presentation_version":"34e72459a68c184238060f8a82419521cb4b70815db82d37189dc89ca7e64f88"}