{"id":"AIBD-0635","title":"TIGTA finds that the IRS Return Review Program's identity-theft models alone selected $1.88 billion of confirmed identity-theft refunds in the 2015 filing season","summary":"TIGTA found that IRS Return Review Program identity-theft models alone selected $1.88 billion, or 47.8%, of $3.92 billion in confirmed identity-theft refunds in 2015.","date_observed":"2015-01-01","date_basis":"event","date_precision":"year","benefit_status":"reported","source_collection":null,"source_entry":null,"account":"The Internal Revenue Service built the Return Review Program to screen individual returns that claim refunds before the refunds are paid and to replace its older Electronic Fraud Detection System. The Treasury Inspector General for Tax Administration writes that both systems use predictive models developed with proprietary machine-learning algorithms as their main detection method. In an audit published on 25 September 2017, the Inspector General examined data from the 2015 filing season, in which the IRS's fraud detection systems together identified 700,560 confirmed identity-theft returns worth $3.92 billion. Selections made only by the program's identity-theft models totaled $1.88 billion, or 47.8% of that amount, while selections made only by the older system's models totaled $60 million, or 1.5%. The Inspector General judged that the IRS made a reasonable decision to retire the older identity-theft models, which it did for the 2016 filing season. In a report dated 24 July 2018, the Government Accountability Office relays IRS figures showing that the program prevented more than $6.51 billion in invalid refunds between January 2015 and November 2017, including about $4.4 billion in the 2017 filing season. The dollar figures come from IRS data. The audit counts unique selections by each system and does not compare the program with no screening, and the program also uses business rules besides its models.","category":"general benefit","unexpected":false,"purpose":"The Return Review Program was built to detect identity-theft and other refund fraud before the IRS issues refunds. The benefit is that purpose.","system":"IRS Return Review Program (RRP) identity-theft models","model_version":null,"developer":"Internal Revenue Service (Information Technology organization and Office of Research, Applied Analytics, and Statistics)","deployer":"Internal Revenue Service","location":"United States (national)","domain":"Tax administration","segment":null,"benefit_type":"Improved Task Accuracy","taxonomy_note":null,"registry_ids":{},"ratings":[{"group":"organizational","depth":2,"reason":"The program's identity-theft models were the only system to select $1.88 billion of the $3.92 billion in confirmed identity-theft refunds in the 2015 filing season."},{"group":"societal","depth":2,"reason":"The IRS reports that the program prevented more than $6.51 billion in invalid refunds from January 2015 to November 2017, as relayed by GAO."}],"ai_role":{"status":"unassessed","statement":"The audit compares which fraud detection system selected each confirmed return. It does not compare the program with no screening, and the program also uses business rules besides its machine-learning models."},"evidence":[{"kind":"report","citation":"Treasury Inspector General for Tax Administration. The Return Review Program Increases Fraud Detection; However, Full Retirement of the Electronic Fraud Detection System Will Be Delayed. Reference Number 2017-20-080, 25 September 2017.","title":"The Return Review Program Increases Fraud Detection; However, Full Retirement of the Electronic Fraud Detection System Will Be Delayed","url":"https://www.tigta.gov/sites/default/files/reports/2022-02/201720080fr.pdf","date":"2017-09-25","written_by":"independent","quote":"In comparison, the RRP IDT model selections that were not selected by any other fraud detection system totaled $1.88 billion, accounting for 47.8 percent of the total $3.92 billion IRS IDT revenue protection."},{"kind":"report","citation":"U.S. Government Accountability Office. Tax Fraud and Noncompliance: IRS Could Further Leverage the Return Review Program to Strengthen Tax Enforcement. GAO-18-544, July 2018.","title":"Tax Fraud and Noncompliance: IRS Could Further Leverage the Return Review Program to Strengthen Tax Enforcement","url":"https://www.gao.gov/assets/gao-18-544.pdf","date":"2018-07-24","written_by":"independent","quote":"IRS reported that between January 2015 and November 2017, RRP prevented the issuance of more than $6.51 billion in invalid refunds."}],"replication":{"produced_by":["The IRS screened every individual return that claimed a refund before paying it.","The models used prior-year returns and third-party information returns such as W-2 forms.","IRS staff adjusted the models and the fraud score settings to balance fraud caught against legitimate refunds delayed."],"must_hold":["Prior-year returns and third-party data must be available when new returns are scored.","Score settings must keep false detections low enough that legitimate taxpayers are not burdened."],"echoes":[{"date":"2017-01-01","setting":"IRS 2017 filing season, in which GAO relays IRS figures that the program prevented about $4.4 billion in invalid refunds","model_version":null,"outcome":"recurred","source":"https://www.gao.gov/assets/gao-18-544.pdf"}]},"paired_incident":{"database":"AI Incident Database","identifier":"101","title":"Dutch Families Wrongfully Accused of Tax Fraud Due to Discriminatory Algorithm","url":"https://incidentdatabase.ai/cite/101","severity":null,"group":"organizational","basis":"Both involve a tax authority that uses a fraud-risk model to screen claims for payment, and in the incident the tax authority itself was harmed when its model falsely accused thousands of families of fraud."},"headline":"*Return Review Program* catches refund fraud that older filters missed","metric":"selected $1.88 billion, or 47.8%, of $3.92 billion in confirmed identity-theft refunds","comparator":{"against":"The older Electronic Fraud Detection System identity-theft models","before":"$60 million (1.5%) of confirmed identity-theft refunds selected only by the older system","after":"$1.88 billion (47.8%) selected only by the Return Review Program models","basis":"measured","quote":"With the EFDS IDT models contributing only 1.5 percent of the $3.92 billion in total confirmed IDT selections not already detected by the RRP or the DDb in the 2015 Filing Season and the risk of increasing the RRP FDR (thus increasing taxpayer burden), we believe that the IRS made a reasonable decision to retire the EFDS IDT models.","url":"https://www.tigta.gov/sites/default/files/reports/2022-02/201720080fr.pdf"},"later":{"status":"persisted","date":"2018-07-24","note":"GAO relays IRS figures showing that the program kept stopping invalid refunds through November 2017.","quote":"IRS reported that between January 2015 and November 2017, RRP prevented the issuance of more than $6.51 billion in invalid refunds.","url":"https://www.gao.gov/assets/gao-18-544.pdf"},"value":null,"developer_country":"United States","deployer_country":"United States","unrated_because":null,"brief_notes":null,"brief":null,"revisions":[{"date":"2026-10-29","change":"Added the case.","source":"https://www.tigta.gov/sites/default/files/reports/2022-02/201720080fr.pdf","editor":"AIBD"},{"date":"2026-10-29","change":"Linked a comparable incident in the AI Incident Database (incident 101).","source":"https://incidentdatabase.ai/cite/101","editor":"AIBD"}],"confidence":"studied","shade":"high","claim_status":"Independently studied","claim_basis":"Sources: two independent reports. The cited evidence includes no finding by a regulator or court.","case_jurisdictions":["United States"],"profile":{"case_id":"AIBD-0635","catalog_version":"1.0.0","source_fingerprint":"378268edf6b77992ff0d97512875a84ce274ee8cbde28c6d7fcb7b379f92d82c","collection_refs":[{"collection_id":"aibd","version":"2026-10-29"}],"source_ids":["AIBD-0635-source-0001","AIBD-0635-source-0002"],"families":{"strategic assessments":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"evaluations":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-0635-evaluation-1"],"role":"Evaluations of the benefit"},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":[],"source_refs":["AIBD-0635-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"partial","record_ids":["AIBD-0635-evaluation-1"]},"monetary observations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"institutional decisions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]}},"fields":{"actors[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"developer":"Internal Revenue Service (Information Technology organization and Office of Research, Applied Analytics, and Statistics)","deployer":"Internal Revenue Service"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["developer","deployer"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"actors[].ownership_control":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_contribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"category":"general benefit","ai_role":{"status":"unassessed","statement":"The audit compares which fraud detection system selected each confirmed return. It does not compare the program with no screening, and the program also uses business rules besides its machine-learning models."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role","category"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].alternatives":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].assessor_date":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].depth":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"The program's identity-theft models were the only system to select $1.88 billion of the $3.92 billion in confirmed identity-theft refunds in the 2015 filing season."},{"group":"societal","depth":2,"reason":"The IRS reports that the program prevented more than $6.51 billion in invalid refunds from January 2015 to November 2017, as relayed by GAO."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].display_confidence":{"state":"value_present","value":{"kind":"structured","data":{"confidence":"high","evidence_word":"studied","claim_status":"Independently studied","basis":"Sources: two independent reports. The cited evidence includes no finding by a regulator or court.","interpretation":"Existing display rule preserved; the richer outcome-specific confidence question still needs review."}},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rationale":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"The program's identity-theft models were the only system to select $1.88 billion of the $3.92 billion in confirmed identity-theft refunds in the 2015 filing season."},{"group":"societal","depth":2,"reason":"The IRS reports that the program prevented more than $6.51 billion in invalid refunds from January 2015 to November 2017, as relayed by GAO."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rubric_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].status_basis":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].target":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"The program's identity-theft models were the only system to select $1.88 billion of the $3.92 billion in confirmed identity-theft refunds in the 2015 filing season."},{"group":"societal","depth":2,"reason":"The IRS reports that the program prevented more than $6.51 billion in invalid refunds from January 2015 to November 2017, as relayed by GAO."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"availability":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"brief":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_kind":{"state":"value_present","value":{"kind":"structured","data":{"classification":"program or activity","answer":"The case describes a program or activity. The record describes a national screening program assessed by auditors.","scope":"The period and activity described in the case."}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["account","evidence"],"source_refs":["AIBD-0635-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The case describes a program or activity. The record describes a national screening program assessed by auditors.","inferred":false}},"case_links[]":{"state":"value_present","value":{"kind":"structured","data":{"answer":"Both involve a tax authority that uses a fraud-risk model to screen claims for payment, and in the incident the tax authority itself was harmed when its model falsely accused thousands of families of fraud.","links":[{"database":"AI Incident Database","identifier":"101","title":"Dutch Families Wrongfully Accused of Tax Fraud Due to Discriminatory Algorithm","url":"https://incidentdatabase.ai/cite/101","group":"organizational","basis":"Both involve a tax authority that uses a fraud-risk model to screen claims for payment, and in the incident the tax authority itself was harmed when its model falsely accused thousands of families of fraud.","relation":"comparable_harm"}]}},"provenance":{"kind":"curatorial","basis":"Compared with the incidents in the AI Incident Database snapshot of 26 October 2026; a second reading confirmed that the incident concerns the same kind of system and stakeholder.","record_fields":["paired_incident"],"source_refs":["https://incidentdatabase.ai/cite/101"]},"review_state":"reviewed","coverage":"complete","display":{"answer":"Both involve a tax authority that uses a fraud-risk model to screen claims for payment, and in the incident the tax authority itself was harmed when its model falsely accused thousands of families of fraud.","inferred":false}},"claims[].evidence_links":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_strength":{"state":"value_present","value":{"kind":"structured","data":{"answer":"Independently studied. Sources: two independent reports. The cited evidence includes no finding by a regulator or court.","evidence_word":"studied","claim_status":"Independently studied"}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0635-source-0001","AIBD-0635-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Independently studied. Sources: two independent reports. The cited evidence includes no finding by a regulator or court.","inferred":false}},"claims[].missingness":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].proposition":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"cross_border_flows[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"decisions[].action_outcome":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].grounds":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].institution_task":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].success_criterion":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"deployment_stage":{"state":"value_present","value":{"kind":"structured","data":{"classification":"routine use","answer":"The system was in routine use. The program screens refund returns nationally in every filing season.","scope":"The period and activity described in the case."}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["account","evidence"],"source_refs":["AIBD-0635-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The system was in routine use. The program screens refund returns nationally in every filing season.","inferred":false}},"economics[].amount":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].costs[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].counterfactual":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].currency_price_basis":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].gross_net":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].measure_type":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].period_unit":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].realization":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].recipient_payer":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].return_payback":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].valuation_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"evaluations[].ai_uplift":{"state":"value_present","value":{"kind":"text","text":"Not isolated from the program's business rules."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0635-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Not isolated from the program's business rules.","inferred":false}},"evaluations[].comparator":{"state":"value_present","value":{"kind":"text","text":"Electronic Fraud Detection System and the Dependent Database."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0635-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Electronic Fraud Detection System and the Dependent Database.","inferred":false}},"evaluations[].design":{"state":"value_present","value":{"kind":"text","text":"Other."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0635-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Other.","inferred":false}},"evaluations[].effect":{"state":"value_present","value":{"kind":"text","text":"$1.88 billion (47.8%) unique to the Return Review Program models against $60 million (1.5%) unique to the older system."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0635-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"$1.88 billion (47.8%) unique to the Return Review Program models against $60 million (1.5%) unique to the older system.","inferred":false}},"evaluations[].exposure":{"state":"value_present","value":{"kind":"text","text":"All electronically filed individual returns claiming refunds."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0635-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"All electronically filed individual returns claiming refunds.","inferred":false}},"evaluations[].id":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-0635-evaluation-1"],"role":"Evaluations of the benefit"},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0635-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Other.","inferred":false}},"evaluations[].measurement_method":{"state":"value_present","value":{"kind":"text","text":"Audit of IRS filing-season data on which detection systems selected each confirmed identity-theft return."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0635-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Audit of IRS filing-season data on which detection systems selected each confirmed identity-theft return.","inferred":false}},"evaluations[].metric":{"state":"value_present","value":{"kind":"text","text":"Dollar value of confirmed identity-theft returns selected only by each system."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0635-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Dollar value of confirmed identity-theft returns selected only by each system.","inferred":false}},"evaluations[].reproduction_materials":{"state":"value_present","value":{"kind":"text","text":"None public."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0635-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"None public.","inferred":false}},"evaluations[].setting_sample":{"state":"value_present","value":{"kind":"text","text":"700,560 confirmed identity-theft returns, 2015 filing season."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0635-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"700,560 confirmed identity-theft returns, 2015 filing season.","inferred":false}},"evaluations[].uncertainty_limitations":{"state":"value_present","value":{"kind":"text","text":"Figures come from IRS data; selections by manual processes are not counted; parts of the report are redacted; no comparison with no screening."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0635-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Figures come from IRS data; selections by manual processes are not counted; parts of the report are redacted; no comparison with no screening.","inferred":false}},"external_references[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"paired_incident":{"database":"AI Incident Database","identifier":"101","title":"Dutch Families Wrongfully Accused of Tax Fraud Due to Discriminatory Algorithm","url":"https://incidentdatabase.ai/cite/101","group":"organizational","basis":"Both involve a tax authority that uses a fraud-risk model to screen claims for payment, and in the incident the tax authority itself was harmed when its model falsely accused thousands of families of fraud."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["paired_incident"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"follow_up[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"later":{"status":"persisted","date":"2018-07-24","note":"GAO relays IRS figures showing that the program kept stopping invalid refunds through November 2017.","quote":"IRS reported that between January 2015 and November 2017, RRP prevented the issuance of more than $6.51 billion in invalid refunds.","url":"https://www.gao.gov/assets/gao-18-544.pdf"}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["later"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"human_oversight":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"id":{"state":"value_present","value":{"kind":"record_fields","fields":{"id":"AIBD-0635"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["id"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"infrastructure_dependencies[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiaries":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiary_count":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].benefit_type":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_type":"Improved Task Accuracy"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].claim":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"The Internal Revenue Service built the Return Review Program to screen individual returns that claim refunds before the refunds are paid and to replace its older Electronic Fraud Detection System. The Treasury Inspector General for Tax Administration writes that both systems use predictive models developed with proprietary machine-learning algorithms as their main detection method. In an audit published on 25 September 2017, the Inspector General examined data from the 2015 filing season, in which the IRS's fraud detection systems together identified 700,560 confirmed identity-theft returns worth $3.92 billion. Selections made only by the program's identity-theft models totaled $1.88 billion, or 47.8% of that amount, while selections made only by the older system's models totaled $60 million, or 1.5%. The Inspector General judged that the IRS made a reasonable decision to retire the older identity-theft models, which it did for the 2016 filing season. In a report dated 24 July 2018, the Government Accountability Office relays IRS figures showing that the program prevented more than $6.51 billion in invalid refunds between January 2015 and November 2017, including about $4.4 billion in the 2017 filing season. The dollar figures come from IRS data. The audit counts unique selections by each system and does not compare the program with no screening, and the program also uses business rules besides its models.","benefit_type":"Improved Task Accuracy"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].distribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].duration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].id":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].stakeholder_group":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"The program's identity-theft models were the only system to select $1.88 billion of the $3.92 billion in confirmed identity-theft refunds in the 2015 filing season."},{"group":"societal","depth":2,"reason":"The IRS reports that the program prevented more than $6.51 billion in invalid refunds from January 2015 to November 2017, as relayed by GAO."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].status":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_status":"reported"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_status"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].tradeoffs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].unexpected":{"state":"value_present","value":{"kind":"record_fields","fields":{"unexpected":false,"purpose":"The Return Review Program was built to detect identity-theft and other refund fraud before the IRS issues refunds. The benefit is that purpose."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["unexpected","purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"places[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"location":"United States (national)","developer_country":"United States","deployer_country":"United States"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["location","developer_country","deployer_country"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"purpose":{"state":"value_present","value":{"kind":"record_fields","fields":{"purpose":"The Return Review Program was built to detect identity-theft and other refund fraud before the IRS issues refunds. The benefit is that purpose."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"record_dates":{"state":"value_present","value":{"kind":"record_fields","fields":{"date_observed":"2015-01-01","date_basis":"event","date_precision":"year","revisions":[{"date":"2026-10-29","change":"Added the case.","source":"https://www.tigta.gov/sites/default/files/reports/2022-02/201720080fr.pdf","editor":"AIBD"},{"date":"2026-10-29","change":"Linked a comparable incident in the AI Incident Database (incident 101).","source":"https://incidentdatabase.ai/cite/101","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["date_observed","date_basis","date_precision","revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"replication.attempts[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"replication":{"produced_by":["The IRS screened every individual return that claimed a refund before paying it.","The models used prior-year returns and third-party information returns such as W-2 forms.","IRS staff adjusted the models and the fraud score settings to balance fraud caught against legitimate refunds delayed."],"must_hold":["Prior-year returns and third-party data must be available when new returns are scored.","Score settings must keep false detections low enough that legitimate taxpayers are not burdened."],"echoes":[{"date":"2017-01-01","setting":"IRS 2017 filing season, in which GAO relays IRS figures that the program prevented about $4.4 billion in invalid refunds","outcome":"recurred","source":"https://www.gao.gov/assets/gao-18-544.pdf"}]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"replication.conditions":{"state":"value_present","value":{"kind":"record_fields","fields":{"replication":{"produced_by":["The IRS screened every individual return that claimed a refund before paying it.","The models used prior-year returns and third-party information returns such as W-2 forms.","IRS staff adjusted the models and the fraud score settings to balance fraud caught against legitimate refunds delayed."],"must_hold":["Prior-year returns and third-party data must be available when new returns are scored.","Score settings must keep false detections low enough that legitimate taxpayers are not burdened."],"echoes":[{"date":"2017-01-01","setting":"IRS 2017 filing season, in which GAO relays IRS figures that the program prevented about $4.4 billion in invalid refunds","outcome":"recurred","source":"https://www.gao.gov/assets/gao-18-544.pdf"}]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"revision_history[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"revisions":[{"date":"2026-10-29","change":"Added the case.","source":"https://www.tigta.gov/sites/default/files/reports/2022-02/201720080fr.pdf","editor":"AIBD"},{"date":"2026-10-29","change":"Linked a comparable incident in the AI Incident Database (incident 101).","source":"https://incidentdatabase.ai/cite/101","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sector":{"state":"value_present","value":{"kind":"record_fields","fields":{"domain":"Tax administration"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["domain"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].access_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].citation_locator":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"report","citation":"Treasury Inspector General for Tax Administration. The Return Review Program Increases Fraud Detection; However, Full Retirement of the Electronic Fraud Detection System Will Be Delayed. Reference Number 2017-20-080, 25 September 2017.","title":"The Return Review Program Increases Fraud Detection; However, Full Retirement of the Electronic Fraud Detection System Will Be Delayed","url":"https://www.tigta.gov/sites/default/files/reports/2022-02/201720080fr.pdf","date":"2017-09-25","written_by":"independent","quote":"In comparison, the RRP IDT model selections that were not selected by any other fraud detection system totaled $1.88 billion, accounting for 47.8 percent of the total $3.92 billion IRS IDT revenue protection."},{"kind":"report","citation":"U.S. Government Accountability Office. Tax Fraud and Noncompliance: IRS Could Further Leverage the Return Review Program to Strengthen Tax Enforcement. GAO-18-544, July 2018.","title":"Tax Fraud and Noncompliance: IRS Could Further Leverage the Return Review Program to Strengthen Tax Enforcement","url":"https://www.gao.gov/assets/gao-18-544.pdf","date":"2018-07-24","written_by":"independent","quote":"IRS reported that between January 2015 and November 2017, RRP prevented the issuance of more than $6.51 billion in invalid refunds."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].id":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-0635-source-0001","AIBD-0635-source-0002"],"role":"Existing evidence items, in their original order"},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"sources[].language":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"report","citation":"Treasury Inspector General for Tax Administration. The Return Review Program Increases Fraud Detection; However, Full Retirement of the Electronic Fraud Detection System Will Be Delayed. Reference Number 2017-20-080, 25 September 2017.","title":"The Return Review Program Increases Fraud Detection; However, Full Retirement of the Electronic Fraud Detection System Will Be Delayed","url":"https://www.tigta.gov/sites/default/files/reports/2022-02/201720080fr.pdf","date":"2017-09-25","written_by":"independent","quote":"In comparison, the RRP IDT model selections that were not selected by any other fraud detection system totaled $1.88 billion, accounting for 47.8 percent of the total $3.92 billion IRS IDT revenue protection."},{"kind":"report","citation":"U.S. Government Accountability Office. Tax Fraud and Noncompliance: IRS Could Further Leverage the Return Review Program to Strengthen Tax Enforcement. GAO-18-544, July 2018.","title":"Tax Fraud and Noncompliance: IRS Could Further Leverage the Return Review Program to Strengthen Tax Enforcement","url":"https://www.gao.gov/assets/gao-18-544.pdf","date":"2018-07-24","written_by":"independent","quote":"IRS reported that between January 2015 and November 2017, RRP prevented the issuance of more than $6.51 billion in invalid refunds."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].upstream_sources":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"strategic_assessments[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"strategic assessments"},"summary":{"state":"value_present","value":{"kind":"record_fields","fields":{"summary":"TIGTA found that IRS Return Review Program identity-theft models alone selected $1.88 billion, or 47.8%, of $3.92 billion in confirmed identity-theft refunds in 2015.","account":"The Internal Revenue Service built the Return Review Program to screen individual returns that claim refunds before the refunds are paid and to replace its older Electronic Fraud Detection System. The Treasury Inspector General for Tax Administration writes that both systems use predictive models developed with proprietary machine-learning algorithms as their main detection method. In an audit published on 25 September 2017, the Inspector General examined data from the 2015 filing season, in which the IRS's fraud detection systems together identified 700,560 confirmed identity-theft returns worth $3.92 billion. Selections made only by the program's identity-theft models totaled $1.88 billion, or 47.8% of that amount, while selections made only by the older system's models totaled $60 million, or 1.5%. The Inspector General judged that the IRS made a reasonable decision to retire the older identity-theft models, which it did for the 2016 filing season. In a report dated 24 July 2018, the Government Accountability Office relays IRS figures showing that the program prevented more than $6.51 billion in invalid refunds between January 2015 and November 2017, including about $4.4 billion in the 2017 filing season. The dollar figures come from IRS data. The audit counts unique selections by each system and does not compare the program with no screening, and the program also uses business rules besides its models."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["summary","account"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].autonomy_permissions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].component_role":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].deployment_configuration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].inputs_outputs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_name":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].product_name":{"state":"value_present","value":{"kind":"record_fields","fields":{"system":"IRS Return Review Program (RRP) identity-theft models"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["system"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].provider":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"taxonomy_crosswalks[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"title":{"state":"value_present","value":{"kind":"record_fields","fields":{"title":"TIGTA finds that the IRS Return Review Program's identity-theft models alone selected $1.88 billion of confirmed identity-theft refunds in the 2015 filing season","headline":"*Return Review Program* catches refund fraud that older filters missed"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["title","headline"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"version":{"state":"value_present","value":{"kind":"identifiers","identifiers":["02a05a45be81b8a5fcdb1d81bf0b05d9705cff43bcd6428a9c89b8e1d06e6366"],"role":"SHA-256 of the complete public case and its assessments"},"provenance":{"kind":"curatorial","basis":"SHA-256 of the public case, its assessments, and its displayed answers and notes, excluding its own identifier and the derived coverage summary.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"workflow":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"The Internal Revenue Service built the Return Review Program to screen individual returns that claim refunds before the refunds are paid and to replace its older Electronic Fraud Detection System. The Treasury Inspector General for Tax Administration writes that both systems use predictive models developed with proprietary machine-learning algorithms as their main detection method. In an audit published on 25 September 2017, the Inspector General examined data from the 2015 filing season, in which the IRS's fraud detection systems together identified 700,560 confirmed identity-theft returns worth $3.92 billion. Selections made only by the program's identity-theft models totaled $1.88 billion, or 47.8% of that amount, while selections made only by the older system's models totaled $60 million, or 1.5%. The Inspector General judged that the IRS made a reasonable decision to retire the older identity-theft models, which it did for the 2016 filing season. In a report dated 24 July 2018, the Government Accountability Office relays IRS figures showing that the program prevented more than $6.51 billion in invalid refunds between January 2015 and November 2017, including about $4.4 billion in the 2017 filing season. The dollar figures come from IRS data. The audit counts unique selections by each system and does not compare the program with no screening, and the program also uses business rules besides its models.","replication":{"produced_by":["The IRS screened every individual return that claimed a refund before paying it.","The models used prior-year returns and third-party information returns such as W-2 forms.","IRS staff adjusted the models and the fraud score settings to balance fraud caught against legitimate refunds delayed."],"must_hold":["Prior-year returns and third-party data must be available when new returns are scored.","Score settings must keep false detections low enough that legitimate taxpayers are not burdened."],"echoes":[{"date":"2017-01-01","setting":"IRS 2017 filing season, in which GAO relays IRS figures that the program prevented about $4.4 billion in invalid refunds","outcome":"recurred","source":"https://www.gao.gov/assets/gao-18-544.pdf"}]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"}},"summary":{"total":91,"addressed":91,"known":44,"reviewed":15,"reviewed_complete":15,"reviewed_partial":0,"record_mapped":29,"review_not_assessed":47,"not_assessed":0,"complete":false,"by_state":{"value_present":44,"not_reported":0,"unresolved":47,"not_assessed":0,"not_applicable":0,"withheld":0,"source_unavailable":0}}},"presentation_version":"b38c5b4d025673a2d6e729b142744139d0782201eb53ed623a32d51037bc22c1"}