Benefits
- Benefit
-
Selected $1.88 billion, or 47.8%, of $3.92 billion in confirmed identity-theft refunds
Awaiting review
- Outcome identifiers
-
Unresolved
Not reviewedUnresolved
- Category
-
Improved Task Accuracy
Awaiting review
- Status
-
Reported benefit
Awaiting review
- Beneficiary groups
-
Organizations. Society.
Awaiting review
- Beneficiaries
-
Unresolved
Not reviewedUnresolved
- Reach
-
Unresolved
Not reviewedUnresolved
- Distribution and access
-
Unresolved
Not reviewedUnresolved
- Duration
-
Unresolved
Not reviewedUnresolved
- Expected?
-
Yes
Awaiting review
- Costs and harms
-
Unresolved
Not reviewedUnresolved
AI system
- System
-
IRS Return Review Program (RRP) identity-theft models
Awaiting review
- Model
-
This is an explicit unresolved assignment: the sources on record do not settle this question.
Not reviewedUnresolved
- Release
-
Unresolved
Not reviewedUnresolved
- Provider
-
Unresolved
Not reviewedUnresolved
- Role
-
Unresolved
Not reviewedUnresolved
- Configuration
-
Unresolved
Not reviewedUnresolved
- Inputs and outputs
-
Unresolved
Not reviewedUnresolved
- Autonomy
-
Unresolved
Not reviewedUnresolved
- Workflow
-
The IRS screened every individual return that claimed a refund before paying it. The models used prior-year returns and third-party information returns such as W-2 forms. IRS staff adjusted the models and the fraud score settings to balance fraud caught against legitimate refunds delayed.
Awaiting review
- Human oversight
-
Unresolved
Not reviewedUnresolved
- Relation to AI
-
The audit compares which fraud detection system selected each confirmed return. It does not compare the program with no screening, and the program also uses business rules besides its machine-learning models.
Awaiting review
- AI contribution
-
Unresolved
Not reviewedUnresolved
Measurement
- Studies
-
Other. [1]
Fully reviewed: supporting details for Studies
Review date: 29 October 2026.
Sources: [1]
- Study design
-
Other. [1]
Fully reviewed: supporting details for Study design
Review date: 29 October 2026.
Sources: [1]
- Measurement
-
Audit of IRS filing-season data on which detection systems selected each confirmed identity-theft return. [1]
Fully reviewed: supporting details for Measurement
Review date: 29 October 2026.
Sources: [1]
- Setting and sample
-
700,560 confirmed identity-theft returns, 2015 filing season. [1]
Fully reviewed: supporting details for Setting and sample
Review date: 29 October 2026.
Sources: [1]
- Measures
-
Dollar value of confirmed identity-theft returns selected only by each system. [1]
Fully reviewed: supporting details for Measures
Review date: 29 October 2026.
Sources: [1]
- Comparison
-
Electronic Fraud Detection System and the Dependent Database. [1]
Fully reviewed: supporting details for Comparison
Review date: 29 October 2026.
Sources: [1]
- Results
-
$1.88 billion (47.8%) unique to the Return Review Program models against $60 million (1.5%) unique to the older system. [1]
Fully reviewed: supporting details for Results
Review date: 29 October 2026.
Sources: [1]
- Additional value from AI
-
Not isolated from the program's business rules. [1]
Fully reviewed: supporting details for Additional value from AI
Review date: 29 October 2026.
Sources: [1]
- Limitations
-
Figures come from IRS data; selections by manual processes are not counted; parts of the report are redacted; no comparison with no screening. [1]
Fully reviewed: supporting details for Limitations
Review date: 29 October 2026.
Sources: [1]
Money
- Kind of value
-
The kind of monetary value is unclear.
Not reviewedUnresolved
- Amount
-
The monetary value is not established.
Not reviewedUnresolved
- Currency and prices
-
The currency and price basis are unclear.
Not reviewedUnresolved
- Period and unit
-
The period and unit of the monetary estimate are unclear.
Not reviewedUnresolved
- Realization
-
Whether the monetary value was realized is unclear.
Not reviewedUnresolved
- Recipients and payers
-
The recipients and payers are unclear.
Not reviewedUnresolved
- Gross or net
-
Whether the monetary figure includes costs is unclear.
Not reviewedUnresolved
- Included costs
-
The costs included in the estimate are unclear.
Not reviewedUnresolved
- Comparison
-
The financial comparison is not established.
Not reviewedUnresolved
- Method
-
The method used to calculate monetary value is unclear.
Not reviewedUnresolved
- Return and payback
-
The return on investment or payback period is not established.
Not reviewedUnresolved
People and places
- Participants
-
Developer:
Internal Revenue Service (Information Technology organization and Office of Research, Applied Analytics, and Statistics). Operator: Internal Revenue Service.Awaiting review
- Jurisdictions
-
Benefit:
United States (national). Developer: United States. Operator: United States.Awaiting review
- Ownership and control
-
Unresolved
Not reviewedUnresolved
- Dependencies
-
Unresolved
Not reviewedUnresolved
- Cross-border connections
-
Unresolved
Not reviewedUnresolved
- Geopolitical implications
-
The geopolitical implications are unclear.
Not reviewedUnresolved
Follow-up
- Conditions for replication
-
Prior-year returns and third-party data must be available when new returns are scored. Score settings must keep false detections low enough that legitimate taxpayers are not burdened.
Awaiting review
- Replication attempts
-
1 follow-up entry. Benefit recurred: 1.
Awaiting review
- Later findings
-
24 July 2018. GAO relays IRS figures showing that the program kept stopping invalid refunds through November 2017.
Awaiting review
- Availability
-
Unresolved
Not reviewedUnresolved
- Institutional decision
-
A relevant institutional decision has not been established.
Not reviewedUnresolved
- Grounds
-
The grounds for an institutional decision are unclear.
Not reviewedUnresolved
- Action and result
-
The result of an institutional decision is not established.
Not reviewedUnresolved
- Success criterion
-
The criterion for a successful institutional decision is unclear.
Not reviewedUnresolved
Evidence
- Source numbers
-
AIBD-0635-source-0001–0002
Awaiting review
- Citations
-
Numbered references below.
Awaiting review
- Language
-
Unresolved
Not reviewedUnresolved
- Source relationships
-
Independent
Awaiting review
- Original evidence
-
Unresolved
Not reviewedUnresolved
- Access and versions
-
Unresolved
Not reviewedUnresolved
- Supported findings
-
Unresolved
Not reviewedUnresolved
- Evidence links
-
Unresolved
Not reviewedUnresolved
- Evidence strength
-
Independently studied. Sources: two independent reports. The cited evidence includes no finding by a regulator or court. [1] [2]
- Missing evidence
-
Unresolved
Not reviewedUnresolved
Grades
- Assessed benefits
-
Organizations. Society.
Awaiting review
- Depth
-
Organizations: 2. Society: 2.
Awaiting review
- Reasons
-
Organizations: The program's identity-theft models were the only system to select $1.88 billion of the $3.92 billion in confirmed identity-theft refunds in the 2015 filing season. Society: The IRS reports that the program prevented more than $6.51 billion in invalid refunds from January 2015 to November 2017, as relayed by GAO.
Awaiting review
- Observed or intended
-
Unresolved
Not reviewedUnresolved
- Rubric
-
Unresolved
Not reviewedUnresolved
- Assessor and date
-
Unresolved
Not reviewedUnresolved
- Alternative assessments
-
Unresolved
Not reviewedUnresolved
- Fill
-
Independently studied
Awaiting review
Entry details
- Kind
-
The case describes a program or activity. The record describes a national screening program assessed by auditors. [1]
Fully reviewed: supporting details for Kind
Review date: 29 October 2026.
Sources: [1]
- Related entries
-
Both involve a tax authority that uses a fraud-risk model to screen claims for payment, and in the incident the tax authority itself was harmed when its model falsely accused thousands of families of fraud.
Fully reviewed: supporting details for Related entries
Compared with the incidents in the AI Incident Database snapshot of 26 October 2026; a second reading confirmed that the incident concerns the same kind of system and stakeholder.
Sources: incidentdatabase.ai
- Field
-
Tax administration.
Awaiting review
- Purpose
-
The Return Review Program was built to detect identity-theft and other refund fraud before the IRS issues refunds. The benefit is that purpose.
Awaiting review
- Dates
-
Observed: 2015. Updated: 29 October 2026.
Awaiting review
- Stage
-
The system was in routine use. The program screens refund returns nationally in every filing season. [1]
Fully reviewed: supporting details for Stage
Review date: 29 October 2026.
Sources: [1]
- Version
-
SHA-256: 02a05a45be81…
Awaiting review: supporting details for Version
SHA-256 of the public case, its assessments, and its displayed answers and notes, excluding its own identifier and the derived coverage summary.
References
- Other identifiers
-
AI Incident Database: 101.
Awaiting review
- Category mappings
-
Unresolved
Not reviewedUnresolved
- Citable summary
-
Unresolved
Not reviewedUnresolved
- Revisions
-
2 revisions. Latest: 29 October 2026.
Awaiting review
Sources
-
Report Independent author
Treasury Inspector General for Tax Administration. The Return Review Program Increases Fraud Detection; However, Full Retirement of the Electronic Fraud Detection System Will Be Delayed. Reference Number 2017-20-080, 25 September 2017.
In comparison, the RRP IDT model selections that were not selected by any other fraud detection system totaled $1.88 billion, accounting for 47.8 percent of the total $3.92 billion IRS IDT revenue protection.
-
Report Independent author
U.S. Government Accountability Office. Tax Fraud and Noncompliance: IRS Could Further Leverage the Return Review Program to Strengthen Tax Enforcement. GAO-18-544, July 2018.
IRS reported that between January 2015 and November 2017, RRP prevented the issuance of more than $6.51 billion in invalid refunds.