Most staff found Insight useful for reviewing hearing decisions

About 74% of 302 staff who answered a survey by SSA's Inspector General found Insight useful in reviewing hearing decisions, and 71 users reported problems with its flags.

  • 1 an organization

    About 74% of the staff who answered the survey found Insight useful and valuable in reviewing decisions. The Inspector General could not determine whether it improved decisions in the hearing offices.

Depth runs from 1, a convenience or time saved, to 5, a life or a livelihood preserved.

Status
Reported and intended benefits.
Evidence

foundstudiedreportedclaimed

An independent audit report supports the benefit.

Compared with

Cases of appellate analysts who did not use Insight, in the appellate office's own study. Measured.

Without the system
21.5 percent of non-Insight users' cases returned for quality issues
With it
12.6 percent for Insight users' cases
The system’s part
Not isolated. The Inspector General's own evidence is a survey of staff opinion. The comparison of users with non-users comes from the appellate office's own study, which the Inspector General relays without testing. The Inspector General describes Insight as a program that uses natural language processing and artificial intelligence technologies.
Limits
Not yet checked.
Costs and harms
Not yet checked.
Later
Not yet checked.

Download data Cite Suggest a correction

Benefit type
Improved Task Accuracy
System
Insight (SSA quality-review software for disability hearing decisions)
Made by
Staff of SSA's Office of Appellate Operations (SSA's Office of Analytics and Improvements took over the program after its release)
Used by
SSA Office of Appellate Operations and Office of Hearings Operations
Where
United States (national)
Stage
Expanding
Date
Kind
Aggregate result
Broad benefit
Evaluating Information to Determine Compliance with Standards, Analyzing Data or Information
Collections
Scrutiny and findings
Last revised
31 October 2026

Sources

  1. Audit report, by authors with no tie to the system.

    Social Security Administration, Office of the Inspector General. The Social Security Administration's Use of Insight Software to Identify Potential Anomalies in Hearing Decisions. Audit report A-12-18-50353, 19 April 2019 (archived copy of the report at oig-files.ssa.gov).

    In addition, about 74 percent agreed the program was useful and valuable when they were reviewing decisions, while about 26 percent disagreed (see Figure 2).

Details

AI system

Workflow

Insight extracted information from the text of each hearing decision and flagged possible quality issues. Each flag linked users to the relevant policies so that they could check the issue.

Follow-up

Conditions for replication

Users must check each flag against policy before acting on it. The flags must be accurate and consistent with the agency's disability policies.

Entry details

Sector

Social security (disability appeals).

Purpose

Staff in SSA's Office of Appellate Operations built Insight to flag possible errors of policy and internal consistency in decisions from disability hearings.

Not yet checked: model, release, provider, role, configuration, inputs and outputs, autonomy, human oversight, AI contribution, beneficiaries, reach, distribution and access, duration, every measurement question, every money question, ownership and control, dependencies, cross-border connections, geopolitical implications, replication attempts, availability, institutional decision, grounds, action and result, success criterion, every evidence question, observed or intended, alternative assessments, related entries, other identifiers, category mappings.

Cite this record

Peter H. Vartanian. (2026). Most staff found Insight useful for reviewing hearing decisions (AIBD-0674). Artificial Intelligence Benefits Database, edition of 31 October 2026. https://petervartanian.xyz/aibd/records/AIBD-0674.html (archived at https://doi.org/10.7910/DVN/AADSII, version 4.0)

BibTeX Data Field definitions