What happened
After the Tax Court allowed predictive coding in 2014, the parties agreed on a protocol. The Commissioner coded two random sets of 1,000 documents and twelve smaller sets to train and test the model. He chose a recall rate of 95%. The taxpayers ran the model over about 406,000 documents and ran a second algorithm to find privileged material. In January and March 2016 they delivered a production set of about 180,000 documents. The Commissioner kept 5,796 and returned the rest. He then moved to compel documents that his Boolean search had hit and the model had left out. On 13 July 2016 Judge Ronald L. Buch denied the motion. He held that the inquiry was reasonable and that the response was complete. He assumed for argument that the model's recall fell short. He noted that the 95% recall setting brought a precision of 3%. He called the belief that human review is perfect a myth.
Sources
-
Court document, by authors with no tie to the system.
Petitioners made a reasonable inquiry in responding to the Commissioner's discovery demands when they used predictive coding to produce any documents that the algorithm determined was responsive, and petitioners' response was complete when they produced those documents.
Details
AI system
- Workflow
-
The Commissioner coded the seed sets that trained the model and chose its recall rate. The parties agreed on a protocol and filed 16 joint status reports. A second algorithm screened the production set for privileged material.
Follow-up
- Conditions for replication
-
The requesting party takes part in training the model. The court measures the response by a reasonable inquiry and does not demand perfection.
Entry details
- Sector
-
Legal services / e-discovery.
- Purpose
-
A vendor that the order does not name built the predictive coding software to find responsive documents, with a model trained on the requesting party's coding.
Not yet checked: model, release, provider, role, configuration, inputs and outputs, autonomy, human oversight, AI contribution, beneficiaries, reach, distribution and access, duration, every measurement question, every money question, ownership and control, dependencies, cross-border connections, geopolitical implications, replication attempts, availability, institutional decision, grounds, action and result, success criterion, every evidence question, observed or intended, alternative assessments, related entries, other identifiers, category mappings.
Cite this record
Peter H. Vartanian. (2026). Tax Court finds a predictive coding response to the IRS complete (AIBD-0834). Artificial Intelligence Benefits Database, edition of 31 October 2026. https://petervartanian.xyz/aibd/records/AIBD-0834.html (archived at https://doi.org/10.7910/DVN/AADSII, version 4.0)